Pavel Panchekha

By

Share under CC-BY-SA.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

Dedicated Crossword Practice

In my last post on crossword science, I looked at whether people get better at crosswords over time. They don't.11 Or, they do, for the first few months; but not after that, on average. But that just looked at doing one crossword every day. What if you did dedicated, sustained crossword practice? That's today's post.

Joe's weekend

Joe has always talked of getting good at crosswords. When my model provided him with a single numeric skill estimate, Joe decided the time had come for some sustained practice. So, over one weekend, he completed roughly four hundred crosswords. Note that his average crossword time is about a minute, so that's something like six hours straight of crosswords!

Since the crossword bot tracks submission time, I isolated his period of dedicated practice and wanted to know whether he got better over that period. To do so, I looked at trends in his residual: the prediction errors my model gets for each submission. Since my model does not assume improvement over time, the prediction errors should trend negative if Joe gets better.

crossbot-joe-dates.png

Figure 1: Lower is better

In this plot, the blue points represent residuals, the red line is a smoothing of the blue points, and the x-axis just records all the crosswords Joe has ever done, in order. The period of sustained practice starts at 85 and ends at 495.

Analyzing the data

You can see that there is no overall trend; Joe finishes the practice period roughly where he started. There is a period from roughly 225 to 275 where Joe is noticeably worse. That corresponds to a practice session on the morning of 5 August (from 9:00 to 14:00). Perhaps Joe was tired that morning,22 The data report he'd stayed up until 3am solving crosswords the prior night, so he'd had about six hours of sleep. and it affected his crossword times. It's a good sign that that's something we can see out of the data.

Looking just at the distribution of blue points, it also seems that Joe's performance got more variable over time. Toward the left hand side of the plot, the blue dots are roughly evenly distributed around 0, with a standard deviation of around 1—basically normal, in other words. Toward the right, the points mostly cluster below 0 (suggesting improvement), plus many extreme points as high as 2 or 3. So Joe seems to have gotten better at the median puzzle, but gotten worse at the rare extreme puzzle.

I also wanted to check whether Joe gets better or worse over the course of a practice session. To find out, I looked through the data by hand and picked out nine periods of sustained practice (of 10 to 112 crosswords long) and replotted the points counting from the start of the practice session:

crossbot-joe-sessions.png

Here the points have different opacity because there can be multiple points for one x value; in fact, up to nine, one for each practice session. As the smoothing curve shows, there's not a clear trend over the course of a session.33 With plots like these, there's a chance of a "composition effect", because different x values have different sessions contributing to them (since sessions have different lengths). I repeated the analysis once per session, but still didn't see much of a coherent trend.

This might be a good time, by the way, to bring in Joe's thoughts. He feels he did get significantly better thanks to his dedicated practice,44 This fits with the fact that he over-performed more often, but his under-performances stayed bad enough for no average movement. but that that occurred between, not during, practice sessions. So not seeing any movement in the plot above is not crazy.

Conclusions

It seems that on average, Joe did not get better by doing many crosswords over the course of a weekend. Maybe you won't, either. However, it does seem that Joe got faster at the crosswords that he solved quickly, and it seems that getting a full night's sleep can have a big impact (30% faster!).

I speculate that to get better at crosswords, it's not enough to just do a lot of them (beyond the beginner period). It helps to sleep well and do them when you're well awake, and it might help to regularly read the news, consume more classic American art, and expand one's vocabulary.

1
Or, they do, for the first few months; but not after that, on average.
2
The data report he'd stayed up until 3am solving crosswords the prior night, so he'd had about six hours of sleep.
3
With plots like these, there's a chance of a "composition effect", because different x values have different sessions contributing to them (since sessions have different lengths). I repeated the analysis once per session, but still didn't see much of a coherent trend.
4
This fits with the fact that he over-performed more often, but his under-performances stayed bad enough for no average movement.