Nearly 300 readers have now contributed Embedded Figures Test results, so it’s a good time to review them.
The Neuroscience page describes lab tests which show how cognitive flexibility is adversely affected by even slight stress, and the Implications for Software Engineers page discusses the need for exactly this kind of flexibility if we are to juxtapose multiple considerations in our minds and be good at programming - or any other kind of creative work.
The problems with most tests of cognitive flexibility are that they are costly to administer and they are not repeatable per subject. Once a subject has seen a given test it won’t be a puzzle again, so we can’t apply it once, alter the conditions and try it again. To apply this stuff in industrial contexts, some way to objectively identify high value initiatives quickly and efficiently would be really good.
The EFT resembles some known tests of cognitive flexibility, it is repeatable per individual and can be automated. So if it does vary with stress then it could be very useful. Of course, the experiment on this blog is only indicative - because the self-assessment of stress is quite simplistic and the test conditions are not closely controlled it can’t produce definitive answers. But if it looks promising, it might give professional psychologists an incentive to look at it in more controlled conditions.
Within these limits, the results so far look quite promising. As before, you can download the raw data
eft19jun2008.txt and the analysis program:
EftAnalyzer.cpp. To compile the program, download and build wxWidgets, and simply cut and paste the source code into the sample program
minimal.cpp. It should build and run on Windows, Linux and MacOs X with no problem. The graphs here are screenshots from MacOs X.
There are 299 contributions in the database. 268 of those pass validation as sensible. It’s still a sample strongly biased towards male geeks - 255 males and 13 females, 180 geekly occupations. While there’s no evidence that people who feel nauseous when bored have EFT results which are different to those who don’t, I’m amazed that 60% of respondents experience the nausea effect! This may be something worth looking into further. (I asked about it because nausea is a side effect reported by people taking dopamine raising drugs for Parkinson’s disease, dopamine is raised by stress, and I suspect that groups become habituated to raising dopamine by subjecting themselves to miserable, boring meetings.) Here are the numbers:
Most respondents scored between 2000 and 3999 milliseconds per figure, but there is a “long tail”:
There’s also a good spread of self-assessed occupational stress, slightly biased towards the unstressed side of the survey questions. (The positive or negative wording of the questions is mixed to discourage people from just clicking down a vertical line. A strong unstressed response gets 2 “chill points”, and strong stressed response gets -2 “chill points”. Weaker responses get 1 or -1 “chill points”.):
There’s a small correlation between taking longer to see the figures and increasing age. The graph plots the mean score and standard deviation for each decade wide age band. It also calculates Spearman’s rho - a statistical measure of the correlation between ranked values. (We have to be careful because it’s only good for some kinds of correlations - see this page by William D. Richards for the details.):
A similar plot of stressors by average score shows no correlation at all - and this is the meat of the experiment! But look at the standard deviation - the spread gets narrower as stress reduces:
A scattergram shows what is happening. Here I’ve colour coded the best, middle and worst thirds in each stress band. We see that the best and middle bands seem to get noticeably worse around -7 chill points, but the worst third spreads out a lot as stress increases. This fits with a known wide variation in individual responses to stress. Apart from one outlier, we could fit a 45 degree line against the worst scores as stress increases!
Given the uncontrolled tests and crude self-assessment of occupational stress, I think this scattergram shows the EFT is worth further study as an automated and repeatable test of groups’ occupational stress levels, if not individuals. However, its value is by no means established yet!
The final graph shows the frequency distribution for the small proportion of respondents who reported use of prescription or non-prescription drugs. For what it’s worth, none of them seem to make any difference to EFT results, with the exception of the 14 respondents who are taking SSRI anti-depressants. Even there, the use of SSRIs doesn’t equate to a slow score - the curve’s just a bit flatter. Since there does seem to be some correlation between occupational stress and the spread of results we could reasonably assume that we aren’t looking at the effects of the SSRIs so much as the circumstances which have led to the prescriptions:
It’s a shame that the sample is still so biased towards male geeks. A sample that is more representative of the general population would be interesting. So please try to encourage female and/or non-geekly people to try the test and contribute their results!