Embedded Figures Test - Tracking Confirmed

I’ve received a very encouraging email from an academic psychologist who is kind enough to have patience with my unusual thread of research, confirming that the distinct signature that has emerged in the embedded figures test results is also found in other tests of cognitive flexibility, which fulfills the strong requirement described in my previous posting Embedded Figures Test - Distinct Signature Emerges. Following the etiquette of personal correspondence I’ll just quote the meat of it:

That’s really interesting! We are actually resubmitting a paper finding the exact same best performers/worst performers dichotomy with stress in a set of college students. Our studies have focused on young volunteers, generally. Also, I like the spatial aspect of your task. Whenever we tried spatial tasks, I suspected our lack of findings are due to power issues. Your evidence supports this. The one obvious weakness is, as you point out, the lack of control of testing conditions, but so long as these do not differ systematically in some way, then you can overcome it with statistical power.

That’s very encouraging! I don’t suppose there’s a preprint on arXiv or similar is there?…

Not yet- but remind me and I’ll be glad to send one once I know I have a final version!

I shall certainly be looking out for the preprint, and will post details as soon as possible!

So not only does the EFT track other tests of cognitive flexibility as modulated by stress, so long as it is applied to big enough groups it also reproduces the different responses seen in different levels of performance. It also predicts a similar signature where the subjects’ age modulates the score but amongst the worst performing third, while stress affects the best performing third. It looks like I’m actually ahead of the game on this one!

A useful exercise now will be to go and do some stats on the raw data and try to make a statement about how big is big enough. How many people need to be in a work group before we can apply the EFT, take the best performing third, and predict that they are under occupational stress if their mean score is > 2.5s?

At this point it would be worth saying something about why I selected the EFT as a potential test of cognitive flexibility that would be repeatable per individual and automatable. This will put some of the important ideas I’m keen to convey in this blog into a concrete form.

The test is normally thought of as measuring a quality called field (in)dependence. The idea is that some people tend to see the parts and so can easily spot the one part they are looking for, while others see the whole and so find it hard to spot a specific figure. This idea then extends by some sort of loose analogy to poor scorers being “sociable” because they are somehow more conscious of the group of humans around them, better scorers being “misfits” - and furthermore to better scorers being more able at natural sciences because they are natural reductionists. In the semi-scientific world of recruitment psychological testing, better scores are supposed to identify misfits who paradoxically have good leadership potential.

I first encountered the test in the recruitment context, and it seemed to me that when I did it, I looked at the whole and allowed the part I wanted to pop out at me. It was a test of my visual/spacial parallel processing capacity, which is strongly related to the capacity for juxtaposition of complex elements which I emphasise as important in complex design work, and hence cognitive flexibility. As to all the literature discussing field (in)dependence, I didn’t believe a word of it, and the proof is now available. If the quality of field (in)dependence was a personal trait as believed, then scores would be constant per individual, and not modulated by occupational stress as I have shown they are.

So why are poorer scorers thought of (and indeed observed to be) more social, and better scorers observed to be better at natural sciences? I believe the answer lies in the background psychosocial stress maintained in most cultures, which has a neurochemically addictive component. Many people are (without realizing it) hooked on performing low level stressful social exchanges, monitoring each others’ compliance to arbitrary social mores, worrying about it, and so maintaining a “stress economy”. People who participate in this are seen to be more social because non-participants are unconsciously resented for not exchanging “stress hits” with others. Unfortunately participation in the stress economy renders them less able to do complex work, and in extreme cases can lead to perverse group behaviour. Groups suffering from this effect are not aware of it, because the changes to brain chemistry are similar to those found in (for example) cocaine addiction - which is known to produce unawareness of problems and unjustified feelings of complacency. Perhaps the poorest performing third who are not made any worse by occupational stress are already weakened in terms of cognitive flexibility because the stress they don’t get at work, they find somewhere else! When it comes to natural sciences, the belief that reductionism is all is mainly found amongst people who don’t do them! In reality nature is doing everything at once, and the skill which psychologist Simon Baron-Cohen calls systematizing involves seeing the key relationships in multiple phenomena occurring in juxtaposition.

The important idea then, to take away from all of this, is that being good at complex work is neither reductionistic nor a property fixed at birth. It’s something anyone can get better at, by learning how to reduce their psychosocial stress to the point where they can use the full faculties they were born with. When it comes to groups, the amount of stress (bad) and the quality of debate (good) within a group is amplified by re-reflection within the group. Initiatives to reduce background psychosocial stress such as I describe in the Introduction can bring runaway benefits to a groups even as they make only incremental benefits to each member of the group. Beware though of The Dreaded Jungian Backlash - the buildup of unconscious resentment directed against whole groups who drop out of the stress economy. The best counter to this effect is to make everyone involved - particularly the senior management of work organizations - explicitly conscious of the effect. The Backlash only presents a danger when people do not understand the unconscious origins of such resentment directed against teams enjoying improved performance, and which (as I have described) can involve clerical and janitorial workers who are in no way competing with the improving team.

Neuroscience, All Postings, Stress Addiction

Dirty Little Secrets - Response to Grady Booch

Skynet 5 has been fully operational for less than a month, yet already one of our finest minds is chained up in a bunker somewhere, and has been replaced by one of those Terminator skin job thingies.

It would be hard to overstate the contribution of Grady Booch to the young discipline of software engineering. From his own practical experience, his ever-questing mind has observed and crystalized deep wisdom into a collection of writings and products which have benefitted us all. These include the object oriented bits of the Unified Modelling Language (UML), the Rational Rose round-trip tool and his masterwork, Object-Oriented Analysis and Design with Applications. In his definitive book The C++ Programming Language, Bjarne Stroustrup says Booch is the only person worth reading on object oriented design.

So I was astonished to see an interview, Software’s Dirty Little Secret on the Scientific American website. The interview seems wrong-headed in so many ways, yet so typical of the usual unhelpful dross that we see (way below the standard of normal Booch), that I’ve been thinking about it for most of the last few days.

The essence of the interview is hardly novel. The Booch 1000 says that unlike other disciplines, software engineering is a mess because there are no rules for its practitioners to follow. If there was a rulebook, software would not be a mess. That this situation exists is a dirty little secret kept by the software engineers. Oh the shame of it all!

Yes, We Have No Bananas

First off, the claim that:

In other disciplines, engineering in particular, there exist treatises on architecture. This is not the current case in software, which has evolved organically over only the past few decades

is absurd. When I Googled software engineering I got over 54 million hits! We certainly aren’t wanting for rules, proceduralisms, methodologies. Every one of them has been touted as the solution, and still it seems some people can’t understand that there is No Silver Bullet.

Perhaps the Booch 1000 is a member of the this time for sure! school of thought.

No-One Knows Everything

We don’t actually do so badly compared to other disciplines. With thousands of years of prior art behind them, your local builder can construct staircases with good repeatability. We can do the same thing with graphical user interfaces. After all, when Apple implemented the iPhone there wasn’t much doubt that the GUI could be made to work! The builders can do roofing and we can do database engines. In fact, considering some of the debacles I’ve witnessed I’d say we’re rather better at doing database engines than builders do roofing. Builders do commodity guttering, and we do on-the-fly data compression the same way. The fatuous belief that other disciplines are better at this kind of thing comes in part from the fact that builders have to spend time repeating jobs they have done many times before, while often we just link a library. If we typed in the code every time, we’d quickly see the true comparison. (There are some firms which exploit the “business world’s” limited understanding of this reality by typing in the same billing system over and over again, but they are one of the perversities of the field.)

Linking a library is like your local builder following the building regulations. But the building regs only go so far, covering small, localized issues. Outside the limited contexts where the rules can describe the situation exhaustively, the builders can easily get into trouble.

The London Millennium Bridge (image from the Wikipedia article) was a high visibility show-off project using a novel design. As soon as it was opened to the public it revealed a design flaw leading to unpleasant horizontal swaying. Significant modifications were required before people could cross in comfort.

We know about resonant effects. Military formations know they must break step when crossing bridges or they might shake them to bits. Everyone has seen the film of the Tacoma Narrows Bridge:

Beyond bridges, advanced work has been done swinging massive weights around to protect tall buildings from resonant effects during earthquakes. Even so, no-one anticipated the high visibility cock-up in London. As software engineers often do in our young discipline, the designers were trying something just a little new. There was no prior art to guide them, no simple procedures, scripts, building regs for them to follow. And in this case, they ran into trouble. As we often do. The proposition that if only we had a book of rules like the builders do, we’d be perfect like them, is thus shown to be doubly bogus.

Everyone Suffers Feature Interaction

But it doesn’t stop there. Even doing simple things that builders have been doing over and over again for thousands of years, and following the building regs at every point, builders can still be caught out by something we often see too - feature interaction.

These photos below show two reciprocal diagonal views across a pleasant, recently built housing estate that I know. There is a central green space, with houses on two sides:

The other two sides have a stone wall which defines the space and walls the green off from other houses, themselves built around their own greens:

As you can see, the developers went to a lot of effort to create a pleasant and livable space where children can play safely. The houses are well constructed to provide a good quality of life for all. The only problem is a strange acoustic effect. For some reason, the square is like a whispering gallery! The conversations of people walking near to the wall can be clearly heard on the other side of the square, as if they were standing right outside the window. So little energy is lost by whatever focussing effect is in play, that the voices can be heard through the double glazing! At first it’s a very un-nerving experience. It sounds like there’s someone standing there and talking in a muffled way such that their words can’t be discerned, but there’s no-one there! Each bit of the system is in full compliance with building regs, the overall design is a best effort, but this extraordinary interaction of the compliant features happened nevertheless.

Just like what happens when we try new things, this kind of thing cannot be anticipated in building works. In fact if I’d seen a photo of the wall and not known what actually happens, I’d have guessed that it would have anechoic properties:

This is actually an area where software engineers are making better progress - at least there is hope that at some point automated support for CSP theory will reduce our exposure to some kinds of feature interaction in ways that other disciplines cannot hope to achieve.

Then There’s the Cock-Ups Too…

I’ve still not finished my escalation of examples to refute the Booch 1000’s double error, claiming that if we had rules like other disciplines, we’d be predictable and repeatable just like them. The most common cases aren’t like the examples I gave above. Commonly we have rules, just like builders have rules, but at an organizational or regulatory level we choose to ignore them, or follow them in an incredibly stupid, counterproductive way!

The picture below is taken from Google Earth’s view of Stevenage, in south-east England:

There are five roads leading in and out of a roundabout (or traffic circle as amazed Americans call them). If you look closely you will see another network of smaller grey roads threaded adjacent to and flowing under the five wider ones. These are foot and cycle paths. They must have looked wonderful in the artificially constrained context of the pretty watercolours shown to the Planning Department of the local Council. Imagine the atmosphere of self-congratulation as those involved performed their administrative rituals and assured themselves of perfect compliance.

The trouble is, none of those foot and cycle paths are illuminated, and they run like culverts below the level of the road. At dusk they become a muggers’ paradise. Only people with a death wish would venture there. Worse, because of all the wonderful sunken paths, there are no footpaths running by the roadsides. Attempting to walk along the roads exposes people to multiple lanes of high speed traffic. So after dark (which happens by 16:00 hours in winter) there is no pedestrian access to huge areas of the town. It’s like living under a curfew, and as a result Stevenage is a cultural and social desert. And yet… we know all about constructing spaces where people can move about, spontaneously interact, and maintain self-sustaining communities. Spaces where businesses thrive, arts are practiced and a sense of ownership ensures that vandalism doesn’t occur.

The problem is not about having rules. The perpetrators of Stevenage have lots of rules, yet they still accomplished an entirely avoidable cock-up, of the kind which we as software engineers should be interested in, because this is the kind of preventable cock-up which we also commit with monotonous regularity. Even if we allow for the greater call for novelty in software engineering and the amount of feature interaction we are vulnerable to because of the Kolmogorov complexity of our requirements, we still see far too many of these preventable messes.

So unlike the same old, same old proceduralists I deny that we are any worse than other disciplines at achieving repeatable and reliable performance. We have nothing to be ashamed of in this respect, although we are as vulnerable as the others to endemic problems within the culture, and the great promise of our work exposes those problems more.

Don’t Forget - Success is Possible

Architects do sometimes achieve some spectacular successes. Norman Foster’s Millau Viaduct (image from Wikipedia article) is a transcendent blend of form and function that renders further comment superfluous:

I. M. Pei’s Louvre Pyramids (images again from Wikipedia) are an equally masterful blend of an aesthetic which perfectly mirrors the light precision of their context while celebrating the full richness of both eras:

While at the same time magnificently performing the task of filling a vast public area, unimagined when the original Louvre was constructed, with light:

If you haven’t done so yet, go and see it. It’s worth it. As you gaze on these wonders, remember that the vehicular traffic bourne by the one, and the mass of wondering humans hosted by the other, must be measured in the megatonnes. These are very hard working systems.

Getting Proactive

Rather than hanging our heads in shame before the other self-deluded cock-up producers in our culture, and humbly accepting whatever fatuous behavioural prescriptions they hand down to us, we should be taking the lead in sorting out our common problem.

Human culture is a complex multi-layered phenomenon which is capable of great things, but is subject to intermittent faults leading to ridiculous failure. It has something wrong with it and needs debugging. I’m not the only person to notice this - the physicist David Bohm said,

… thought is a system. That system not only includes thought and feelings, but it includes the state of the body; it includes the whole of society - as thought is passing back and forth between people in a process by which thought evolved from ancient times.

He discusses his proposed plan to look for the error in his book, Thought as a System

No rules could have led to the fabulous accomplishments I discussed above, nor yet differentiated between them and the monumental cock-up in Stevenage. Fortunately architecture benefits from its history. Students have to learn plenty of materials science and mechanics, but it is still taught as an Arts subject. The aesthetical judgement - the ability to juxtapose the elements of a problem and composition and see beauty or its absence - of the students is developed. This aesthetical sensibility is shared by great programmers (who use words like elegance to describe it). So I argue that progress in this area will most likely be made by looking at the cognitive state of the practitioners, not the shelfware they execute. It’s about how we perceive and understand the world, and we exercise this kind of paradigmatic flexibility every time we coax a messy set of business requirements into a robust and implementable system design.

We are the systems thinkers, the debuggers, the stars of applied cognitive flexibility. It’s high time we stepped out of our specialization, became fully eclectic in our considerations, and made an effort to sort this stuff out in its own terms - not by just regurgitating the failed excuses already found in the culture.

In the Introduction I propose a picture where psychosocial stress - raised by petty nagging about trivialities - acts to shut down our aesthetical sensibility by pushing us into a kind of focussed attention where working memory is reduced and we lose the juxtapositional ability. We know this happens in lab studies of stress and working memory. As a result we become less able to respond to the circumstances before us and more reliant on heuristics - rules. Beyond that, we know that stress has a similar effect on brain chemistry as addictive drugs, and battery farmed animals subjected to stress become addicted to it. So groups of people can become addicted to nagging each other about petty rules, such that they can only think in terms of rules, and their resulting reduced performance increases the stress. Rule obsession is the problem, not the solution, and it leads to a spiral of decline which ultimately destroys nations and firms.

So we keep on hearing the same old rubbish, always presented as the next big idea, and never mind that it never works. To provide an illustration of this, I’m going to quote a posting to The Joel on Software Discussion Group. The posting describes how things get when groups have free rein to indulge their stress addiction. My proposal is that the behaviours described are attractive not because they are pleasant for the people involved, but because they are stressful. So everyone knows how dreadful it is, but no-one is willing to fix it. Remember that you pay for this twice - once when you pay the victims to behave like this every day, and again when your society suffers the opportunity cost of the kind of maladministration that results. The cognitive limitations have become so widespread that this kind of attitude has become paradigmattic, so that’s it’s now hard to even question why it happens:

- The head of IT has been in government for 27+ years. She is the one making purchasing decisions and setting strategic direction. She does not own a computer or have an external email address. She does not buy on-line, and she has the web monitor set so tight you can forget about using the Internet. No IM. No webmail. Then to ensure you cannot possibly reach the outside, no non-government equipment permitted in the building without a specific exception. This will be your organizational leader as the rule - not the exception.

- Change is glacial. Two weeks I did nothing because they could not allocate a machine for me to access the network, until I had a badge. To get the machine, a form needed to be signed by my boss, his boss, her boss, and his boss. My boss walked it around, and was told that was inappropriate, and it still took 4 hours. Then it went to the help desk who took 9 business days to process and deliver the equipment. Deliver in this case meant coming to my desk and providing a user id and password to a machine sitting there. To install software requires you to have an exception form filed. Another round of 4 signatures in my department, one from security, one from the helpdesk and one from a person who no one can explain except they need to sign the exception form - six days. Complain or bother anyone about a request and it will disappear. Petty is an understatement.

- It is true that no one gets fired. But worse, making decisions is career limiting. The solution is to ensure you never do anything of risk. What you want is to be tied into a project you can claim some responsibility with, if it goes well, and disavow any relationship if it fails.

- No _one_ makes any decisions because there is safety in numbers - very large numbers. All decisions are by committee and expect it to take weeks. They spent 12 weeks, with a minimum of six people in nearly 20 meetings discussing a database key. One key. It is an extreme example but that it can happen says it all.

- If you take any type of leadership position and do anything that employees do not like - they submit a grievance. Why? Because while one is pending, they cannot do anything to you, include request work, or deny automatic or scheduled promotions in grade. A 25 year developer? here has had a grievance on her various leaders, continuously for over 12 years. She is proud of it. “They don’t tell me what to do!”

- Write off leaving government. You can always leave right? Wrong. Would you hire someone from an environment that fosters the behavior above? With very few exceptions, spend more than a year or two as a “gov-e” and you can kiss the private sector good bye. Who would want this behavior and if it took you more than a year to figure out you should quit, that says volumes.

We often see people falling into this swamp. I think it happens to politicians, who often seem sincere when they enter office - and those close to them insist that they are. It doesn’t take long though, before their daily exposure to bureaucrats fixing chronic internally generated psychosocial stress gets to them and they become trapped in the same prescriptive, proceduralist, unimaginative state as the zombie they ousted. And so it goes on.

To conclude this roundup of thoughts triggered by the Grady Booch interview, I was struck by his talking about a dirty little secret. He sounds like a self-hating software engineer!

I have sometimes wondered if there isn’t actually a dirty little secret that is kept by members of groups suffering chronic stress addiction. I suggest that people in this state get a hit off stressful situations and behaviours. They are cognitively impaired and the calibre of debate within the group is poor as a result, but perhaps this provides opportunities to raise stress in a particularly unpleasant way. When a person acts in a manner contradictory to their conscience, this itself is stressful. Might it be that such people are tempted to knowingly engage in wrongful, stupid or damaging behaviours because they know they are performing wrongdoing? The story Pensioner, 81, ordered to remove flat cap as he has a quiet pint - because it’s a security risk is perhaps a mild example of such willful and perverse misinterpretation of guidelines. A kind of willful misinterpretation which is becoming all too common at the moment, as stress addiction becomes deeper and more widespread, and more once bright minds sink into the swamp.

All Postings, Stress Addiction, Programming

The Benefits of Low Level Lighting

Few people choose to fill their homes with bright fluorescent lighting. Perhaps in the kitchen or utility room where bright lighting is helpful for the kinds of things they do in those rooms, but not in the living room, study or bedroom, where they wish to feel comfortable, relaxed and secure.

On the pages Neuroscience and Implications for Software Engineers I describe how feeling comfortable, relaxed and secure - rather than stressed and anxious - enables us to use the faculties we need to be good at programming.

So logically, we should light programming shops in the same way that we light our studies at home. I believe that commercial organizations drift into doing exactly the wrong thing because raising stress and anxiety has an insidious and previously unsuspected allure. The neurochemical response to stress - raised dopamine and norepinephrine - is similar to the response to addictive drugs!

I’ve recently been working in a shop that’s good in several ways, and one of their good practices is low level lighting in several areas where programmers have their desks. Some of the old-timers have told me that the lighting is an accident of history, but I don’t care how it got that way. It’s still worth sharing!

It’s a bit tricky sharing a lighting level - darned modern cameras :-) So in the two photos below, notice how strong the illumination seems where the lighting shines directly on the walls. And remember I took them to make a point, not as works of art. So they’re “warts and all”, and therefore I’ve taken care to avoid catching anything identifiable in them. The areas used for greeting customers are much smarter:

Compare with these two shots of the kitchen area, where natural daylight can be seen coming through the windows:

The low level lighting brings several interacting benefits. The sense of cosiness, similar to my living room at home, is certainly there. Beyond that, there’s an improved sense of privacy. This is more of a perceptual illusion than a real effect, because we are all as visible to each other as in any open plan environment. But it feels more private, and that’s what matters for reducing stress and anxiety and so enabling juxtapositional thinking.

There’s also an effect similar to being in a library. People respect the sense of quiet and cosiness. They keep their voices down, and tend to go elsewhere to use their phones. There’s a nice kitchen they can go in for a chat. Not all noises are the same, and we know that noise containing intelligible data (even if it’s not intelligent) is the worst kind for breaking our concentration and preventing us seeing complex pictures. So if we must suffer open plan, anything which encourages people to keep their voices down is good. The office depicted in the photos above has a babble annoyance level which is nearly as good as a two person office. It’s remarkable. I would never have guessed this effect would be so pronounced if I hadn’t had the chance to observe it happening.

Then there’s eye strain. We all spend our days looking at screens, which emit their own light. They don’t need external illumination like pieces of paper do. In fact, the more environmental light there is, the harder we find it to see the screens. The blatant extreme case of that is glare when the angles are wrong, but flicker has a more serious effect on its victims, wearing them down over time. The flicker from the kind of cheapskate tubes found in many commercial shops is bad enough, but when it beats with displays which are refreshing at nearly the same frequency it can be (almost) visible to most people. There are plenty of people who think that computer displays emit noxious rays which give them headaches when actually it’s that beat frequency flickering away at the edge of their awareness that’s doing them in. Simply explain what’s happening to them and they stop being so annoyed by it - try it! Obviously, the less fluorescent light there is in an office, the less these effects will occur.

The whole flicker thing has another side to it. Many of the best programmers are the sort of people who are classified as being on the “autistic spectrum”. Now I’m very suspicious about such classifications. As I explain on the page Other Applications, I believe we can understand the name-calling as unconscious resentment by those trapped in the psychosocial stress/dopamine economy towards that don’t participate and exchange fixes. The greater sensitivity of the non-participants can be understood as normal sensitivity, not blunted by excess dopamine constantly sloshing around their brains. But whatever the cause, many of the best programmers can see the 50 or 60 Hz flicker of fluorescent lighting without any monitors beating away nearly in sync, and it’s extremely annoying to them. Reducing it improves their ability to concentrate. Simple.

The final benefit I’ll identify is economic. Many organizations operate acres of floor space, all lit with tubes. Take half of the tubes away and the lighting bill is cut in half.

I’ve given several good reasons for reducing the light levels of programming shops. Some readers may notice there’s an argument I’ve not made. There’s plenty of anecdotal accounts of full spectrum lighting improving the performance of schoolchildren. The trouble with this stuff is that it is very anecdotal - the more I look for substantial original research on this, the vaguer and more associated with interested parties it becomes. The remaining research doesn’t do a good job of excluding multiple sources of variance, or “confounds” as they’re called in the jargon. The report Full-spectrum fluorescent lighting: a review of its effects on physiology and health by McColl and Veitch, funded by the National Research Council of Canada does a good job of exploring the lack of solid evidence in this area. So although you might choose to experiment with full spectrum lighting, bear in mind that it is more expensive, and the evidence for its benefits is not yet strong.

Experimenting with reduced light levels is a different matter. It has a negative cost. So try it. Persuade your boss to take away some of the tubes (don’t make things too dark - make sure illumination is enough to be safe for example), and see if your working environment becomes cosier, more private, quieter and more relaxing. See if your ability to concentrate improves, and over a release cycle see if that has any effect on the accuracy of your estimates, the necessary sufficiency of your code, and your bug reports.

Neuroscience, All Postings, ADHD, Stress Addiction, Programming

Process Degradation and Ontological Drift

In the Introduction I describe how the cognitive effects of stress - particularly as studied in neuroscience labs and military contexts - degrade the very faculties we need to be good at programming. I also propose that since the neurochemical effects of stress - particularly the release of dopamine and norepenephrine - are so similar to the effects of addictive drugs, we might expect whole groups to become addicted to maintaining a locally comfortable level of psychosocial stress. Because the members of the group don’t realize they are stuck in an addictive loop and are experiencing both cognitive impairment and gratification of their addictive need, they have little chance of gaining insight and instead will tend to rationalize their stress-raising behaviours as inevitable or desirable, but won’t be able to say why.

No part of this is terribly radical, but taken together it certainly is. For example, it implies that to really understand what is going on in a workaday engineering situation, we have to look outside the context and think about the whole culture in an unusual way. Like the old puzzle about connecting 9 dots with 4 connected and unbroken lines:

We have to “think outside the box” - but we have to really expand our contextual awareness and not just say we are doing so. The trouble is, we need access to the very faculties that stress takes away to be able to think juxtapositionally and tell the difference!

So in this post I’m going to look at a real programming task, and talk about how people often think about such tasks. I’ll look at the opportunities to do a good and interesting job that we miss all too often, how our perception of work can change to become a mockery of itself and how process can amplify this change. I’ll also discuss how this change distorts our appreciation of our tools, and touch (just a little) on the profound social implications outside engineering work.

The Task

I recently had to apply a bug fix across a large codebase (around 1.1 MLOCS in the relevant part) implementing some complex business logic for several large telecoms customers. Some colleagues had successfully tracked down a problem which cropped up rarely, during an even rarer failover event in an underlying Sybase database. (A failover happens when one copy of a database becomes unavailable for any reason, so Sybase transparently switches to a hot backup.) Like so many others, the problem was rooted in the object-relational impedance mismatch. To summarize, the object and relational models don’t play well together. In this case we had some classes to handle the database operations and nicely encapsulate their state. The biggest lump of state was the result sets that some operations could return. Errors were handled by C++ exceptions, but when an exception was thrown, it could leave a result set waiting in the database connection encapsulated by the classes. It was the waiting result sets which blocked the failovers and prevented them performing as expected. The solution was to make sure that every operation - and every catch - purged any result set that might be waiting. Once understood, the fix was easily stated. The problem simply came from the number of cases which had to be considered.

Fortunately the code was well-layered. Database operations were rigourously contained within a specific set of classes, clearly separated from higher level business logic. Even better, they were all wrapped in SQLAPI++, which provides a (somewhat) platform independent collection of classes to wrap database operations. So every possible cause of problems could be found by looking for uses of the SQLAPI++ method SACommand::Execute(). So as we first understood the fix, I had to make sure that each such was embedded in a pattern:


         // Do stuff
catch(SAException &e)
         // Do stuff

Of course, it was important to get it right in every case. The edit would involve checking many cases, each with their own variations, spread over (I later determined) 133 different source files. This must be done without introducing more errors, including source code corruption caused by simple finger trouble while doing such a large task!

Perceiving The Task

Just as I was getting interested in the challenge of this source manipulation problem, someone made a comment recognizing that it was boring. For a moment I was amazed - it really hadn’t occurred to me to see it as a boring problem rather than a difficult one! I realized that the atypical perception was on my side, and thinking that observation through led to some interesting realizations.

Firstly, I’ve been doing this kind of thing for about 30 years now. I know that there’s no magic fix. No matter what technologies we use to help us work at convenient levels of abstraction and sem-automate the tasks, if we are going to own and maintain large and sophisticated programs, we will always come up against this kind of problem. Dealing with them effectively is an intellectual challenge for grownups.

Secondly, I’ve never been seduced by the rationalization of laziness that makes people speak of “mere code”. This is the idea that highly intelligent people are supposed to occupy themselves with grander things, and find “mere code” to be beneath them. I’ve always been suspicious of this. Code is the product. It doesn’t matter if your code undergoes mechanical translation, perhaps from some IDL or something to C, thence to assembly language and finally to opcodes. In that case the IDL is your product. Dismissing it as “mere code” and wandering off to do something else is like a cook who sneers at “mere food” and prefers to criticize the decor of the kitchen instead. It’s thoroughly misguided. Coping with large codebases requires skill, experience and a challenging mixture of rigour and ability to hold temporary partial hypotheses - and keep in mind which is which! It’s difficult, learned on the job, and cannot be performed in knee-jerk fashion after skimming “Codebases for Dummies” in a lunchbreak. The simple fact is, the presence of at least a proportion of people who have mastered these skills is necessary to the success of shops doing real, grownup work.

Thirdly, I’m well aware of the danger of boredom, and recognize it as one of the things which must be managed in order to do a good job - not just accepted as an unavoidable misery of life, leading to certain and inevitable failure. Effective attention can be maintained in two regimes, with a danger area in the middle:

It’s easy to maintain careful attention on an interesting task, and it’s also easy to perform a strictly mechanical task (like lawn mowing or ironing) particularly with an mp3 player available. It’s the tedious and repetitive tasks which require some attention, such that we cannot disengage, yet neither is there anything to keep us engaged, which are the problem. So faced with a task which initially looks like it sits up on the peak of the boredom curve, I know that the way to do it well is to decompose it into two tasks which are on the ends. That means investigating and thinking about it until I can structure the remaining parts in a very mechanical way. Then I can pick some matching music and get a rhythm going with my fingers!

Performing The Task

It was pretty easy to find all instances of string Execute in the source tree and examine them. I quickly found there were more instances of Execute than just calls to the method of class SACommand. On the other hand, my exhaustive examination of all Executes quickly showed that all the ones that mattered declared an SACommand in the same file. (I admit I’m simplifying here to keep the posting on topic - I have a point to make.)

It was then very easy to constrain my scans to the relevant files, by using a little shell one-liner:

$ vi `find . -print | while read fname
> do
> if test -f "$fname"
> then
> if grep SACommand "$fname | grep Execute > /dev/null
> then
> echo "$fname"
> fi
> fi
> done`

If you don’t know shell, that says:

Edit a list of files comprising:
List all files below current working directory. For each file:
If it’s a regular file (not a directory or anything else):
If the file contains SACommand and Execute:
Include it.

The thing is, if we are willing to use little shell one-liners like that, we can very easily customize searches of any complexity we need, as we need them. That’s why many of us stick with the “old fashioned” command line tools, even in this era of eye-candy encrusted Integrated Development Environments. Those tools can do some very clever things with a click of the mouse (and tools like Eclipse are very clever). But they are limited to the functionality imagined by their designers, rather than what the user needs at any moment. The teaching that A Jedi needs only his light saber contains a deep truth, even though it has come in for some ridicule in recent times:

(The two twerps in that video sound like they come from near my own home town. It’s not Nottingham, but maybe Mansfield. English English accents really are that fine-grained.)

You probably see where I’m going with this: I reckon surrendering to the inevitability of boredom (and so making errors) is part of a pattern which includes becoming dependent on (and so constrained by) the functionality available in prerolled tools. It’s about lowered aspirations, increased passivity and reactiveness. These perceptual and behavioural shifts are in keeping with the effects of stress, as I discuss on the page Neuroscience.

While I was writing this posting, I had an idea and tried a small experiment. I used the same one-liner, on the same codebase, but on Apple’s MacOs X instead of Sun’s Solaris. It gagged, complaining:

VIM - Vi IMproved 6.2 (2003 Jun 1, compiled Mar  1 2006 20:29:11)
Too many edit arguments: "-"
More info with: "vim -h"

I guess that’s why Sun can still sell stuff to grownups!

By the time I’d finished my exhaustive scan I’d divided all use cases into simple cases as above, and a couple of variant patterns where it was appropriate to put the result set clearing snippet elsewhere in an inheritance hierarchy containing the Execute. I got my corrected snippet ready on the clipboard (I was using a Windows XP box running PuTTY as a terminal), and I thought I was ready…


I quickly discovered a problem. There were some formatting problems which had crept into the codebase over time. The indentation wasn’t very rigourous, and even more annoying, there was a mixture of spaces and tabs used to create the indentation. This didn’t matter when editing a few lines, but the unpredictable editor behaviour when scaling to many lines pushed the job back into the error-prone bit of the boredom curve. I had to delay my edit and prepare the source files first.

That was a two stage process. First I fed all the relevant files through bcpp, the classic C++ beautifier. (Another little shell one-liner accomplished that.) Then I went through all the files to neaten them up. The beautifier is good, but it can’t make aesthetical judgements about stuff that should be lined up to improve readability - like this:

buffer << "Name: "    << name
       << " Quest: "  << quest
       << " Colour: " << colour;

Once those problems were out of the way the noise level was lowered, and I was able to crystalize my awareness of another problem that my subconscious had been screeching about, but I didn’t know what it was.

If the Execute() / isResultSet() / FetchNext() sequence threw an SAException, I still had to purge the result set. But doing that involved further calls to isResultSet() and FetchNext(), which could themselves throw SAExceptions - and probably would if any part of the database connection was confused! It was important to surround the purge within the catch{} in it’s own try{} block and catch{} these SAExceptions too, or the uncaught exception could cause the server to crash ungracefully!

This demonstrates the importance of reducing the noise level to the point where we can hear the voice of our own experience - and why Paul Graham says most commercial programmers never reach the state of awareness where they can do good work.

So far the task had taken about 5 working days - and I hadn’t made a single keystroke that actually implemented the edit I needed to perform. But then I did the edit, as an uninterrupted mechanical process which took less than a day - and I got it right.

The Code Review

How do I know I got it right? After all, the problem is usually not so much being correct, as it is being able to prove we are correct. In this case the biggest potential source of error was the manual straightening up process after the automated beautification. There were plenty of opportunities for mis-edits - unintended modifications - in there. Usually the CVS diff command will find the edits, but in this case the beautification made the diff too big to handle.

Fortunately I’d pulled a second copy of the source tree before I started. (CVS diff compares with the version the local copies may have diverged from, but pulling the a separate copy of the right version of everything would be tricky.) The CVS diff could at least give me a list of the files I’d touched, if I used grep to find all the lines that started with the string RCS, and edited them to produce a simple list of files.

I reckoned it would be much easier if I could consolidate all the files into a single pair of directories - before and after. That would be possible if the file names were all different (ignoring the directory names). I could check that easily by using each line of the file list as an argument of basename and doing a unique sort of the list:

$ cat CvsDiffFileList.txt | while read fname
> do
> basename "$fname"
> done > BaseNames.txt
$ wc -l BaseNames.txt
$ sort -u BaseNames.txt > UniqueNames.txt
$ wc -l UniqueNames.txt

Oh dear. There was a name collision. Let’s find out about it:

$ diff BaseNames.txt UniqueNames.txt
< main.cpp
$ grep main CvsDiffFileList.txt

OK. The name collisions were just mains for running test stubs. I didn’t care about them, so I deleted them from the file list, and knew I could copy the before and after versions of all the touched files into a pair of directories. So I did that with another little one-liner:

$ mkdir Before
$ mkdir After
$ cat CvsDiffFileList.txt | while read fname
> do
> cp src/"$fname" Before
> cp src/"$fname" After
> done

Then I removed all the tabs and spaces from both sets of copies. That removed any differences caused by changing tabs to spaces, the indentation, or differences in if(x) compared to if ( x ) style, so the remaining differences would be my substantial edits or finger trouble:

$ cd StrippedBefore
$ for fname in *
> do
> cat "$fname" | tr -d " " | tr -d "t" > fred
> mv fred "$fname"
> done

The result was 15,000 lines of diff - much better than the 100,000 I started with, and mainly consisting of blank lines and easily recognizable patterns of intentional edits. I then looked through the diff to make sure it was possible, checking for unmatched business logic type lines, before asking two colleagues to do likewise. A much simpler process also enabled us to review the intentional changes by looking for the Executes, like I had done originally.

Process Degradation

There’s a pattern to what I did, and we’ve all learned about it ad nauseum. It’s W. Edwards Deming’s Plan-Do-Check-Act Cycle. However, there’s a major difference between what I just described and the way “process” is normally represented. The difference is ownership.

I believe in the Deming Cycle. It works. Through reflection we become aware of what we are trying to achieve. We ask how we can do this as reliably and efficiently as possible. We form a plan, and when we do what the plan indicates we ask if it’s working or not, and correct as necessary. We find a way to check our work as an integral part of the cycle (like I pulled that second copy before I started), and when we’ve finished we act on what we’ve learned. In this case I realized the whole process formed a wonderful model of something deeper that I wanted to discuss.

Unfortunately the Deming Cycle is rarely applied as I did in the tale above. There, I was responsible for each step, I corrected my process as necessary, and I even discovered that I had to recurse, performing a sub-process within my outer Do stage.

Imagine how such a process would degrade under the cognitive distortions due to stress, described on the page Implications for Software Engineers. People under stress:

  • Screen out peripheral stimuli.
  • Make decisions based on heuristics.
  • Suffer from performance rigidity or narrow thinking.
  • Lose their ability to analyze complicated situations and manipulate information.

I was analyzing (reasonably) complicated situations, and using my shell like a kind of mental exoskeleton to manipulate information all the time! If I couldn’t do that, how could I know what to do? I’d have to become a robot myself, and execute a program that I’d got from somewhere else. If that’s not making decisions based on heuristics I don’t know what is! Of course, the motions I’d be going through wouldn’t be very effective, but I wouldn’t be able to compare my performance with a gold standard that I might imagine for myself based on the question, “How do I know?”, because of my narrow thinking. I wouldn’t even notice my mistakes, because I’d be screening out peripheral stimuli.

I’d be lost. Instead of being able to rely on my own capacity to operate a Deming Cycle and move forward with confidence, I’d be “piggy in the middle” - responsible for my failures, yet ill-equipped to avoid them. It would all be very stressful. I would obviously be keen to embrace a culture which says that results don’t matter, only compliance. Then when the failure occurred, I could pass the buck. Of course, I would always have to be on the lookout for others trying to pass their bucks to me, and I would have to be clever in my evasiveness and avoidance of responsibility.

The processes I could use would not be ones I cooked up myself, always following the meta-process of the Deming Cycle. Instead they would be behavioural prescriptions collected into shelfware, and micropoliced in a petty and point-missing stressful fashion, as my non-compliances were smelled out. All the stress would lock me into a cognitively reduced state, and if I got hooked on the stress chemistry, the raised dopamine and norepinephrine, the whole business would start to develop a self-evident rightness. If I suffered this decline in the company of others, we’d develop language and a local mythology to celebrate our dysfunctional state!

All this, of course, is exactly what we see as Quality programs degrade into reactive pettiness and individuals lose their problem solving abilities. Deming’s big thing was devolving process improvement to the people nearest to the work, but in the degraded version process is handed down from the organization. We have an entire belief system predicated on the belief that:

The quality of a system is highly influenced by the quality of the process used to acquire, develop, and maintain it.

Rather than the skills, imagination and dedication of the people applying it!

So we must always be very careful of process. When applied with knowledge and understanding, we get the growth of awareness, efficiency and personal fulfillment that some practitioners of the Deming method report. But with a supervening cultural tendency to stress addiction always in play, it can easily degrade into the exact opposite of what Deming intended. A spiral of decline where individuals become progressively less capable and empowered while the process becomes more externalized and disconnected from the work at hand.

Ontological Drift

After a while, groups suffering from a spiral of psychosocial stress start to behave in very peculiar ways. Being unable to trust their own good senses to even select a useful script, but unable to proceed without one, they resort to grabbing any old script and following it, no matter how silly the result. The deeper they get into trouble, the worse their problems become.

This way of looking at things provides a simple explanation for some very odd things. Consider the hilarious BBC story of the man who was told to remove his shirt at Heathrow because it bore a cartoon image of a Transformer which has a stylized gun for an arm - Gun T-shirt ‘was a security risk’:

A man wearing a T-shirt depicting a cartoon character holding a gun was stopped from boarding a flight by the security at Heathrow’s Terminal 5.

Brad Jayakody, from Bayswater, central London, said he was “stumped” at the objection to his Transformers T-shirt.

Mr Jayakody said he had to change before boarding as security officers objected to the gun, held by the cartoon character.

Airport operator BAA said it was investigating the incident.

Mr Jayakody said the incident happened a few weeks ago, when he was challenged by an official during a pre-flight security check.

“He says, ‘we won’t be able to let you through because your T-shirt has got a gun on it’,” Mr Jayakody said.

“I was like, ‘What are you talking about?’.

“[The official’s] supervisor comes over and goes ’sorry we can’t let you through and you’ve a gun on your T-shirt’,” he said.

Mr Jayakody said he had to strip and change his T-shirt there before he was allowed to board his flight.

“I was just looking for someone with a bit of common sense,” he said.

That’s stress addiction induced ontological drift for you. Of course, it isn’t so funny when whole cultures go that way. “Mr. K, the committee will see you now…”

Neuroscience, All Postings, Stress Addiction, Programming

Ah… Love ‘Em!

On the pages Why No-One’s Noticed This Before, The Dreaded Jungian Backlash and Logical Effects I describe how the effects of spending too much time in focussed attention conspire to conceal themselves. This includes the simple truth that we don’t notice what we don’t notice, our culture’s accomodation of focussed attention by leading us to describe our work in reactive, prescriptive terms, and reduced expectations of - and reliance on - self-consistency. Most strangely I argue that groups of people trapped in focussed attention are liable to escape contradiction when things go wrong by deluding themselves. All of this fits with studies of groups under stress. We know that stress makes groups more reliant on heuristics, less analytic, less aware of context. There are elegant “choice blindness” and “conformity” experiments which reveal the surprising way that ordinary people will backfill their own memories with fantasy.

It is not my purpose to dwell on these issues. What matters is that it can happen, it’s a spiral of decline as each fantasy leads to another, and it always reduces efficiency and quality. We need to be aware this kind of thing can happen, and ensure we maintain conditions to prevent it. It’s like finding rotten timbers - the exact growth pattern doesn’t matter, we just need to get rid of it. Confusion can reach a point where we need to rule off, take a deep breath and start again. Over-analyzing each mess is of little value - unless we are studying messes!

To demonstrate how crazy things can get and provide a little entertainment, I’m going to analyze a story that’s been in the news this week. It makes no sense at all until we think about it in these terms, and then what is happening is crystal clear. Do remember that this is a good textbook example - and textbook examples are inherently misleading. Real life is rarely this clear cut. Anyway, on with the yarn…

It comes from the Chicago Sun-Times, a gem entitled Voters are told pen had ‘invisible ink’. Ahem…

Voters are told pen had ‘invisible ink’

BY ANNIE SWEENEY Staff Reporter/asweeney@suntimes.com

When it comes to election shenanigans, Chicago has been accused of just about everything.

But invisible ink?

Twenty voters at a Far North Side precinct who found their ink pens not working were told by election judges not to worry.

It’s invisible ink, officials said. The scanner will count it.

But their votes weren’t recorded after all.

“Part of me was thinking it does sound stupid enough to be true,” said Amy Carlton, who had serious doubts but went ahead and voted anyway.

As it turns out, Carlton was one of 20 voters at the precinct who were given the wrong pen to use. They were also then told, apparently by a misinformed judge, that the pens have invisible ink, elections officials said.

As a result, the votes were not counted. But officials insisted there were no dirty tricks involved.

“This one defies logic,” said Jim Allen, a spokesman for the Chicago Board of Elections. “You try to anticipate everything. But certain things just … they go beyond any kind of planning you can perform.”

By late afternoon, five voters had been contacted and told to come back to the polling place to vote again. And elections staff had left messages at the homes of the rest, Allen said.

Carlton and Angela Burkhardt, another voter who was told the same invisible ink story, spent a good part of the day calling and e-mailing the Board of Elections to get answers.

“I am furious and devastated and I just feel stupid,” Carlton said. “I feel so angry.”

Both women agreed that this election meant a lot. They had spent a good deal of time researching candidates.

“I have been voting since I was 18,” said Carlton, 38. “This is the most important election of my life so far.”

Burkhardt planned to go back to vote late Tuesday. She worried about those who might not be able to return.

“I worry about the other people who were there,” she said. “Maybe [they] can’t get off work. I am a person of privilege. I can go back. What if you couldn’t?”

OK. First let’s dismiss the conventional kind of explanation. The piece suggests election shenanigans, but that’s just not a good theory. For one thing the disenfranchised voters left the polling station aware that something funny was going on. We need to find another cause.

If Chicago is like most places, the election official will be a local bureaucrat, postal officer or other person renowned for behaving in a predictably focussed attention kind of a way. There will likely be a strong addiction to the kind of low level background stress that such organizations generate within themselves. This will lead to a deeply habituated tendency to defer to heuristics - a fixation on proceduralism. Now elections are important and are taken seriously. The official might even need to be sworn in, and the apparatus of the Voting Procedure will have been explained in great detail. There isn’t much to it, but the Sacred Pen, Sacred Ballots and Sacred Box will each have been considered with due solemnity. There will have been no doubt that the Sacred Pen is an integral part of The Procedure, which is given a strange, ineffable mystery because it gives its devotees a stress hit with chemistry similar to the effects of cocaine.

So with the system boundaries and heuristics thus arranged, we arrive at the moment when a human being deferred to the authority of a broken pen. This caused a broken narrative when others also noticed, which the overly “compliant” official, filled with background fear, stress and defensiveness, subconsciously filled in with confabulation. The pen uses invisible ink! We can forget our pride in our wonderful technological society. The fantasy is funny not because it is alien but because we all recognize the poetic resonance with the Brothers Grimm!

Confabulators have a convincing certainty about them - it’s probably something to do with having suddenly reduced their fear. That, the tendency to groupthink, and likely the deluded one’s Armband of Office would all have pressured all the other workers and the voters into a textbook conformity experiment. The rest, as they say, is history.

It’s important to realize that no white cats were stroked in the making of this mess. The key event - the invention of the invisible ink - occurred without conscious reflection, in a mind squeezed like a foot in a too tight shoe, at a peak of reactive fear. Unfortunately, no conspiracy is necessary for there to be damage. On this occasion the delusion has seen daylight and at least some voters have had their ballots collected, but what tends to happen is that the nonsense kicks around and causes knock-on evasive fear much worse than the threat posed by a broken pen.

I once saw a work in progress management system that was working very well, but for some reason scaled really, really badly when it produced reports. The thing was quite well structured, there were about a half dozen people on the team, and a myth that a database server was inevitably compute intensive. The business logic never directly used the database - instead there were higher level calls. There was something odd about those calls though. When I tried to understand them, at first I failed. I just wasn’t getting it. I asked around, and realized that the team members weren’t talking straight to me. There was something very funny going on.

Eventually I got to the bottom of it. The guy in charge of the database stuff had a blind spot. He obviously had no concrete mental model of what a C program does and/or he’d never done the thing of thinking things through, round and round, explaining everything to himself. He’d written some frighteningly good code by applying rote learned incantations, but there are limits to the possible. He didn’t realize that an SQL query could be assembled in a buffer with the things to search for filled in between the SELECT, WHERE, JOIN and so forth. All his queries were in hard coded literal strings. He’d got some quite clever literal strings, which worked well with the somewhat distorted database schema to implement the bizarre programming idiom. These strings would be issued to the database, and then he’d parse and filter the result sets to find the records he wanted, doing memory management like crazy while he was at it, but never cottoning on to the fact that his query strings were just char * pointers that could point to any text he wished to cruft up!

There must have been blood on the carpet at some point (I didn’t find out about that, and didn’t want to), because this whole business was a no-go area. When discussions approached it, otherwise rational people would start logic-chopping and weaseling, as they attempted to make a continuous transition from the truth to Mr. Database’s misunderstanding. The sheer amount of processing needed to support the wriggling must have detracted from their work. My job was supposed to be tweaking the database and UNIX platform to make it go quicker, but the Project Manager was a canny fellow. I eventually figured out the real reason he’d hired me was to walk in, speak the truth and then leave discreetly.

The funny thing was, Mr. Database wasn’t stupid, just fixated on a misunderstanding and without a rich enough model to get him out of it. I wrote a “test program” that prepared SQL with sprintf(3) to INSERT and SELECT some pattern data, and showed it to him along with some “timings” (which I didn’t really care about at all). As soon as he saw what the program was doing he became very excited, and immediately saw how make things go faster.

Still, I’m looking forward to the Chicago Council coming to order in their splendid new Robes of Office. Will there be webcams? ;-)

All Postings, Stress Addiction, Programming

Obstructing and Assisting Effective Thought

This week I saw a quote from Marvin Minsky, said to be from his book The Emotion Machine (which I haven’t read yet):

… the cascades that we call Suffering must have evolved from earlier schemes that helped us to limit our injuries — by providing the goal of escaping from pain. Evolution never had any sense of how a species might evolve next — so it did not anticipate how pain might disrupt our future high-level abilities. We came to evolve a design that protects our bodies but ruins our minds.

I know exactly where he’s coming from. Last week a bit of a dental filling chipped off, resulting in more pain than was reasonable for such a small thing. Something was clearly Very Wrong, so it was straight round to the tandarts (Flemish dentist) for me. Three root canals later I was nursing a wicked post-surgical insult to my jawbone, and feeling very glad that I avoid taking Paracetamol unless I really, really need it. All told I spent a week quite unfit for anything involving a brain.

This was a timely reminder that although the issue of background stress making the prefrontal cortex unavailable for programming work is central to this blog, there are other things - such as pain - that can also obstruct or assist intellectual function.

The Boston Globe recently featured an article, Don’t just stand there, think, describing studies that show clear improvements in performing a variety of intellectual tasks when the subjects were encouraged to move while performing them:

The brain is often envisioned as something like a computer, and the body as its all-purpose tool. But a growing body of new research suggests that something more collaborative is going on - that we think not just with our brains, but with our bodies. A series of studies, the latest published in November, has shown that children can solve math problems better if they are told to use their hands while thinking. Another recent study suggested that stage actors remember their lines better when they are moving. And in one study published last year, subjects asked to move their eyes in a specific pattern while puzzling through a brainteaser were twice as likely to solve it.

Of course, we programmers already know about this. Here’s the excellent Mr. Randall Munroe referencing it:

It’s even got a name, and a Wikipedia entry - stimming. My favourite stim involves glancing pats to the back of my head - a motion not far removed from shining a shoe. It’s remarkable that I’ve still got plenty of hair on the back of my head, really. So it’s nice to see that there is now quantitive confirmation that stimming helps thought, because it’s one of those things that reveal the strange disconnect running through a society where most of the people aren’t in a position to access their full faculties, most of the time. For example, here’s Dr. Russell Barkley, an ADHD expert who doesn’t seem to recognize anything but focussed attention, getting very worried about stimming on his ADHD Fact Sheet:

2. Excessive task-irrelevant activity or activity that is poorly regulated to the demands of a situation. Individuals with ADHD in many cases are noted to be excessively fidgety, restless, and “on the go.” They display excessive movement not required to complete a task, such as wriggling their feet and legs, tapping things, rocking while seated, or shifting their posture or position while performing relatively boring tasks. Younger children with the disorder may show excessive running, climbing, and other gross motor activity. While this tends to decline with age, even teenagers with ADHD are more restless and fidgety than their peers.

Crikey! We can’t have people tapping things or shifting their posture, can we? Where might it end? This kind of thing seems so silly, the so-called expert’s ignorance seems downright willful. This is the kind of thing which makes me think a lot of the ADHD debate is actually based in neurotic responses to healthy children on the part of adults trapped in focussed attention, and demonizing those with mismatching social stress levels as I describe on the pages The Dreaded Jungian Backlash and Other Applications. To add weight to this, it’s worth realizing that many people who technological Westerners regard as primitive encourage children to rock as an aid to study, rather than calling it a “symptom” and reaching for the Ritalin regardless of the effect. Check out this sobering bit of video which lasts for less than a minute, half of which is caption:

The really interesting thing about this is that once we know about it, this disconnect can be found in other places, allowing the possibility of understanding things which are unintelligible without allowing for it. For example, although Dr. Barkeley recognizes focussed attention but seems unaware of the existence of juxtapositional awareness as leveraged by effective programmers and artists or the motile component of effective thought, there are traditional schools of psychology which do describe these things. For example, 20th century esoteric teachers George Gurdjieff and Rudolph Steiner both continue to attract interest from many hackers I’ve interviewed over the years. They both claimed that human consciousness is the result of the interaction of three distinct subsystems, and encouraged students to distinguish the operation of each in order to improve their awareness. Gurdjieff describes three main centres:

  • Intellectual or thinking center. This center is the faculty which makes a being capable of logic and reasoning. This one is located in the head.
  • Moving or physical center. This brain is located in the spinal column.
  • Emotional or feeling center. This faculty makes beings capable of feeling emotions. This brain is dispersed throughout the human body as nerves which have been labeled as the “nerve nodes” . The biggest concentration of these nerves is in the solar plexus.

Note that the “emotions” of the feeling centre are not the base, reactive emotions such as hunger, fear, lust and so on. Those are handled by the moving centre. Instead the feeling centre is about impressions - stuff that we get in an “all or nothing, insight kind of way”. It seems to map very well to what I’ve called the juxtapositional faculty. Does the prefrontal cortex integrate processing from the solar plexus? Recently we’ve realized that a lot of processing does go on in the enteric nervous system - nerves surrounding the gut. As the Wikipedia article says:

There are several reasons why the enteric nervous system may be regarded as a second brain. The enteric nervous system can operate autonomously. It normally communicates with the CNS through the parasympathetic (eg, via the vagus nerve) and sympathetic (eg, via the prevertebral ganglia) nervous systems. However, vertebrate studies show that when the vagus nerve is severed, the enteric nervous system continues to function.

The complexity of the enteric nervous system is another reason for its status as a second brain. In vertebrates the enteric nervous system includes efferent neurons, afferent neurons, and interneurons, all of which make the enteric nervous system capable of carrying reflexes in the absence of CNS input. The sensory neurons report on mechanical and chemical conditions. Through intestinal muscles, the motor neurons control peristalsis and churning of intestinal contents. Other neurons control the secretion of enzymes. The enteric nervous system also makes use of the same neurotransmitters as the CNS, such as acetylcholine, dopamine, and serotonin. The enteric nervous system has the capacity to alter its response depending on such factors as bulk and nutrient composition.

It seems to me that the empirical knowledge of this stuff which Gurdjieff (apparently) gained from traditional Orthodox Christian, Buddhist and Dervish sources describes what we experience (unless we are trapped in focussed attention) and is now at least partially confirmed by modern science. Perhaps it isn’t so surprising that thousands of years of study actually produced something useful!

In the Google book Knowledge of the Higher Worlds: How Is It Achieved? Steiner describes:

… the thinking-brain, the feeling-brain and the willing-brain.

The bizarre thing about Steiner is that he discusses the three brains in the context of clairvoyant perception! Are we talking… like… spooks or something here? I strongly suspect that the answer is no, and here we have an opportunity to resolve one of the great mysteries of the ages. Remember Barkley, recognizing only focussed attention. To him, the fruits of juxtapositional awareness which we can all enjoy if we are sufficiently destressed, and which arrive in an “all or nothing, insight kind of way” without any stepwise “working out”, are mere hallucinations. The human-normal sensibilities of people who are in a position to do juxtapositional awareness are just “procrastination”. Creative people “make things up” - probably as a result of a “failure of inhibition” (Mozart’s reams of note-perfect new music were a kind of complicated epileptic fit). This kind of model is actually quite common - is the way that ISO 9001 is usually applied anything other than an infinite regress of people standing behind other people and telling them exactly what to do?

To such a person, an experienced professional who can look at a spec and know that there is something really difficult in there without yet being able to say exactly what it is, might as well be “clairvoyant”. But there is nothing spooky going on - just the amazing effectiveness of the neural net between our ears when it has a chance to work correctly. This also explains why some people are so keen to advance theories of how creative people “make things up”, when every creative person denies making things up, instead insisting that they just see and attempt to capture what is there. From the limited point of view of focussed attention such statements make no sense, and can be dismissed out of hand!

With these ideas in mind, it’s actually possible to make sense of some of Steiner’s writing in informational and structural terms. I’m not going to offer an example here - or even encourage you to find one - because of the other big problem with Steiner. He was writing in High German, to a very straightlaced turn of the century upper middle class audience that makes Barkley look positively funky in comparision - and the writing gains nothing in translation. Turgid, pompous, long-winded and obfuscated are just some words we might apply to it here and now. But my central proposal here, that Steiner is describing improved cognition of this reality, rather than focussed cognition of a different reality, might be of interest to some people. Of course the audience, knowing only focussed attention, took the description of improved cognition and “interpreted” it as a description of an alternate, spook reality. The relevant graph is not this one from Randall Munroe, correct though it is:

But instead this one from Dehnadi and Bornat’s fascinating paper The Camel Has Two Humps, where they show the two distinct clusters of excellent and average programmers:

Human psychology seems to be a better fit with this bit of pop culture from The Shamen featuring Jhelisa Anderson than the theories of the ADHD experts - so stim away and if Dr. Barkley doesn’t like it… it’s another reason to get rid of open plan!

Neuroscience, All Postings, ADHD, Stress Addiction

Culture Influences Brain Function?

There’s a very interesting abstract, MIT: Culture influences brain function, which says that newly arrived East Asians are better at visual tasks requiring context sensitivity than Americans, but the Americans are better at tasks requiring absolute judgement.

The authors relate this to American culture emphasizing the individual, East Asian culture emphasizing the collective. I suppose it’s possible, but there’s an obvious alternative - sadly cruder and less poetic.

The Wikipedia entry for East Asia says that culturally, East Asia consists of societies:

  • displaying heavy historical influence from the Classical Chinese language (including the traditional Chinese script)
  • Confucianism and Neo-Confucianism
  • Mahayana Buddhism/Zen-Chan Buddhism
  • and Taoism (Daoism)

Politically it consists of:

  • People’s Republic of China (including the Special Administrative Regions of Hong Kong and Macau)
  • Republic of China (Taiwan)
  • Japan
  • North Korea
  • South Korea

Now the Japanese gave Karoshi to the world, so we mustn’t over-generalize, but there’s a great many agrarian and early industrial people in there, as well as Buddhist, Zen and Taoist influences. It’s reasonable to assume that many East Asians have lower habituated background stress than most information age Americans (we’re back to Whybrow’s American Mania), and the cognitive differences reported would fit right in.

Neuroscience, All Postings, Stress Addiction

Seam Carving - A New Thing!

How wonderful to see something new! At SIGGRAPH last autumn Avidan and Shamir presented Seam Carving for Content-Aware Image Resizing, together with a striking video:

There’s now a GPL implementation, seam-carving-gui. It’s amazing. I grabbed an OS X Universal Binary, and was instantly goofing off. It does everything the video shows, except that you can’t stretch and shrink images interactively. Instead you type in the new size and it goes away and thinks.

I started with Trafalgar Square:

Then I crudely painted over Nelson’s Column and removed it to produce an alternate reality - Place De Le Emperor Napoleon perhaps:

Then I stretched it without modifying the appearance of the people to create another timeline with Albert Speer’s aesthetic in London:

I suppose it’s the graphical analogue of those weird boxes that beat-match tunes without distorting the pitch!

The last new thing like this that I saw was Dasher, an innovative approach to text input, back in 2002:

Why is something new such a refreshing delight? Because it’s rare. Innovation is not as common as we would like to believe. Sure, to some extent we are looking at an S curve, because a lot of traditional activities have now been automated and we haven’t thought up many new activities that require heavy data processing support yet. Facebook is hardly a paradigm shift!

A corollary of that relates to the way people just don’t worry about sizing so much any more. There came a point, maybe about five years ago, when the processing and storage requirements of a whole bunch of apps were suddenly met by standard domestic PCs. So far, humanity hasn’t generated many problems > 2GHz / 2GB RAM / 50GB disk. Even those lost UK government records were on an Excel database!

The sobering truth is that for many years, the big progress has involved delivering and incrementally improving features that I had on my Sun pizza box at BT in 1990, to the domestic market. Hats off to the materials scientists who have accomplished their repeated die reductions, but that’s the limit of the true innovation. In it’s day the KIM-1 was a solution looking for a problem, and in those days there was more innovation. Dasher and seam carving prove that innovation is still possible in the context of a single user with a PC.

I’m not the only person to see this dearth of things to be enthusiastic about. In his recent Yuletide greeting (and discussing the consumer market), 2007: The Miserable Year in Review, John C. Dvorak said, “I was not a fan of 2007. It was another crappy tech year—just the latest in a string of bad years dating back to 2000.”

As you might expect, I assign this reduction in true innovation to the effects of increasing background stress. If we can get that under control, we’ll be back on track for the Singularity. Hooray!

All Postings, Stress Addiction, Programming

The Kafka Shuffle

In the Introduction pages I describe how low level stress is enough to reduce working memory and access to the semantic and associative network. In this focussed attention hardware state, people can’t program or perform other tasks central to software engineering. If a person spends most or all of their waking hours in focussed attention, they’ll stop expecting or relying on self-consistency, distrust the evidence of their own senses, and develop an increasing reliance on heuristics (procedures) and their policing. They will also be conditioned to resent others who do not match their stress level, either stressing them too much, or not performing the anxiety raising social stereotypies enough to give an addictive hit of stress chemistry. Just as with drunk humour, a group of people suffering the same distortion of their thinking will produce a distorted culture. Once a distorted is established, it can sustain itself by driving newcomers into stress, and conceal itself by rationalizing or even celebrating the goings on.

Such a culture is the kiss of death to any commercial activity. It is operationally inefficient because workers aren’t alive to the possibilities for intelligent optimizations of their tasks and instead develop buck-passing skills. It produces constant, invisible opportunity costs because of the emphasis on reactive behaviours. It produces conflict with customers and business partners as dealings develop the character we call “bureaucratic” - because government bureaucracies are places where stress addicted culture can get really bad. I emphasize that a stress addicted culture will eventually doom any enterprise - those that get by doing predictable business with such a culture are always vulnerable to sudden death by innovative competition, or even trivial changes to the business environment if they cannot adapt.

In this posting I’m going to look at an example of such a stress addicted culture, showing how deep and pervasive the reality warp can get, and how it creates conflict with non-members. In a commercial context we should ask how the attitudes revealed will affect motivation, quality, efficiency, flexibility, creativity and other good things. Conversely, think about how such a culture will inevitably display the extreme rejection of improvement and hence organizational inertia I call The Dreaded Jungian Backlash, while endorsing the correctness of each member’s distorted perceptions. Consider the idea that there’s a kind of organizational hygiene that anyone responsible for leading others should be aware of, and stress addicted culture is a disease of morale that can render the wage bill worse than wasted.

The example is an often repeated Internet article written by a physician turned homeschooler called Matt James. James went on to write Homeschooling Odyssey, a well received book with the tag line, “Hour-a-day homeschooling has sent our six children to Stanford and beyond. While academic achievements are gratifying, they represent a drop in the bucket of benefits in store for families who choose to homeschool.” The most direct citation for James’ article that I’ve found is Little Manchurian Candidates, but because the model of stress addicted culture fits so well with everything he observes and expresses, I’m going to quote the whole thing here:

Little Manchurian Candidates

by Matt James
“One ring to rule them all, one ring to find them,
One ring to bring them all, and in the darkness bind them.”

Our six-year-old daughter was so excited to start school. At our first parent-teacher conference, Barb and I expected to hear the usual compliments and heartwarming anecdotes about our bright little angel. From our experiences with activities like T-ball and soccer, or dance and music recitals, we had learned that parents always say nice things about the children of others. If the compliments are sometimes unrealistic or excessive, well, parenting is tough work. We can all use the encouragement. I guess we had been spoiled. Jenny’s teacher got right to the point. She had some negatives to address. For one thing, Jenny was struggling with her reading. The teacher confessed that one of the most difficult parts of her job was deflating parents with the news that their children were simply not exceptional. Jenny was, at best, an average reader. She was not an Eagle; she was a Pony. Our job was to learn to enjoy her as a 40-watt bulb rather than a bright light. Was it my imagination, or did this middle-aged matron’s sweet smile contain a trace of malice as she related these tidings? I was confused by this assessment of Jenny’s reading abilities because it simply didn’t fit in with her prior history. She had a love affair with books for her entire childhood. We have a photograph of her at 11 months of age staring earnestly at the contents of an open book. I remember reading to her when she was three. I stopped for some reason, but she continued the narration. She knew her stories by heart. Like many other children, Jenny had learned to read at home. She was a bookworm, and she was an experienced and passionate reader before she ever started first grade. The teacher went on to explain that Jenny cried too much at school and that we needed to correct this problem with the appropriate discipline. Barb and I exchanged glances but didn’t argue. We were in shock.

I was curious about the crying. Jenny was such a happy child. I asked her that night what made her sad at school. Expecting to hear about something on the playground, I was surprised by her answer. The listening-hour stories made her sad:

Once upon a time there was a daddy duck with seven ducklings. They ranged in age down to the youngest (who reminded Jenny of a first grader). The daddy was mean. One day he demanded that all his children learn three tasks, such as running, swimming, and diving. If a duckling was unable to master all of the tasks, he would be banished from the family to live with the chickens. The youngsters struggled under the cruel eye of their father. When it came to diving, the first grader floundered and was sent away to live with the chickens.

This was the story Jenny related, in her own words, as an example. I heard it told a second time several years later, by my cousin Nancy, as a sample of objectionable curriculum. We were impressed with the coincidence, since our families resided in different states.

Jenny told me she also cried over stories in her readers. They made her sad and frustrated in some way. What a mess! In one evening we had found out that Jenny was unhappy at school, that her teacher thought she was a poor reader and a dim bulb, and that she heard mean tales during listening-hour that I wouldn’t repeat to hardened convicts. What in the name of heaven was going on at this school?

I was determined to get to the bottom of things. Since they didn’t send books home with students in the younger grades, I went to the school the following day and spent a couple of hours reviewing the elementary readers. As I read, my eyes opened wider and wider. I had assumed the purpose of the reading curriculum was to stimulate the juvenile imagination and teach reading skills. Instead, I saw material saturated with, to borrow another parent’s language, “an unadvertised agenda promoting parental alienation, loss of identity and self-confidence, group-dependence, passivity, and anti-intellectualism.”

I once daydreamed through a basic psychology class in medical school which described the work of Pavlov and B.F Skinner in the twentieth century. Their conclusions were that animal (and human) behaviors can be encouraged or discouraged by associating them with pleasure or pain. This is such an obvious fact of nature. It is amazing that anyone would bother to prove it with experimentation, as if the carrot and the stick haven’t been used since time began.

In behaviorist experiments various stimuli, such as food or electrical shocks, were used as rewards or deterrents. Over time, due to animal memory, a pattern of behavior could be established without food or shocks coming into play. This educational or training process is called “conditioning.” With enough conditioning, the dog will stop chasing cars.

As I read the stories and poems in Jenny’s readers, I was astonished to discover that they were alive, in their own way, with the theories and practices of these dead scientists. But the animals to be trained weren’t dogs or rats. They were our young students. Pleasure and pain signals were embedded into the reading material in a consistent way. Given the vicarious nature of the reading experience, and by identifying with the protagonists in the stories, it was our first graders who were “learning” certain attitudes and behaviors.

When a child-figure in the stories split away from his group, for example, he would get rained on, his toes would get cold in the snow, or he would experience some other form of discomfort or torment. Similar material was repeated ad infinitum. Through their reading, our students would feel the stinging rain and the pain of freezing toes. They would learn the lesson like one of Pavlov’s dogs: avoid the pain, stay with the group.

The stories in the readers consistently associated individual initiative with emotional or physical pain. Consider the example of the little squirrel whose wheel falls off his wagon. When he tries to replace it, the wagon rides with an awkward and embarrassing bump, noticeable to his friends, who then tease him about it. Another attempt to repair the wheel results in an accident, with bruising and bleeding and more humiliation. The cumulative effect of this and similar story lines, given the vicarious nature of the reading experience, would be to discourage initiative and reduce self-confidence in the first grader.

Animal dads, moms, and grandparents were portrayed over and over in various combinations as mean, stupid, unreliable, bungling, impotent or incompetent. Relationships with their children were almost always dysfunctional; communication and reciprocal trust were non-existent. A toxic mom or dad, for instance, might have stepped in to help our youthful squirrel repair his wagon, only to make matters worse and wreak emotional havoc in the process. Jenny’s heart would be lacerated by stories which constantly portrayed parent/child relationships as strained, cruel, or distant. I could see her crying with hurt or frustration.

It occurred to me that over the long run, at some level of consciousness, our daughter would have to hold us accountable for permitting her to be tortured in school. Logically, Barb and I had to be stupid, unreliable, uncaring, or impotent, just like the parents in the books. By sending her to school, we were validating the message in her readers, contributing significantly to the parental alienation curriculum. Continuing in her school-based reading series, Jenny’s relationship with us would have become tarnished or eroded, and an element of bitterness or cynicism might have crept into her personality.

I borrow the term “anti-intellectualism” to describe another dominant theme in the readers. Many of the compositions were, essentially, word salad. They lacked intrinsic interest, coherence, or continuity, and they often demonstrated a sort of anti-rationality. The stories and the corresponding questions seemed to require the student to suspend the natural operations of his intellect, such as the desire to make sense out of things or the impulse to be curious. Under this yoke, a student could learn to hate reading or even thought itself.

The following “story” and “comprehension” questions are representative of the anti-intellectualism that I found in the readers:

Once upon a time there was a little green mouse who hopped after a tiger onto a yellow airplane. The plane turned into a big red bird in flight, and the mouse turned into a blue pumpkin. The pumpkin fell to the ground and its seeds grew into pots and pans. Blah, blah, blah

1) “What color was the mouse?”

2) “Why do mice turn into pumpkins?”

3) “How do seeds grow?”

I can see children getting frustrated over material like this. It is debatable as to which facet of the exercise is more onerous, the reading or the “comprehension.” I almost incline to the latter. Among other concerns, I wonder if it is a good thing to pressure children to respond to stupid or unanswerable questions. Such a process would lead to passivity and a loss of confidence, to a little engine that couldn’t.

According to Pavlov and B.F. Skinner, repetition of unpleasant reading experiences would turn a student off to the reading activity. Predictable consequences would be a child who hates reading and loses out on vast intellectual benefits and development. In addition, his reading failure would tax his self-confidence, and he could be branded with one of society’s popular labels such as dyslexia.

I considered Jenny’s reading struggles in the context of performance expectations as well as grading and comparisons with other children. It seemed as if she faced a nasty dilemma: force herself to read alienating material, or disengage and then disappoint parents, teachers and self. What an impossible predicament for a young child. Once sunny and blue, the skies had turned dark and stormy for our happy little girl whose only offense had been to attend her friendly neighborhood school at the innocent age of six.

It has occurred to me that the cause of America’s illiteracy crisis has been discovered. It is the reading curriculum in our schools. Unfortunately, the damage to children appears to extend way beyond reading failure. One wonders if the hidden agenda in the readers has created our victim culture, a generation of withdrawn and resentful children, alienated from themselves, their parents, society, books and ideas.

I was reminded of the plight of our neighbors. The father and mother were loving, dedicated parents. He was an accountant and she was a homemaker and community leader. They were nice people, and so were their children. The two teenagers were bright but got poor grades and hated school. They hung out with the crowd and participated in the kind of self-destructive behaviors that are commonplace today. I asked these young people why they would behave in ways which would cause pain for themselves or their loved ones. They smiled quizzically and professed not to know. Maybe the ideas that moved them truly were subconscious.

We are all familiar with kids like this (Our own kids are kids like this, or they come too close for comfort). They spend a lot of time “doing nothing” with like-minded friends. Passive-aggressive with suppressed individuality, they all seem cut from the same mold. Self mutilation with tattoos and body armor is almost universal. Some of their groups are virtually masochistic cults. Sadism is the other side of the masochism coin.

That so many of these dysfunctional teenagers come from loving homes and neat families is inexplicable and shocking, until you realize that they have all been tortured together in school since the first grade. They are a batch of little Manchurian Candidates with attitude, victims of the obscure behaviorism that I found, and that others have found before and since, in school readers.

Barb and I had seen some perplexing changes in Jenny’s reading since she started in first grade. For one thing, she had stopped reading her favorite books and stories at home. Before starting school, she had feasted on Grimm’s Fairy Tales. Although she still begged us to read these to her, she now explained that she was not supposed to read them herself, according to her understanding from her teacher, because they contained big words and content in advance of her abilities. Barb and I, holding our tongues, exchanged tortured grimaces and cross-eyed glances.

When reviewing the school readers, I had noticed an impoverished vocabulary, composed mostly of three and four letter words. I brought this up with the teacher. She explained that the readers were integrated into a district policy that no more than five hundred new words be introduced to students during any grade level. The idea was to protect children from the dizzying and confusing effects of an overabundance of words and ideas. I nodded as if I understood, but I didn’t really get it.

Barb and I had clearly used the wrong approach with Jenny. We had allowed her to read anything she wanted and had provided her with a flourishing home library. Furthermore, we had encouraged her to run around in the grassy meadows and on the sandy beaches. She must have collided with great numbers of unfamiliar words and ideas, as well as a perilous diversity of flowers and sea shells. It’s a wonder she survived at all.

We considered the various elements of Jenny’s brief experience in first grade. She had a clueless teacher. She was regressing in her reading skills, vocabulary, and enthusiasm. She was being indoctrinated with character destroying qualities like passivity and group dependence. Her intellectual development was being stunted and she was being bombarded with a curriculum of parental alienation.

Judging by her crying in the classroom, she was part of a captive audience being repeatedly exposed to painful stimuli. To put it plainly, she was the victim of ongoing torture and cruelty. Along with her classmates, she was becoming, as one of her school poems pointed out, “Small, small, small, just a tiny, tiny, tiny piece of it all.”

In our state at that time, compulsory education began at the age of eight. Jenny was not obliged by law to attend school. With our various concerns, we pulled her out of school while we tried to figure out what to do.

First let’s consider the idea that the reading primer is an artifact of some sinister conspiracy that is consciously using behaviourist conditioning to stunt children’s growth. So many people have formed the impression (that is, they’ve spontaneously noticed a pattern with their juxtapositionally active prefrontal cortexes) of something like this going on that it’s a familiar idea in the homeschooling world. The trouble is, like many conspiracy theories it’s an implausibly complicated idea. The conspiracy must be huge. Even the junior grade teacher seems to be in on it.

How does it work? Do the buses go round collecting all the teachers on a Friday night, and take them off for New World Order indoctrination sessions where the sinister plans for manipulating everyone else are revealed to them?

Who does the catering? “Twelve thousand cheese rolls, six thousand classic, six thousand lite, a hundred boxes of eye in the pyramid paper napkins? Sign here please…”

It isn’t very plausible, yet the impression remains. I think popular culture has a better sense of the kind of thing that happens at times of high background stress. The junior grade teacher, along with various other people, has gone weird - and the people who have gone weird are in spooky agreement with each other!

Of course I’m not suggesting that the education industry has been infiltrated by pod people, but I am saying that it forms a closely connected social group which can run up very high levels of purely socially generated stress, so stress addiction can reach binge levels, and everyone involved will experience the same cognitive impairments. There is hostility towards those with mismatching stress levels, and a certain self-satisfied complacency that is also seen in people who have produced the same changes in brain chemistry using addictive drugs. So the junior grade teacher will exhibit a patronizing and smug malice towards those not sharing her crazy ideas.

What are her crazy ideas and where do they come from? In the posting Response to a Reddit Comment I expand on some of the logical effects of not being used to seeing self consistency. People stop expecting it, looking for it, or reasoning from it. Without being able to construct and validate their own chains of reasoning, it’s hardly surprising that they fall back on heuristics, and concentrate on demonstrating compliance as their principle tactic of blame avoidance. They become reactive, and live in constant fear of random misery coming their way. This is how background stress becomes a vicious circle, and giving people good reason to be self-confident becomes a virtuous one.

Consider the supply chain for the reading primer. The authors are probably ex teachers, or in some way connected with the education industry. That’s how they differentiated themselves to the publishers, who will be proud of their “close contacts” with the bureaucracy which will include the purchasing committee. In common with many other industries, the people involved will sometimes change employers, but stay within the same group. Teachers will become authors, committee members will go to work for the publishers, perhaps the publisher’s chief executive will chair the bureaucracy. They will all complain of how stressed they are and express pride in their “knowledge of the industry”, but not notice that they raise the stress amongst themselves with their industry practices or put two and two together. (Those that do become disillusioned and leave.)

So everyone from author to teacher has the same stress addiction warped crazy ideas, and it’s not surprising that the reading primer they deliver to children with great pride reflects those crazy ideas. Demands are random, cruel and impossible. Nothing has narrative continuity, causality disappears. Safety is in the herd, trying anything will lead to danger and worse - group mockery. From their point of view, the reading primer is a collection of worthy tales, perhaps with a vague memory of The Gashlycrumb Tinies mixed in without humour or taste.

The most powerful line in the article for me was:

I wonder if it is a good thing to pressure children to respond to stupid or unanswerable questions. Such a process would lead to passivity and a loss of confidence, to a little engine that couldn’t.

How often have I seen the “Oh just put something and get on” mentality, seen it make a shallow pretence - even mockery - of whatever simple task supposedly intelligent people have been trying to do! Humans need causality - reason - to stay sane. We are not healthy if we spend all our time in focussed attention, unable to establish context or weigh significance. Saint Bob saw this when in response to Brenda Ann Spencer’s trend setting school massacre he wrote,

And he can see no reasons
Cos there are no reasons
What reasons do you need to be shown?

There is no-one stroking a white cat. No conspiracy to damage children. The reading primer just reflects the frightened and frightening, sorry state of the people who produced it, and that state of mind has the property that it will drag other minds down to its own level if they are exposed to it for too long. But its effects are far deeper and far worse than any fiendish plan could conceive. Until we recognize and address it, this stupid socially generated disease will be in control.

It’s a state of mind that destroys firms and causes nations to stagnate as well as producing spontaneous explosions of nihilistic behaviour in schoolchildren.

Full Disclosure

I am prejudiced against claims of the world-girdling power of white cats. Twirip of the Mists has received immense investment as I try to build up “special static electricity”, but despite developing a very encouraging Gremlin impersonation she has not yet attracted a mononomative1 girlfriend, yet alone phalanxes of boiler-suited minions.

1: Having only one name.

All Postings, Logical Effects, ADHD, Stress Addiction

Low Level Stress Matters

In this posting I shall expand on the amount of stress which is needed to stop people programming, how it differs from the usual understanding of workplace stress, and how giving people good reason to be self-confident can control it.

The kind of stress that people usually talk about in workplace settings might be called extreme stress. For example, the UK Health and Safety Executive offers these bullet points on their page, Why tackle work-related stress?:

  • Work-related stress accounts for over a third of all new incidences of ill health.
  • Each case of work-related stress, depression or anxiety related ill health leads to an average of 30.2 working days lost.
  • A total of 13.8 million working days were lost to work-related stress, depression and anxiety in 2006/07.

That’s serious! Clinical depression and anxiety related ill health are certainly not trivial. What we can take from this is an indication of how much stress is actually running around some sites, but it’s a far higher level than I argue is needed to stop people being able to program. Because most people maintain enough background stress to stop them programming most of the time, we would normally think of a worker who is in fact too stressed to program as “unstressed”. I’ve found almost nothing on the websites of firms who do stress management training (and there are lots of them) about the vast untapped resource of intelligence which is concealed by background stress. A notable exception is one David Lee, who talks about this on his page Employee Stress and Performance:

One obvious implication of this research is that employee intellectual functioning can be very powerfully influenced by their environment. In workplaces where employees feel helpless and disempowered, they are less likely to think in intelligent, creative ways. Another important implication, and this is born out by other research, is that perceived control plays a major role in whether a person is affected by a potentially stressful workplace. Workers in jobs with similar demands, but different levels of control, exhibit very different psychological and physiological responses. With the same demand level, workers in low control workplaces are significantly more affected by their work.

This does not mean that we are doomed before we start. It is true that we can’t build gelled teams where the members are suffering the kind of stress that the Health and Safety Executive worries about, but it is possible to arrange the conditions of an individual team to reduce such stress, even if the rest of the organization continues to pass it around wretchedly.

The Dreaded Jungian Backlash can then be expected, but senior managers can defuse that by simply telling the people who are getting snotty not to worry about it, and just get on with doing their stuff. As soon as they are confident that they are not to blame, they won’t feel the need to translate the resentment they feel because of stress mismatch into microsabotage. They will no longer feel a need to “teach lessons” in case, by omitting to exercise stress-addicted sanctions, they might become “non-compliant” themselves. A few moments of strong leadership can blow this stuff away, but it’s amazing how many line managers are unable to do this because they are themselves trapped in the stress addicted organization’s climate of fear and unmentionables, and react to mounting resentment in a way which endorses and encourages it.

So just as the Health and Safety Executive say, we can back off from chronic, health destroying stress by identifying and removing stressors, and helping people organize their lives in a less stressful way. We also have to manage the Dreaded Jungian Backlash, which (as far as I know) is a novel part of this analysis.

Unfortunately we’ve still not done once we’ve got our teams sitting calmly and not having heart attacks. Even if they were the most unstressed people on their commuter train, they’d still be too stressed to be able to program (unless they happen to be a member of the lucky minority). We have to reduce their stress levels below ambient, and we have to do this without packing them off to a meditation centre in the mountains. It isn’t practical, and they’d probably object. We can’t get any further by removing stressors. Fortunately we can cut over to another trick - displacement.

By carefully working through everything a team member will need to call on, making sure they understand their job, and how it fits together with everyone else’s, and getting them to demonstrate their control to themselves, we can create the perception of control that Lee talks about. As all the self-improvement traditions say, good thoughts drive out bad ones. The worker might have a very stressful life outside, but at work they know where they are, their tools work as advertised, they know the tricks to make their libraries work. So they know they can load up a problem and get on with productive stuff, and as soon as they do that, they’re in flow. (At first, you will have to stimulate juxtapositional thinking. Use The Original Talks as a starting point - the chapters are available through the links at top right of the starting page.)

There’s another interesting point on Lee’s page that’s particularly relevant to software engineers working in open plan. He says:

One group was subjected to a loud noise in the middle of the exercise and told there was nothing they could do about it; they had to “grin and bear it.” The other group was subjected to the same loud noise in the middle of the exercise, but they were told they could have the noise stopped if they chose. The results were both fascinating and disturbing in their implications for organizational performance.

The group that had no control demonstrated a significant deterioration in their thought process during and after the noise. Their thinking became unemotional, unimaginative, and dull. It was as if they became temporarily dumb in order to endure the stressful situation. Even more interesting was the other group’s response. Although they were told they could stop the noise if they needed to, not one person chose to do so. Therefore, they experienced the same amount of unpleasant noise as the group which wasn’t given that option.

An engineer listening to CDs using earphones controls the tune and the volume. A naked engineer is exposed to the torture of waiting for the next phone to ring or conversation to start, and can’t stop them when they do. Earphones are obviously very valuable in open plan, and should never be banned. Going further, a lot of people don’t like the tiny phones which fit into the ear because they chaff, leading to the popularity of light headphones which sit on the ear. The problem with these is that they leak more sound, and this can be annoying to people who are not listening to music. Given the evidence for the debilitating effects of uncontrollable noise generated stress in open plan, I think it would make more sense to allow leaky headphones and suggest that everyone uses them.

Neuroscience, All Postings, Stress Addiction