We can ground many of the known difficulties of software engineering in cognitive problems caused by stress. Software engineers must not be stressed if they are going to be effective. At this point, it may seem that practices such as meditation, good dietary and sleep regimes would be needed. Although individuals can certainly assure their performance by doing that sort of thing, plenty of beer chugging young Trekkie Monsters will have no interest in doing so, and (fortunately) software team leaders have no right to control the private lives of their workers. The good news is that it isn’t necessary. It’s quite possible to take practical steps and set up islands of confidence within organizations, where full cognition is available. Confidence blows away stress like sunshine dispels the morning mist. Here is your Five Step Programme To Success:

1. Environment.

We all know how bad open plan offices are. In Peopleware DeMarco and Lister rail against open plan. So does Spolsky, and anyone else who’s actually done programming. Later on I’ll discuss why organizations persist in deploying open plan, when we know they might as well require people to work with blindfolds on and enter code using DIP switches, and it’s not just evil accountants. So avoid open plan like the plague. If you have no choice about open plan, do your best to mitigate it. Keep noise down. (Did you notice that one of the papers cited above explicitly used noise to accomplish cognitive impairment?). Provide some privacy. Most people get more stressed when they’re in a fishbowl, and I’ve interviewed plenty of people who suffer agoraphobic panic attacks when trying to concentrate in open plan. So arrange the cubicle farm to make people feel secure.Make sure you pass Spolsky’s Joel Test. (Obviously point 12 assumes your product has an HCI, and remember to put plenty of automated regression testing in your nightly builds.)

Use common sense. I recently saw a site where they had source code control alright. They had six different Visual SourceSafe repositories, and doing a build required the programmer to manually make a patchwork source tree from fragments taken from all of them. In the chaos there were different versions of the same files, with the same names, in the multiple repositories. Building anything required picking a compatible set of all the multiples. So the team kept compiled binaries on a hard disk, and was not able to show that they could build the whole product. I commented that it looked pretty messy, and the team leader said, “Yes! Imagine how bad it would be if I hadn’t compartmentalized everything!”. Weep for your species.

Above all, keep it simple. Sufficiently simple systems allow the workers to have confidence that they’ve got things right, and avoid having to battle or think about them. Don’t get into another situation at the site described above, where they had some bits that needed VC++ 5, 6 and 7 to compile, and of course these version’s outputs were incompatible.

2. Provide Good Reason To Be Self-Confident.

This is the most important bit. Talk to your people. Make sure they understand the tools and APIs they will use. Get them to write little demo programs to exercise the features they will use, to make sure everything works as advertised and also to make sure they know that they can use them. A person who knows that that they are fluent in the technologies they must use because they’ve done it will be secure. That’s the kind of solid, grounded self-confidence you need, and that’s what blows away the anxiety. People are often embarrassed to say they don’t understand something, particularly if they’re accustomed to pretending fear of disembowelment. So you have to be a mother hen, and make sure. This is not micro-management, it’s care. With all the extraneous stuff sorted out, you will be able to define and bound the unsolved problems that your engineers will work on, and a bounded intellectual challenge is something that a person can solve and so increase their own good reason to be self-confident.

I emphasize the good reason bit. It’s easy to encourage people to be bombastic, in fact it’s one of the biggest problems in the post-dot-com world. But bombast doesn’t make self-confidence, instead it erodes it corrosively. The worker who has successfully implemented a rendition of Hugga Wugga distributed across your local net will have good reason to be confident of their ability to handle the networking APIs and implement some banking transactions.

3. Achieve Ignorance.

This is related to the point about bombast. Particularly amongst younger workers, there is a tendency to avoid admitting that they don’t know how to do something. But in software, intelligent workers never re-invent the wheel. If they have been given a problem, it’s because no-one’s solved it before. Before we can solve a problem, the first thing we have to do is acknowledge that we don’t yet have the answer. Therefore the first milestone for any sensible team is achieving ignorance. From that point, real progress can be made. But sometimes, ignorance is a real achievement - perhaps the hardest part of the whole project.

4. Stimulate Juxtapositional Awareness.

If you’ve got the previous steps right, you will now have a crew of workers who are capable of thinking juxtapositionally. The trouble is, if they have no experience of doing so, they won’t know it. If you woke up tomorrow with wings, you wouldn’t know that you could fly! This was the first bit that I realized I was doing. I developed a hodge-podge of anecdotes and examples, that got people thinking is a critical way about their work, looking at it deeply and holistically, and verifying what they believed to be true by attempting to prove their knowledge to be false. This idea of falsifiability is the core of the scientific method. An idea is of no value unless it makes statements which can be tested, and so shown to be false if they are wrong. People who are used to pretending fear of disembowelment do not attempt to falsify what they are told. Instead, they passively “comply”. Then the holes in a shoddy analysis or design don’t show up until late in implementation, threatening the project.You can find some transcripts of my final series of talks here. In those days, I tagged people stuck in stress-induced focussed awareness as “packers”, and people who could access juxtapositional awareness as “mappers”. Discuss them with your team. Use them for inspiration, and adjust to taste. If you find any other good ideas, let me know!

5. Maintain Intelligent Good Practice.

Never do anything for the sake of it. Keep perspective. Remember that the least important requirement often ends up driving the design (I think I got that one from Jeffery D. Kooistra, but I’m not sure). Avoid “brainstorming”, which usually degrades into ill-considered bombast sessions where the most thoughtful people quickly get shouted down. Instead, present a problem to the group, ask them to think about it, research, and kick it around to taste, and then schedule a meeting where each person can present their findings. Be grown up!

Remember that your product is code. Software engineers who sneer that they have no interest in “mere code”, are in fact seeking displacement activities which will allow them to avoid the stuff that really matters. This is hardly surprising given that most engineers aren’t in a cognitive state where they can handle that stuff, but when we can, we have to remember that the contempt for code was always an excuse. Keep it clean. Seek necessary sufficiency. Be rigourous. Make sure it looks good. In particular, reject coding standards which prevent you making an aesthetically pleasing layout. If you do this, you’ll find that you can print out your code, stick it on the wall, and see bugs from six feet away. Really! Ask yourself if you would be happy for someone to pick a file at random, print it out, and display it in Reception. If not, what have you got to hide?

Expectations

You’ll soon know if you’re getting it right. Each person will quickly develop humour and emotional engagement with the work. You will meet the whole person, and nothing less is up to the intellectual challenge of software engineering. A good Project Leader should be like a technical sergeant, responsible for the well-being and effectiveness of every member of the team. If the people don’t know how to do something or need support, the Project Leader must be the go-to person who takes the responsibility. Never, ever, leave team members with problems they cannot solve. That leads to stress, which destroys their effectiveness.

Project Leaders and Project Managers perform different functions. Broadly, the Project Manager handles outward looking stuff including admin while the Project Leader looks inwards towards the well-being of the team and it’s work. I have always found that I am at my best when working back to back with a good Project Manager. In a situation like that, we really can use generic managers, who do not know about the field that they are working in. But without such a partnership, it is essential that the Project Manager is an expert programmer. Otherwise the manager has no way to judge a sensible idea from a bad one, and word games, buck passing and bombast inevitably grow without limit. As well as ensuring that stress grows, shutting down the whole team’s cognition, this is, in my view, the cause of the three Great Abdications - an inevitable result of trying to do the impossible!

The first Great Abdication occurred in the early 1990s, when managers who weren’t programmers, and therefore had no understanding of the work, abdicated their responsibilities to proceduralism in the form of perverse misinterpretations of ISO9001. When that didn’t work, in the late 1990s, they moved to the second Great Abdication. In this, they decided that uniquely amongst commercial activities, software engineering is best handled as a democracy, where the horde of JavaSchooled script kiddies all had equal votes to the minority of experienced people who could have led and taught them. As Richard Gabriel has observed, “It takes twenty years to get good at this stuff”, so of course that was a failure. The third Great Abdication was to ship the whole mess offshore!

Project Managers can directly contribute to the performance of a gelled team by controlling the commercial demands put upon the team. This does not mean keeping the demands low, because gelled teams actually need tough challenges to cohere around. The trouble is, every new demand is also a potential cause of stress. The team gets control, and the stress is reduced. Stress, demands and performance are related by a cusp catastrophe, described by Rene Thom. On the diagram, a low stress, low demand team is represented by the rear, right hand corner. If demand rises but stress remains low, performance reaches the high level at the left. But if stress then increases too much, performance can suddenly drop to the lower level and then it takes a lot of effort to get past the hysteresis and jump back to the upper level. Later I’ll discuss Embedded Figures Tests, which I have great hopes will be able to quickly and repeatably test for cognitive flexibility, and can even be automated. But I see these as most valuable for research purposes and in fields other than software engineering. For engineers, the step function in effectiveness produced by getting this stuff right is so huge it cannot be missed.

Next: What To Do Per Team