Tuesday, October 23, 2012

Horrible Bosses and Horrible Colleagues

If you've not read Dunning and Kruger's seminal paper Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments then you should absolutely go read that first, and then come back to this article. This missive explores the ideas of that paper further, especially as applicable to engineering organizations.

In pursuit of this goal, let us assume that you, whom we shall consider the paragon of competence, work for an incompetent manager. Unfortunately because of what is called the Dunning-Kruger effect, your manager thinks of himself as supremely competent; and thus, when in a technical argument with you, he considers his point of view at least as good as yours, if not better.

To further compound the misfortune, he usually overvalues his "experience" - the sum total of whatever he assumes he knows, has learnt and has distilled from the pure engineering consciousness that only he can peer into the recesses of.

How do you ever tell such a person that he is wrong on some technical topic? After all, he can always pull up how in his day, all he could play with was a PDP-11 into which he shoved in cards punched with his fingernails. Or how in his day, a multimeter was all that was needed to debug his fantastic circuits? And how you, young whippersnapper, who may have a command of Verilog not seen since Phil Moorby and Prabhu Goel, or a command of C not seen since Brian Kernighan and Dennis Ritchie ever measure up to his arbitrarily high standard of designing ripple counters with 74 series logic?

One suspects that you have been set up with such magnificent examples of irrelevant experience. You can never satisfy such a boss - because he is so full of himself that he can't really see beyond his nasal orifices.

Then there's the boss who while technically incompetent, is extraordinarily competent when it comes to people and to politics. He knows the ins and outs of the organization, who's being promoted, who got passed over, who's got which car, who plays golf with whom and so on. He can grease any wheel with a rapidity that makes you feel minuscule in comparison.

Such a boss expects you to play his games - if you fail to do so, woe unto you, and if you do, then you can kiss your technical competence goodbye, because so embroiled will you become in his machinations, that will be all you can think about.

If you see a pattern developing here, one congratulates that competence we credited you with at the beginning of this piece. Cutting right to the chase, as it were, such horrible bosses abound in the industry. It is a miracle that Moore's Law has applied for this long, because growing numbers of incompetent people in any organization must eventually spell its demise.

The difficult question is what you, the competent engineer, will do about it. On the one hand, there is that nagging question of having an income. On the other, there is the torture that so many engineering organizations excel at doling out. Angst such as this is what led to the creation of Dilbert, or the movie Office Space.

If you are in a situation like this, then you will want to know what to do about it, and if a solution to the problem exists. But we are getting far ahead of ourselves. We need to explore the incompetence theme a bit further first.

Firstly, we have to consider how the incompetent boss or colleague evaluates you - the competent engineer - and secondly we need to investigate how you evaluate them.

Per Dunning and Kruger, the incompetent do not possess the metacognitive ability to understand your skills. This means that in their eyes, you are no better than them, a corollary of which is that no matter how hard you try, you can't convince them of your skill. Again, per the same paper, you won't think them so utterly incompetent, and you'll likely underestimate yourself. Add to this the general feeling of nausea that the competent get when singing their own praises, and you have the worst possible situation: Incompetent bosses who think they are better than they really are and you worse than you really are, compounded with you more or less unable to disagree.

Now consider another scenario - say you aren't really that competent, but your boss is. In this case as well, you will be under-appreciated, but rightly so.

Looking at this entirely from your point of view, all that is visible to you is that you are under-appreciated. This bit of data - most likely true - doesn't reveal anything about reality, though. Similarly, looking at it from the boss's point of view, all that is visible is that you don't measure up - and this bit of data also reveals nothing of the real state of affairs. Both cases are like looking at the result of a logical-OR operation and trying to decide which inputs are in which state. It can't be done.

This is a bit of what is called a deadlock. Both parties are right and wrong at the same time, and no move can be made. Unsurprisingly, situations such as these end in separation; the engineer quits or is made to quit.

For the sake of completeness, there are two more cases to consider. You, the astute reader have already figured this out - we've done cases 01 and 10 - each bit denoting the competence of the boss and you. We have to do 00 and 11, that is the cases where neither is competent and the case wherein both are.

The case of 00, wherein neither your boss nor you are competent, is easy to dispense with - if such is the case, then this article is of the remotest possible interest to you or your boss, and vice-versa.

The last case is also easy to dispense with - it is a utopia that one hopes everyone will live in, but is nearly impossible to come across in real life. A priori, one might postulate roughly a 1 in 10,000 chance of this happening - or at least a low enough probability to not waste words over.

Among the two cases we first discussed, the most interesting case is that of the smart engineer with the um, thick boss. The case of the smart boss and thick engineer usually gets resolved easily enough - with a well deserved pink slip.

At this point we are ready to start thinking about what to do when you are a smart engineer who is living the nightmare. The key to solving the problem is very helpfully given to us by the very same paper, and nicely enough, right in the abstract.

The insight is that when the skill level of the incompetent is raised, it also raises their metacognitive ability to recognize their own lack of skill - which while seeming paradoxical even to the authors is actually absolutely logical and correct. This harkens back to that old truism that only the wise understand the wise.

And thus the solution presents itself - if you desire to stay on in the same organization, working with the same boss - and there may be pressing reasons for you to do so - then you must raise his level of competence enough to at least appreciate yours.

Naturally, this is no easy task, and you may even think that putting in so much effort may not be justified given that you can probably get a different job easily enough (you are uber-competent, right?) but while contemplating your choices, it is worth keeping in mind the abysmal probability of your new boss being competent enough to appreciate you. (Now there's a tantalizing thought - "Is it possible to identify a competent future boss during the interview process?" -  and as usual, we'll leave it open for some future post.)

And finally this missive must end with that all important question - you think that your boss is dense and that you are smart - and this article has only really focused on that 01 case - but how do you know this for sure? One can only answer with another old truism - "Know thyself". But seriously - and this is the actual essence of the Dunning-Kruger paper - if you are really smart, you'll find a way to prove it to yourself.

No comments:

Post a Comment