Failure – the fertilizer of continuous learning

INTRODUCTION

Failure is a stigma – we prefer to avoid failure instead of seeking it out, controlling it and learning from it. In the modern world and especially the software industry where continuous learning is of utmost importance, a failure-avoiding attitude is fatal.

In the first part of this article we'll take a look at success and failure in general, related emotional costs, and how to turn failure around. These are the useful things you might hear from a generic success coach.

In the second part we'll examine the same thing in the context of thinking processes. In other words, we'll look at 'failure of understanding' instead of 'failure of doing'. Since these are mostly semi-subconscious processes, I will be using metaphysical imagery, analogies, and hazy concepts such as 'identity' and 'grokking '.

In the third part we'll examine if any of the insights from the first and second parts are applicable on an organizational and cultural level, especially for companies based on emergent leadership principles.

Be warned, this article is a lengthy one. If you're low on time or metaphysical musings are not your cup of tea, please skip directly to part three.

PART I – EMBRACING FAILURE IN GENERAL

Defining success in general

Success can mean many different things to many different people, but most definitions fall into a few major categories:

• Success as achievement of goal — e.g. a completed task.
• Success as state of being — e.g. a prosperous entrepreneur.
• Success as the image of success — e.g. a well-branded product.

The first definition of success is the most straightforward one. It implies that there is a goal, a process by which an attempt is made to achieve that goal, and a way to determine the outcome.

The second definition sees success as a state of being: one is successful if one's set of goals are fulfilled more often than not, in a certain time-frame.
Unfortunately, goals are seldom clearly defined. A large set of hazy goals is nigh impossible to communicate, so it's often assumed that the goal is financial wealth, with stuff like political power or fame thrown in – or you know, not dying, depending on who you talk to.

Because of the difficulty of clarifying and communicating goals, society sees success as a reflection of standing in society. Thus, we come to the somewhat nonsensical definition that success is the image of success.

Defining failure in general

Sticking to the straightforward definition of success, it's relatively clear what failure is: the stated goal has not been fulfilled.

In the strictest sense, the reason for failure is always the mismatch between expectation and reality: the mental model we had for getting from A to B was proven inadequate. The world is complex and dynamic, and as long as the mental model of the world is distinct from the world itself, this mismatch will be present.

The sensible way to deal with failure is to accept that it happened, identify the major reasons for the mismatch, and adjust the mental model and its context accordingly – or give up if the cost proves to be greater than the potential reward.

By acknowledging that the risk is inherent (since a mismatch always exists), we can attempt to map out that risk and manage outcomes instead of failures. An analytical approach goes a long way for turning failures into successes.

However, we are only human, and our successes are generally our greatest failures – we make mistakes during the execution of our vision, but we also make mistakes during the goal-setting phase. Most of our stated goals are just proxy goals for unclearly defined, higher-order goals. Have you ever wondered what the meaning of life is? Well, then you have this problem .

Accepting the fact that we don't truly know ourselves, failure becomes a tool for self-exploration.

Emotional cost of failure

An outcome which we perceive as failure incurs a certain emotional shock.

This is a feature, not a bug. We are not just observers; these are our goals, our processes, our successes and failures. Overcoming the shock of our limitations is how we bridge the past with the present and how we grow.

Infants are perhaps the purest examples of this: when a baby experiences failure, its reflexive solution is to burst out crying. Now, this explosive solution is very powerful. It obliterates the world (or rather, the window to the world which presented the failure) and clears the slate. When the bad perception is washed away, all is right with the world again – until a new goal disturbs it.

In my opinion, adults don't do this kind of emotional release enough. Sure, the adult way is to take that initial emotional shock and use it as an impetus to change the associated mental model, but the necessary emotional cleanup is often omitted. Thus, the emotional damage compounds and festers, beginning a vicious circle: subsequent changes to the mental model are made not with the stated goal in mind, but to minimize future emotional damage.

With fear as the primary motivator, doubt takes over and failure becomes a state of existence. Since the goal is no longer 'success' but simply 'not failing', there is no vision of a successful state at the end. This results in wear-and-tear on personal identity and hastens burnout .

To make things worse, society keeps looking over our shoulders, making sure we don't run out of anxiety, and reminding us that the image of failure is failure itself – so best not even try.

Turning failure around

Addressing the emotional component directly instead of ignoring it helps us see more clearly. An emotional shock or hurt pulls our attention away from the larger context and focuses on the now – very useful if a predator is chasing you in the woods, but quite detracting for most modern problems.

Controlling the distance of emotional investment using self-discipline is quite a useful skill. The emotional impact doesn't have to come as a surprise – it's better to be prepared and consciously accept the risk. Instead of interpreting failure as defeat, we can choose to see it as success-in-making.

There's no way around feeling like a failure sometimes, and I believe it is important to embrace failure-as-a-state too. When you do, you move from a state of panic and disorder where you mindlessly flail around wasting energy trying to escape, to a calmer state which is tolerable and within the domain of your control. This frees up emotional and intellectual capital to be risked and invested elsewhere. Simply said, embracing failure is empowering. If you don't tie your identity to failure, you become more stoic because you're no longer just knee-jerk reacting to the outside world.

This is a wholesome state of emptiness. A state of expecting, but not of expectations. Also a relief, because you no longer have to forcefully prop up a one-sided, static view of the world. This state of emptiness is often described as a hunger, which serves as a source for ambition and drive, but it is also a space where you are free to find the creativity and wisdom for making truer choices. For higher order goals of self-actualization, visiting this place often is advisable.

As for that pesky third definition of success where society partially defines it with unclear goals, there's not much to do. If the image of success is success itself, there's a lot of faking to be done…

And the first approach is exactly that: 'fake it till you make it' and 'pull yourself up by your bootstraps'. You'll have to fake it even after you make it, and the danger with this approach is that the fakeness sticks and becomes part of your identity. Since this is pretty much an unsolvable catch-22 situation, accordingly the second approach is to reject the dichotomy, master the subtle art of not giving a damn and grow a backbone. The drawback is you have to be okay with showing vulnerability.

In reality, a combination of the two approaches works well. Lasting fulfillment comes only from self-actualization, but external validation can be a nice-to-have part of it.

PART II – EMBRACING FAILURE IN THINKING PROCESSES

Now that we have a few ideas about success and failure in general, let's see if we can apply them to the mental process of learning itself. In other words, what happens if we change our focus from 'doing and failing' to 'understanding and failing'?

Both are levels of learning, but on this intrapersonal level we have a lot more power, since nearly the whole process occurs within our being.

Defining success in understanding

Attempting to understand something can be likened to shifting one's view around until the last puzzle piece clicks into place.

This leads to a pleasant and cathartic 'Aha!' moment; the world was disturbed by the attempt but is now better for it. It is a fulfilling experience – something meaningful was achieved by one's own effort, and the learner's identity changed a tiny bit in a positive way.

When learning for knowledge, the same process is to be repeated as often as necessary, letting it take up more mindspace. When learning for deeper understanding, the process is to be repeated using different views and pathways and parameters, making the whole mental image more stable and robust. This approach is actually more effective for knowledge too, with the side-effect of raising more questions.

And if the thing is really complex – well, have the self-discipline to keep at it! Sooner or later you'll achieve mastery, having an arsenal of puzzle pieces and powerful patterns for viewing them, as well as habits for obtaining new ones – and realize you have improved yourself in the process.

Let's use the puzzle analogy to define the holy trifecta of learning:

• Knowledge — The puzzle pieces.
• Intellect — The efficiency of shifting views of puzzle pieces around.
• Wisdom — Choosing which puzzle image will be visible.
• Identity — This one is implicit. The learner and process as a self-exploratory whole; grokking .

Society can catch a reflection of these inner processes, and in theory values knowledge, intellect, and maybe even wisdom highly.

Defining failure in understanding

Since most of the learning process takes place within our heads, the difference between failure and not-yet-success is generally not a distinction worth noting. If we regard the process as a basic algorithm of mentally trying new things until something works, we have only a few states to potentially regard as failure:

• The last puzzle piece hasn't clicked yet.
• We got stuck in a loop.
• We changed our minds and terminated the process.

This is not too insightful. We are only human, and for failure to have meaning in this context, we must know the 'why'.

In the short term, mental fatigue is a common reason. Time constraints, cost-effectiveness, prioritization, lack of information, changed goals etc. are also realities we face. It may be useful to point out such a constraint as the cause of failure in order to evaluate it in a larger context.

In the longer term, sustained emotional damage is the main culprit in permanently harming the learning mindset and attitude of a person – this definitely deserves the 'failure' label.

Emotional cost of failure

Thoughts flow. Movement and change and perspective are built into how we think, and this generally doesn't cross the threshold of being perceived as failure. For small almost-failures, there's no cost other than a tiny amount of decision fatigue.

As the process lengthens, the ups and downs become more pronounced. Prolonged or frequent downs lead to annoyance, which may turn into frustration: Why is the thing not the way I want it to be?!

Indeed, why won't the mountain come to Muhammad?

If we insist on the outer world changing but refuse to change ourselves, we hit a wall of our own making. We look for the lost key not in the darkness where we lost it, but under the streetlight where it's more comfortable. Our thoughts bounce back, and with time settle into negative self-reinforcing patterns – our own efforts pull us under like quicksand.

Such deadly loops do not come into existence by mistake; each hides a Muhammad yelling in frustration at the mountain.

When this kind of anti-mastery solidifies, you become convinced that thinking in particular directions is painful. So to protect yourself, you carve it out of your identity, renounce it, and purposely forget about it. It becomes a source of anxiety and a blind spot in your mental vision. Argue for your limitations, and sure enough they're yours.

Thus failure becomes a permanent state: you close off the world in an attempt to keep your identity from changing, but the universe moves beyond you .

Even if you kept your annoyances and frustrations in check, avoided building walls and getting stuck in loops, and handled the mountain with appropriate respect – the dangers are still not all gone. You may be flirting with impostor syndrome on a daily basis in order to maximize your learning potential.

Let me switch to another familiar analogy, that of learning as an expanding isle. Your island of knowledge expands as you learn, and the shoreline connecting the known and the unknown grows longer too. The more you know, the more you know what you don't know.

Though your intellect delivers the desired speed, you might not be giving your identity enough time to adjust; the deeper processing of knowledge to become part of yourself takes time. Impostor syndrome is when your island of knowledge expands faster than your identity can catch up.

There is a certain feeling of compulsion at play: feeling inadequate leads to acquiring knowledge faster, which leads to an expanding shoreline with the unknown, which leads to even greater anxiety and feelings of inadequacy. Collapse is inevitable.

On the flip side, the Dunning-Kruger effect is what happens when you stop looking outward and shrink your island to protect your identity from failure. If you push this process to the extreme, you become the narcissistic blind king of your small, closed-off world… until you get hit by a bus.

As mentioned before, society claims to value knowledge, intelligence and wisdom – but measures it mostly through displays of financial wealth and status. It influences the learning attitude and process of the individual, inflicting potentially quite deep emotional wounds. Kids have it the worst in this regard, since their identities are malleable: society tells them they're stupid and they believe it, letting it become a self-fulfilling prophecy. Or society tells them they're smart, which puffs up their egos and they get stuck at the first local maximum. For many, the school system pretty much kills the learning attitude – how many people do you know who hate math? That hate didn't get there on its own.

Though learning and the inherent failure have potentially significant emotional costs, not learning has the higher opportunity cost of giving up self-exploration: one becomes a bit less alive.

Turning failure around

One approach to turning failure around is to improve the mental efficiency of the learning process, so that fewer hiccups get interpreted as failure. Here are a few tips and tricks to that end:

• Low on knowledge? Improve the external flow of information or take the time to come up with puzzle pieces of your own. Low on intelligence? Maybe you're just juggling too many pieces; cut down to speed up.
• Observe your own learning processes and learn from them. Build up mental habits of self-analysis, even though they might reveal things you don't like. Truly learning how to learn comes only through such meta-cognition.
• You're a verbal thinker? Go learn a few more languages to have a more flexible mental map. You're a visual thinker? Use synesthesia to open up new pathways of thought.
• Put things into context. Think in systems and don't. Be mindful of unnecessary context-switching, because the cost is high.
• Don't fill your head with junk. If you must, prioritize accordingly and do regular mental housekeeping. Physical possessions too take up space in your mental catalogue.

Which brings us back to the topic of emptiness. Brute learning power is nice to have, but since it is sustained emotional damage which causes deeper failures, the other approach is to address the emotional side of learning.

As we've seen in the previous section, repeatedly failing to understand something can be a blow to the ego. Nonetheless, you can choose not to interpret the blow as an attack. The trick is to be aware of your emotional response, and since that is completely within your domain, to adjust it as deemed appropriate.

Make effort to embrace the state of not-yet-knowing. Knowledge is just an attempt to hold the universe still, and Archimedes' lever works on thusly affixed points only in a very limited fashion.

The emptiness required has to be consciously cultivated within. It is preparedness, readiness, and the calm before the storm. It is zen; an unborn universe, and still water flowing. Not-knowing is a natural baseline state, which is disturbed by the lesser acts of understanding and knowing. Such acts are an exception – to be disposed of once their purpose is fulfilled.

Now, it is important not to confuse such a state of emptiness and tabula rasa attitude with ignorance. Ignorance is not not-knowing; it is the state and act of knowing little, and walling off the world in an attempt to keep it from moving. Contrary to that, the zen emptiness is used as a springboard into the world, fully embracing it into one's identity.

Getting rid of mental junk is a good start for cultivating emptiness, but to really kick it up a notch, follow the advice of the Queen in Alice's Wonderland: make a habit out of imagining impossible things ! Paradoxes and impossibilities at the mental level only serve to point out our own rigid and brittle deficiencies.

When suffering from impostor syndrome, spend time cultivating emptiness to help avoid panic attacks. Rest. Let it simmer. Less learning, more grokking. Self-discipline helps with embracing emptiness more fully, by not letting failure and skewed comparisons to others erode your self-worth.

Since you managed to read this far, Dunning-Kruger probably doesn't affect you much, but you might recognize some of its traits in your thought processes. Are you so sure of your expertise that you gave up on curiosity and stopped learning? Again, the solution is to embrace failure – or at least reject it less strongly.

No matter how far one goes, one cannot map out the universe through intellect alone. Understanding only makes a lesser copy, a reflection of it, and knowledge renders it static. Understanding yourself is the most complex subject – mirroring effects might drive you to insanity if you cling to your ego too tightly, but this is generally a worthy problem to have.

PART III – EMBRACING FAILURE IN ORGANIZATIONAL LEARNING

The software industry seems to be doing well with embracing failure on a methodological level. After all, we understand the value of short feedback cycles; of failing fast and often.

In my opinion, the defining point of 'agile' is whether failure is truly embraced or not – you can't say you value the things on the left more than the things on the right otherwise. If failure is truly embraced on a cultural and organizational level, the organization will fail controllably, insightfully, and cost-effectively all the way to success.

However, some things get lost in translation. A fail-fast methodology is useful, but it is often at conflict with the underlying mindset of people, either on a personal or political level – and then it becomes an organizational problem. Using an agile methodology without a culture of embracing failure is a recipe for disaster. For many companies, this leaves the whole 'agile transformation' thing in a weird spot. You can't really do agile without emergent leadership , either.

Methodologies have a nasty habit of turning into formalized, dogmatic processes, maybe even to the point of inducing religious zeal. When this happens, the process itself becomes the goal. This can happen with agile methodologies too. Do your dailies feel like they're about justifying your own existence? Well, then the whole thing took a wrong turn somewhere.

Blame-seeking makes the original problem much worse; it can turn up in many different ways, and wherever it turns up, it points to a serious underlying aversion to failure. A toxic or otherwise dysfunctional culture spreads this aversion like rot, and what rots away is the true foundational value of a company: trust.

Cultivating a blameless culture may not be easy, but at least the basic tenet is clearly understood: if a person failed, it's because the organization and its processes failed the person, and the sole blame and responsibility lies with the corporate person. We can maybe make an exemption for intentional sabotage or the occasional creative incompetence, but adjusting organizational processes in response is not always worth the effort, since overdoing it may jeopardize trust and create overhead.

Continuous learning is a defining part of the software industry, and cultivating a blameless culture is also important because of how blame factors in with impostor syndrome and personal learning in general. Doing otherwise is tantamount to instructing the employee to learn continuously, but never show the continuous failing necessary for the learning process.

We've been conditioned by society to avoid showing weakness or failure, so it makes sense even for a company with a neutrally blameless culture to exert additional effort in turning failure-aversion around. Here's a bit of food for thought: Would your company consider failure as a reason for celebration? If not, why not?

We are lucky (or unlucky) that the more extreme sufferers of Dunning-Kruger seem to go into politics, but this effect applies to all of us to some degree, and it influences corporate culture accordingly.

Though we're all experts, an expert who knows everything is someone who stopped learning. For a person with such a 'false expert' attitude, inflating the ego is more important than learning. This attitude contributes to elitist culture and causes bottlenecks. For such a person, being a bottleneck is desirable, since it offers validation and a bit of power. In a way, the Dunning-Kruger effect amplifies the organizational problem of silos and interfaces, with the worst case of creating small fiefdoms. Nevertheless, the effect is beneficial in controlled amounts, since it is a healthy part of decisiveness, confidence and assertiveness.

On a similar note, methodology can't replace creative and visionary thought. Though most of our work requires analytical thought, creativity is still a significant part of what we do. Creativity requires time, physical space, mental space, the kind of expectant emptiness as discussed in the second part of this article, and it has other challenges as well.

Now, to make the jump from analytical and creative thought to visionary thought, the self-exploratory aspect of learning is needed, since it provides the deep motivation and identity-changing grokking to see not only the big picture, but also the holistic picture in its dynamic nature. As Saint-Exupéry's little prince learned, it is only with the heart that one can see rightly .

Companies should fully support all three levels of thought and learning, for entirely selfish reasons: analytical learning makes better experts, creative learning makes better communicators and solution architects, and visionary learning makes better emergent leaders. (Remember, emergent leadership requires a degree of buy-in to the company identity.)

Since emergent leadership is about creating a bridge from personal responsibility to corporate responsibility, as well as personal identity to company identity, this is the true cornerstone where company identity and culture can be truly defined. Putting up desired company values and mission statement on a piece of paper – though a good start – can only be the tip of the iceberg. Values become true only when lived, through full support of the learning processes outlined in the first two parts of this article.

If we look at a company in a larger context, we can see that it is subject to external constraints and influences as well. External stakeholders have probably not embraced failure as an integral part of learning, and communicating the failures necessary for healthy self-exploratory processes without doublespeak might pose quite a challenge for stakeholder management.

It helps to look at a company as a dynamic, living entity. Such an entity too learns on different levels, and establishes its own identity through self-exploration. Entities with healthy learning processes become alive and evolve to thrive on the economic battlefield, while others wither into deformed shapes and die.

CONCLUSION

We examined failure and learning through three differently tinted lenses: failure in doing, failure in understanding, and failure in organization.

Failure-avoidance is not just a personal attitude or mindset; it is also a cultural and organizational paradigm. It stems from fear of failure, and causes numerous personal and organizational problems.

The only true solution is to adopt an attitude, paradigm and culture of embracing failure, which enable mastery over it. In doing so, failure ceases to be a stigma; it becomes a tool of self-exploration, rather than the emotional and sociopolitical sledgehammer it's quite often used as.