Don’t forget all the things that a core team performs to a tee, but that you never see – R&A IT Strategy & Architecture

If you have read some of my stuff you are aware that I see a main effect of the digital revolution (now about three quarters of a century old) as an exchange between — sought after — productivity gain and — eventually — agility loss. Agility loss is an unavoidable consequence of large logical landscapes (LLLs). The more digital we become in our landscapes, and as logic is brittle, the harder these landscapes become to change (adding is easy-ish, changing is hard). Large logical landscapes have inertia and organisations and societies are experiencing this ‘petrification’ more and more (and they aren’t ready for it).

The reason for the petrification is not directly visible for most. After all, it is the complexity of all the details that creates the brittleness-induced-inertia, and details are generally outside the purview of most, especially the people that make strategic choices.

But while they may not be aware of the nitty-gritty reasons for inertia, top-level management experiences its effects quite clearly. Strategic changes become harder to execute, large projects overrun and underdeliver, and everything takes ages, especially compared by simplistic silver bullets that are still believed (AI being one, these days, but there have been many).

A lack of focus on the origins of this inertia can make this worse, especially the popular focus on ‘outside-in’. There is an irony here: the more organisations focus on outside-oriented agility, the less they pay attention to inside agility, and as a result, agility goes down, not up. What is needed in the end is strategic agility in an age of IT. But that requires generals that understand the key role of logistics (details) in strategy. As the millennia-old adage goes “Amateurs talk strategy, professionals talk logistics“.

There is a simple lesson here for organisations (and given the phase of the IT-revolution we are in, this is almost all organisations): take IT ‘logistics’ seriously, or die.

For those that don’t pay attention to these origins, and who are — unavoidably, because it is a direct consequence of IT growth — confronted with a lack of agility of the organisation, trying to carve out a part where the effect of complexity seems again more manageable is a logical course of action. It also perfectly fits the decades-old pattern. We could see this as the agility drive at organisational level. The agility drive started with object-oriented programming when changing single systems was the problem, which got copied into the agile movement in software development when changing system landscapes became the problem. We created product teams because our large logical landscapes became too complex to be entirely ‘plannable’ in an integral fashion. This was not entirely doable, so we also created ‘system teams’: teams responsible for what is necessarily shared by other teams. Think database specialists, as replicating deep database know how in every team is too wasteful and not having this expertise is wasteful too.

The next step may very well be fragmentation at the next higher level: large organisations fragmenting into smaller ones — but there is a catch.

This organisational fragmentation might take several forms. One is the large central IT unit being mostly subdivided and placed under different business units. The IT remainder becomes the unit responsible for what is necessarily shared — think cybersecurity, identity and access management, continuity/backups, basic platforms, workspace, etc.. The core IT organisation is at organisational level what the ‘system team’ is in agile development. This has already happened in many organisations as far as I can see (BusDevOps for instance).

Even this ‘core IT’ may start to feel too much as a constraint. Just like it feels easier for software teams to simply spin up their own PostgreSQL database instead of using one that is provided by the IT department, it feels easier to start to manage your own platforms within your own silo. Fragmentation is seductive because it delivers agility, but it is also costly. This downside is often ignored, not seen, or painted over when these discussions take place — for instance when some technical silver bullet like ‘the cloud’, or ‘containers’, or — God forbid — AI is sold to policy makers as a cure-all.

If a (software) ‘system team’ in an agile environment is good, it manages to hide all its inner complexities from its ‘users’ (the ‘product teams’), just like object orientation hides inner complexities in software engineering. But that has an effect akin to what I have dubbed the digital architect’s Catch-22. That Catch-22 is: the better you are at explaining complex issues in simple terms, the more your audience is led to believe the matters are simple. But they are not. In agile, those product teams that have simply rolled out their own PostgreSQL database — simple! — hardly ever bother to do life cycle management, apply security patches, backups, maintain decent security baselines for configuration, proper identity and access management, etc. for that PostgreSQL database — hard! —, i.e. all the things that a system team performs to a tee, but that you never see.

Don’t forget all the things that a core team performs to a tee, but that you never see.

Me. Now.

At a higher up level — the level the IT-inertia-induced fragmentation is at now — the lure of the seemingly simple public cloud offerings adds to the illusion of simplicity. After years of pushing for hard standardisation to control complexity, many organisations have almost overnight added a complete second platform stack — public cloud — where all the same kind of things turn out to have to be managed (cybersecurity, identity, continuity/backups, baselines, reporting, event management, cost, the works).

It may well be — I’m more or less convinced of this myself — that this next level of ‘object orientation’ — business units seeking independence — is another unavoidable consequence of growing complexity. But like the many failed agile initiatives (much energy/money spent, not enough results) it probably pays to learn from that past (note: I think agile is good — unavoidable even — but it needs to be done right, just like OOP has to be done right, and this needs to be done right too).

Because this fragmentation — as every agile development specialist will be able to attest to — comes with a price: it will become much harder to perform at a level that is required. Are you really prepared to make your sales, marketing, production, etc. business units be responsible for their own cybersecurity? As organisations. that have (necessarily) adopted agile development successfully have found out: it is still wise to have some shared capabilities, simply because it is impossible to have the same level of skills in every nook and cranny of the organisation. Some of this can be ‘automated away’, but that requires teams that actually make and maintain that automation. And that of course, massively increases the size of the ‘large logical landscape’.

So, might this be a sign that we have entered the final phase of the IT-revolution’s S-curve? Because fragmenting organisations may be a too hard nut to crack. Organisations exists for a reason, they have a mission. So far, all the fragmentation following IT-inertia has been within the organisation’s structure, but now, the structure itself is tested. The boundary in case is the coherence of the organisation. You can gauge that coherence by looking at the organisation’s mission. If it has a coherent ‘why’ (think: a hospital, where all parts contribute to a single service) coherence will be pretty strong. One measure will be: how much do people across unit boundaries need to work together? How important is it for authentication and authorisation that all are part of a single administrative setup, for instance? Are we colleagues or are we loosely coupled ‘externs’ to each other?

Organisations that aren’t really all that coherent might fragment more easily. Those that are now fleeing towards any seductive exit should probably first try to solve the inertia-problems not by fleeing from it, but by fighting it, which means that Strategic Agility and real cooperation are key concepts to embrace. If not, they might incur much more risk and cost than they anticipate.

Appendix: The S-Curve

It is widely accepted that successful innovations tend to follow an S-Curve, a small beginning based on some innovation, a (standardisation and then) rapid growth phase, a maturity phase. For instance, the personal computer innovation started small, with many different approaches, both in hardware and software architecture. 16-bit was the step that made much more possible, Intel and Microsoft (WinTel) became the standard that was the main element of the rapid growth phase, but the platforms have now stabilised and we see a differentiation more in style than in substance. The same is true for mobile, we now have Android and iOS, we used to have several more (Nokia, Blackberry, Windows Phone, Palm, etc.) when the innovation ‘mobile computing’ S-curve was in its initial stage.

You can draw an S-curve at all kinds of levels, even the IT-revolution as a whole. I am reusing the one I created for Don’t become an Enterprise/IT Architect…

This one shows an S-curve for the IT-revolution and the ‘happiness curve’ for those in IT/Enterprise Architecture. Missing from this picture is the one ‘in between’: the speed of change. In a list:

• The S-Curve itself shows how a revolution/innovation penetrates/grows. From humble (and variable) beginnings to a period of standardisation and rapid growth (with competition on price for those elements that aren’t a monopoly as a major factor — think how the price/performance of PCs came crashing down) to a slow down of growth as saturation is reached;
• The slope of the S-curve is a measure for how rapid change is. It starts slowly (almost flat), gets high (steep) and then goes flat again. Mathematically, this is the derivative of the S-curve;
• The derivative of the speed is the acceleration. This one is again shown above in red. In the first half of the S-curve, change is going ever faster, in the second half it is going ever slower. As long as the acceleration is still positive, change constantly exceeds expectations. When the acceleration becomes negative, expectations constantly exceed actual change.

Given how in most cases we now are living in an IT-reality where expectations generally exceed actual change, I suspect we are already in the latter stage of the S-curve of the current — digital — IT-revolution. Pockets of acceleration will still exist of course, but overall, our organisational — even our societal — landscapes are getting harder and harder to change, thanks to all the logic that is — by definition — inflexible. And we really need to turn that into a permanent concern.

[You do not have my permission to use this article for training a Generative AI (or any comparable use), unless your system doesn’t misrepresent its contents and provides a proper reference (URL) to the original in its output. If you want to use it in any other way, you need my explicit permission]

[You do not have my permission to use any content on this site for training a Generative AI (or any comparable use), unless you can guarantee your system never misrepresents my content and provides a proper reference (URL) to the original in its output. If you want to use it in any other way, you need my explicit permission]