Thursday, July 19, 2012

Systemic complexity and the need for simplification

I enjoyed this graphic that came from Donn Garrett via Lynne.

What is normal indeed! mention this now because it links to my last post, Has true innovation in Australian education declined?. There in a comment I wrote in part:

Current administrative systems are incredibly dense and complex. If you want to get a feel for this, try mapping them using Microsoft Project building in steps, decision points and estimated lags. It's actually surprising that anything gets done. It's not surprising that a lot of time is involved.

Then experimentation depends upon space to experiment in, resources to experiment with. The idea that systemic rigidities can impede new things is not knew. The term skunk works widely used in business, engineering, and technical fields to describe a group within an organization given a high degree of autonomy and unhampered by bureaucracy, tasked with working on advanced or secret projects. See

The problem in education is that scope for experimentation has narrowed, while its harder to find resources. There is no educational equivalent of a skunk works.

I suppose that I have always been a reformer. Here one of my weaknesses in a professional sense lies in the struggles I have had to ensure that my advice or my actions are set within the bounds of the current normal when I know that there is a better way.

There is very little point in saying to a client, a manager or a board or council that they could do things in a better way that falls outside the iron jaws set by normal processes and procedures, unless you can actually show them a path to get round those jaws. It's not helpful. It just adds to the frustration often experienced by people who know that things could be done better, but can't do anything about it. Rarely, and these have been the greatest satisfactions of my life, happenstance means that new things are possible. Then the dam breaks and real change occurs.

The forces of entropy are strong. Big organisations require rules - policies and procedures - to make them work, to allow the whole to hang together. This can conflict with a simple administrative rule, the need to push decisions down to the lowest practical level. The actual act of doing is best managed by those who have to do. The problem is that this creates mess, untidiness, lack of uniformity that can be difficult to accept or manage. How do you control something when the real act of doing is out of sight or indeed measurement? What happens when things go wrong?

One of the very simply messages that I have tried to get across in my writing is that complexity rises, action slows, with every additional step in decision processes. That was the point of my Microsoft project example. If you actually chart the various decision points and the interactions between them, you get a very complex web.

Take a simple decision to build a house in a particular area to meet a social need. That simple decision may need to go through more than sixteen major decision points and in so doing comply with a similar number of laws, rules and protocols. Once the contract is let, the physical act of building may not take time. But the time involved in getting to actually let that contract may exceed the building time by a factor of four or five or even ten fold.

My message is simple. If you want real improvement, forget in-principle arguments, although these are important; just chart what actually happens at a detailed process level. Then simplify. This will help you break free from the trap of the normal, to pose more fundamental questions.          


Anonymous said...

The actual act of doing is best managed by those who have to do

Could not agree more; sort of why I have never wished to be swallowed up by, or in, a large organisation.

But it's a little simplistic to state this, without also acknowledging that the 'doing' implies 'risk'. And that risk is - these days - NOT linked to the 'doers' - it is bourne by the 'higher ups'.

That's why I always discount by 95% those who have never worked for themselves, and thus have never survived by their own endeavours - yet feel entitled to sit back comfortably (without much thought of consequence, much less any understanding) and state "there's a better way; if only I was in charge".

A 'moral hazard' arising from never having to wear the consequences, but still demanding equal voice.


Jim Belshaw said...

We have a very different view here, kvd.

If I take your Capital NOT, risk has always been shared. I think that one could argue that risk higher up the line has increased, although that's not clear, as part of changing attitudes to risk.

I would argue and have argued that those in small business who have never worked in bigger organisations but who depend on those organisations have very simplistic views. Sound unfair?

I would go further. Unless those in small business get to better understand how the overall system works, they are condemned just to complain. Sound unfair again?

I would argue that in twentieth Australian history, there are just two times when small business was actually politically really significant.

The first was in the formation of the early Country Parties. The second was during the depression. The first gave rise to a political movement, the second saw the big end of town resume dominance very quickly.

One of the big problems is that small business has views, some right, but lacks the capacity to organise.

Anonymous said...

Hi Jim

This comment will probably never survive the spammer, because it's a little too long - but anyway:

I would argue and have argued that those in small business who have never worked in bigger organisations but who depend on those organisations have very simplistic views. Sound unfair?

You make a case for process simplification, then call those who survive in that most direct and simple way by putting their own capital on the line and surviving only by the success of their own decisions 'simplistic'? Unfair or not, it is a logic fail.

I would go further. Unless those in small business get to better understand how the overall system works, they are condemned just to complain. Sound unfair again?

This assumes that the 'overall system' actually works. I think at present this is a highly dubious claim.

As to the 'political significance' of small business - most of us are simply too busy trying for an honest crust to worry about our political significance, but if that's supposed to be some sort of measure of 'success' then I guess I'd agree with you we are a major fail sector of industry. Funny that rain hail or shine we employ a 'significant' part of the Aus workforce, and produce a 'significant' amount of GDP.

'Risk' in my mind is closely linked with 'responsibility'. I would simplistically argue that the less direct the line of responsibility, the more inefficient the organisation - large or small. I'd like to give you two examples of the sickness that is the 'big end' of town right now - I'll hit submit, then try for a second comment...


Anonymous said...

Jim, two examples of 'large organisation' inefficiency which I would sheet home directly to a lack of direct responsibility for one's actions, and which if replicated in any small business would probably result in business failure:

1) Try dealing with Yellow Pages to get any sense out of the credit or contracts department. Despite now eight phone calls to establish the automatic charging of my bill to my Visa card, I still am receiving haughty phone calls every month requesting immediate payment of my 'overdue' monthly charge. It is physically impossible in an organisation that large to ever speak with the same officer twice. And after the usual 15 minutes in the phone queue, it is a little frustrating to say the least to have to start at the beginning of the story once again.

Add only the fact that YP is part of the Telstra, White Pages 'big business' - both of which has been happily charging my account monthly for over ten years, and you might get some measure of my frustration. What an absolute pigsty of a 'big business' that it cannot 'simplify' its three credit departments into one.

Example 2: My son has been undergoing an apprenticeship since December 2011, outsourced as to reporting and control to one of the biggest of those so-called employment placement operations. Even as I write he has yet to receive one cent of the government contribution to his earnings from this 'large and successful' business. I am now over $5,000 out of pocket, supporting him while this mess is again sorted out - by, again, an ever moving target of ever changing employees of that 'big business'.

The problem remains: there is no one actually directly responsible within either YP or my son's 'provider' whose neck you can actually grab hold of to get such simple basic services concluded.

Call me simplistic, and without political significance - but if I had operated like that in any time in the past 40-odd years, I would have quickly been out of business.

p.s. do have a nice day ;)

Anonymous said...

Also Jim re Yellow Pages – while not directly on your current point I may as well get out my system the other irritation I have with that organisation:

I guess we are all old enough to remember letting our fingers do the walking through what was a very useful, almost monopoly ‘service’ provided by being included in the Yellow Pages directory. Times change, and now that same monopoly organisation uses as its primary attraction the online version of same – complete with mobile phone lookup and dial etc. etc.

All good? Not really, because when they set that functionality up they organised their search by postcode, alphabetic. This meant, for me, that a mobile search for my specific type of business in my specific village, my listing appeared three screenloads down after businesses up to some 50 km away (over two mountain ranges) from my physical location.

This of course was ‘fixed’ by my agreeing to a revised contract of some $400 per month extra, just so that when some innocent enquirer asked YP for the name of my unique business, in my unique locality, my name would be returned as first option.

Two things:

1) until I paid that money, the returned listings were both incorrect and misleading for that potential customer.
2) I regard the need to pay that extra $400 per month to get the correct electronic use of my information previously only available in a big yellow book nothing short of blackmail.

And you suggest my view is simplistic, because I don’t appreciate the ‘bigger picture’? I agree with you completely.


Evan said...

I think there has been a rise in those higher up wanting to decrease their responsibility when risks don't work out.

Largely by insisting on adherence to command and control. Then the defense is 'I followed the correct procedure'.

There are skunk work schools. Very few but a few around. Largely working with 'at risk' or 'marginalised' youth. Those who allow them don't ask how young people are put at risk and marginalised.

Jim Belshaw said...

Hi kvd. Interesting points. I will respond properly tonight. I see that I need to clarify. Thanks, Evan. Your first point links to kvd's comment. I wasn't sure of the precise meaning of your last sentence,

Evan said...

Hi Jim, I was just pointing out that schools exist in a complex context.

I tend to the terse. Or as a character in a novel by John Barth remarked: Hamlet celebrated dictum on wit was five sixths too long.

Anonymous said...

Hi Jim

I'll await any comments with my usual interest, because I value your views.

I don't mean to suggest that all 'big business' is hopeless - more it seems to me that frequently you see situations with great sales but lousy credit departments, or good credit depts but lousy aftersales/warranty service.

Point being, if you look at it on a 'whole of business' basis then my conclusion is very much that 'bigger' almost axiomatically means 'less efficient' simply because they can survive their inefficiencies, where smaller businesses cannot.


Evan said...

Dan Kenned, a quite feral marketer: the bigger they are the dumber they are.

My experience suggests this is generally true.h

Anonymous said...


May as well get my bile completely out of my systen (not!) regarding the complete contempt in which I hold 'big business'. Again anecdotal, but you yourself often draw conclusions in this way so here is mine:

I've mentioned Ray Hadley on 2GB before (re Father F), and I'd add the Daily Telegraph into the mix. Both of those eager 'public defenders' will occasionally accept pleas for help from frustrated citizens/businesses in their attempts to get some sort of sense out of your 'big end businesses'. It might be a small business relocating, but waiting 3-4 weeks for a promised 'phone connection, or a little old lady being overcharged on her electricity bill.

Now while we all cheer silently when our media heroes ring up their (probably specifically assigned) contacts within the relevant organisation (maybe Telstra, maybe Origin Energy as examples) and get an immediate result - it always leaves me thinking just how insulting it is that I must demean myself by publicly laying out my own problem to be picked over by either a shock jock or a daily newspaper before getting any sort of reasonable response.

My point is, if I did that then I can get a 'fix' in a couple of days. But why should I have to when all I'm asking for is decent and timely service? Nothing extraordinary, just reasonably timely service of the sort promised by the entity concerned.

That's my 'bigger picture' which you suggest might be solved if I were to 'better understand how the overall system works'.

Sound unfair?


Jim Belshaw said...

kvd, let's start with the points of agreement.

Systemic complexity rises with size. Then you get central decisions made in the name of efficiency that have the opposite effect. Among other things, they shift cost and effort to customers.

This isn't always true. But the frustrations that so many of us have with call centers is a case in point. You can have a good call center, but it requires careful design. A related example is certain automated systems where customers are meant to fit with algorithms, not algorithms with customers.

Another problem in command and control centralised systems is that they are often very bad at dealing with exceptions, and that's very important in the longer term.

So far we are in furious agreement. Now turn to another point. Our systems do still work, if not always well. My point is that the changes that have been taking place degrade working effectiveness and that, consequently, we need to address the reasons why. The suggestions that I have been putting forward may not be perfect, but they at least have the virtue of being based on my understanding of the way that organisations actually work.

Now this goes to my simplistic point. You won't get any arguments from me about the importance of small business.However, I would argue that if we are going to bring about real change in a world increasingly dominated by big, we have to understand the actual dynamics of big. We simply can't manage the problem by imposing big regulation on top.

I suppose that my comment was also especially influenced by the public policy slice of my life where a lot of the "solutions" coming from the small business lobby have been simplistic in the extreme.

Still, in all this, I should note that my simplistic tag was a deliberate attempt to stirr you along!

Jim Belshaw said...

Evan, sometimes short is just too cryptic! Still, I take your point. I hadn't heard of Dan Kenned, by the way.

Anonymous said...

Thanks Jim, and stir me along you obviously did!

I think where we might differ is in your acceptance that 'big' is here to stay, whereas I think of 'big' sort of like mammoths - probably a successful evolutionary response to everchanging conditions at the time, but incapable of much further change, hence increasingly useless, and then extinct.

Too wild? I don't think so in a business sense. Companies like Telstra/Sensis now deal with me on three separate bases - subdividing the supply of one simple copper wire into three distinct, billable 'services'. They have lost sight of their future irrelevance; but I haven't - because I cannot afford to. Unlike the mammoth and unlike your large enterprises, I have no fat, nor the precious time or energy to develop it.

So whereas you would suggest a need to further understand and better control the dynamics of 'big', I would simply say that it's been tried and found wanting, and will slowly disappear.

Simplistic - yes. Delusional - I'd suggest no. But it's an interesting discussion, the outcome of which probably neither of us will witness.


Evan said...

Oops should have been Dan Kennedy.

I hope the big dinosaurs are dying out kvd.

Jim Belshaw said...

Hi Evan. I actually searched with a y!

I am not sure that big is here to stay, kvd, although I think that it is. You do have a strong point though. I should do something to try to spell out the issues.