The application of evidence based approaches 2 - A note on evidence based policy

This brief note continues my discussion on what are called evidence based approaches. At the start of  The application of evidence based approaches 1 - Evidence Based Medicine, I wrote:

The discussion on Monday Forum - more on evidence based approaches looked at some of the issues associated with evidence based approaches. One of the issues from my viewpoint in that discussion lay in the need to distinguish between the importance of evidence and the application in practice of what are called evidence based approaches whether in medicine, management or public policy. I thought therefore that it might be helpful, at least to me, to consolidate some previous writing on the topic, starting with evidence based medicine.

I have repeated these words to emphasise that I am concerned with the application of particular models, ways of thinking. The Wikipedia article on evidence based public policy is not especially good, but it does refer to the transmission of the particular form of the idea from  medicine to public policy.

A good expression of the idea in an Australian context comes from this Australian Public Service Commission piece, there is no date but I think its 2009, featuring Gary Banks. In her introduction, Public Service Commissioner  Lynelle Briggs  states:

Evidence-based policy-making, while not a new concept, has recently become more prominent in public debate in Australia. The Prime Minister has called it a key element of the Government’s agenda for the public service. He wants policy design to be driven by analysis of all the available options, and not by ideology. This explicit endorsement by the Prime Minister provides us with a valuable opportunity to advance the cause of evidence- based policy-making in the APS.

In Australia, the potential application of evidence based approaches drawn from medicine to policy dates back to at least 1998. As happened with me, it seems to have spread first from medicine to the idea of a discipline of practice and then beyond. 

Mr Banks and the Productivity Commission have played an active role in popularising the application of the concept.in the public policy sphere, claiming lineal responsibility for its application back to the old Tariff Board. In fact, the concept is much older than that. The economic historian C R Fay used the term evidence based public policy in 1919, and by then the approach was quite old. 

The differences between the old and new approaches is that evidence based public policy now involves the application of a highly structured top down model that, to my mind, is riven with potential contradictions that arise from its misapplication. 

I recognise that I have to argue this, I am as prone as anyone else to management fads and fancies. Someday I should write a mea culpa post looking at all the things that I have argued for that I then found did not work in practice. Because I have been a practitioner  working across a number of fields, I have found myself badly bitten by the very things I had espoused. The problem, to my mind lies not in the concepts as such, but always in their sometimes blind misapplication.

In my next post in this irregular series, I will look at evidence based management.

Postscript

I should note, for the record, that I have a very high opinion of the work done by the Tariff Board though its various guises to the Productivity Commission today.

Postscript 2

This link came from Nicholas Gruen: The Trouble With Scientists.There are several points within it that bear upon our current discussion so I wanted to record it for later use.

   .

The application of evidence based approaches 1 - Evidence Based Medicine

The discussion on Monday Forum - more on evidence based approaches looked at some of the issues associated with evidence based approaches. One of the issues from my viewpoint in that discussion lay in the need to distinguish between the importance of evidence and the application in practice of what are called evidence based approaches whether in medicine, management or public policy. I thought therefore that it might be helpful, at least to me, to consolidate some previous writing on the topic, starting with evidence based medicine.

By way of background to what follows, one of my major interests over the last quarter of 2006 and the first quarter of 2007 lay in the development of what I called a discipline of practice, a discipline focused on the way professionals practice their profession. This drew me into a discussion of evidence based approaches, starting with evidence based medicine.

The material that follows in this post consolidates the three posts on evidence based medicine that I wrote at the time. For time reasons, I haven’t attempted a complete edit, just some simplification. In later posts, I will look at the application in management and public policy.

INTRODUCING EVIDENCE BASED MEDICINE

Evidence-based medicine (EBM) is an attempt to more uniformly apply the standards of evidence gained from the scientific method, to certain aspects of medical practice. Specifically, EBM seeks to apply judgements about the inductive quality of evidence, to those aspects of medicine which depend on rational assessments of risks and benefits of treatments (including lack of treatment). According to the Centre for Evidence-Based Medicine, "Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients."[1] Cited from Wikpedia.

In this post I want to continue my discussion on the development of a discipline of practice, a discipline focused on the way professionals practice their profession, by introducing in a preliminary way the case of evidence based medicine.

I first came across the concept in 1998 when I started as CEO of the Royal Australian (now Australian and New Zealand) College of Ophthalmologists. My instinctive first reaction was to say that's odd, I thought that all medicine was evidence based. In fact, that's far from true.

To understand this, we need to look at the way in which doctors are trained, as well as the way in which new medical approaches develop.

As with all professions, training starts with the previous body of knowledge relevant to that practice. For practical reasons, much of this has to be taken for granted. The trainee professional simply has to learn those elements required to begin practice.

The trainee then has to learn to apply that knowledge in practice. In the case of medicine this is done especially in hospitals working under the supervision of a qualified doctor who passes his/her knowledge and experience onto the trainee while checking their application. Again, the trainee is acquiring the current wisdom.

These broad processes continue throughout a professional training that can, in the case of medical specialists, extend as long twelve or thirteen years.

Once the doctor begins practice, he/she continues to learn from experience with patients. Doctors also learn through contact with other doctors and are expected to maintain a program of continuing professional education to keep them in touch with latest developments. Similar approaches and programs apply in other professional areas.

None of this will seem strange to any professional from any discipline. Yet in the case of medicine the process has proved to be seriously flawed. Evidence based medicine attempts to address these flaws.

OVERVIEW OF EVIDENCE BASED MEDICINE

Evidence-based medicine (EBM) is the integration of best research evidence with clinical expertise and patient values ... When these three elements are integrated, clinicians and patients form a diagnostic and therapeutic alliance which optimizes clinical outcomes and quality of life. Centre for Evidence Based Medicine

While this quote focuses on medicine, it also captures four key elements relevant to all professions and professionals.

The first element is research evidence, essentially what works and why.

The second element is professional expertise, our capacity to understand and apply our professional knowledge in the circumstances of the particular case.

The third element is the patient or client, each with their specific attitudes and needs.

The final element is the integration of the first three elements - the diagnostic and therapeutic alliance - to provide the solution that best meets client needs.

Before continuing my analysis I should note that the Australian Broadcasting Corporation's Radio National has just completed a rather good two part series - Facing the Evidence - on evidence based medicine. The first part is available here in transcript, the second here.

Returning to my theme, when you look as I did in my last post at the standard way all professionals are trained, you can see that that training focuses on the transfer of existing knowledge and skills to the new professional, knowledge and skills that that professional then applies. The professional then builds on this base through practice, at the same time using various professional development activities to try to keep in touch with new developments.

But what happens if that existing knowledge base is in fact wrong? How might this arise? To quote from the first part of the ABC program:

Every day doctors and other health professionals use treatments that are harmful, or fail to use therapies that have been proven to work. In the US there is so much medical error that Congress has directed the Institute of Medicine to develop a strategy to improve the quality of care. In its initial report the Institute noted perhaps as many as 100,000 Americans die every year from medical errors, including the use of inappropriate treatments. That's much more than from car accidents, breast cancer or AIDS. Many more suffer side effects and unnecessary costs

How could this happen?

Part of the problem here lies simply in the placebo effect, that fact that patients respond just to the fact of treatment. So the treatment appears to work, thus supporting the original judgment.

Part of the problem also lies in the fact that individual outcomes can be affected by so many variables and over a considerable time horizon so that the fact of adverse outcomes may not be clear in an individual case or, if clear, may be due to a whole variety of factors external to the treatment itself. There is a linked issue here that relates to the size of the population.

Given that individual outcomes vary greatly, the fact that there is a problem and its scale and scope may only become clear if you look at a population as a whole. That is, the individual professional may have no easy way of detecting the problem in his/her individual cases.

A further problem lies in the nature of the models used.

All professions use models to try to explain a complex world. In economics, for example, models are a common method used to analyse economic behaviour and to suggest possible responses at firm and public policy levels. In the case of medicine, biological models are common.

The problem with all models is that they involve selection of key variables and the specification of relations between those variables. Get either wrong, and outcomes may be very different from those projected by the model.

Doctors have always been concerned about adverse or unexpected outcomes.

In 1972 Professor Archie Cochrane, a Scottish epidemiologist published Effectiveness and Efficiency: Random Reflections on Health Services. This plus Cochrane's subsequent advocacy caused increasing acceptance of the concepts behind evidence-based practice. Cochrane's work was honoured through the naming of centres of evidence-based medical research — Cochrane Centres — and an international organisation, the Cochrane Collaboration.

The explicit methodologies used to determine "best evidence" were then largely established by the McMaster University research group led by David Sackett and Gordon Guyatt. According to the Wikipedia article on evidence based medicine, the term "evidence-based medicine" itself first appeared in the medical literature in 1992 in a paper by Guyatt etal. (Guyatt G, Cairns J, Churchill D, et al. [‘Evidence-Based Medicine Working Group’] "Evidence-based medicine. A new approach to teaching the practice of medicine." JAMA1992;268:2420-5. PMID 1404801)

From this point, the spread of the concept and its subsequent inclusion in professional training was rapid.

As with any other approach, evidence based medicine has its own methodological problems. However, it also has important lessons for other fields of professional practice.

PROBLEMS WITH EVIDENCE BASED MEDICINE

In my last post on evidence based medicine, I suggested that professional training focused on the transfer of existing knowledge and skills to the new professional, knowledge and skills that the professional then applied. The professional subsequently built on this base through practice, at the same time using various professional development activities to try to keep in touch with new developments.

I then posed the question what happens if that existing knowledge base is in fact wrong, looking briefly at the reasons why this proved to be the case in medicine, a discovery that had led to the development of evidence based medicine. I concluded that, as with any other approach, evidence based medicine had its own methodological problems. However, it also had important lessons for other fields of professional practice.

The Quality Movement and Quantification

In a series of posts on my personal blog, I explored some of the changes that had taken place in public administration since the war, looking at the influences on those changes.

In one of those posts I looked in part at the way in which standards, the Quality Movement and the importance of measurement had become major global influences. I also suggested that the outcomes here had not always been positive.

Evidence based medicine forms part of the global standards and quantification revolution and suffers from some of the same weaknesses. These weaknesses need to be recognised.

Problems with Evidence Based Medicine: Perception Bias

The first problem can be called simply perception bias.

Evidence based medicine is neither value nor perception free. The questions selected for test and evaluation, a process that can be very expensive, are influenced by prevailing views. Valuable alternatives may be excluded simply because they fall outside conventional wisdom. As evidence based medicine becomes the dominant mode, the effect may, as it has been in other areas, to actually narrow fields of investigation and action.

This links to a second problem, one that I have discussed before, the tendency for all professions to see answers within a frame or world view set by their profession.

A lawyer will give you a legal answer to a problem, a doctor a medical answer. If you have a back problem and see a surgeon, he/she is likely to think about surgical solutions. Go to a chiropractor with the same problem and he/she is likely to recommend spinal manipulation. So professional background helps determine the way the problem is defined, the solution applied.

This flows through into the application of evidence based approaches because the things tested are generally set within the frame of the tester. So evidence based medicine focuses on the efficacy of medical treatment and may leave non-medical alternatives aside.

Problems with Evidence Based Medicine: Causation

As part of my history honours year in my first degree I studied philosophy of history under Ted Tapp. Ted was a reflective man who required us to think about, to debate, the philosophical underpinnings of science and scientific method.

One core problem was the difference between correlation (a and b) as compared to causation (if a then b).

This problem applies in evidence based approaches. Just because a study shows an apparently strong relationship between a treatment and positive patient outcomes does not necessarily say anything about the causal relationship between the two. This has to be deduced and further tested.

Problems with Evidence Based Medicine: Problems of Epidemiological Studies

By its nature, evidence based medicine deals with large groups, populations.

As trials become larger and more complex, it becomes more difficult in statistical terms to establish significant relationships, to separate the effects of different variables. This creates another problem, the establishment of a clear relationship between the outcomes of trials at population level and subsequent application at individual level. As the Wikipedia article notes:

Critics of EBM say lack of evidence and lack of benefit are not the same, and that the more data are pooled and aggregated, the more difficult it is to compare the patients in the studies with the patient in front of the doctor — that is, EBM applies to populations, not necessarily to individuals.

This can create very real difficulties for individual clinicians, leading Tonelli to argue in The limits of evidence-based medicine that:the knowledge gained from clinical research does not directly answer the primary clinical question of what is best for the patient at hand.

Tonelli concludes that proponents of evidence-based medicine discount the value of clinical experience.

Problems with Evidence Based Medicine: Impact of the Observer

Another problem with evidence based medicine, one often seen in all evidence based approaches, is the way the observer affects the observed. This happens at several levels.

The first problem is that the simple act of participation in the trial may have some and not clearly seen impact on individual outcomes. In medicine, this is usually managed by use of a control group using a placebo. The efficacy of the treatment is then measured by the difference in outcomes between the control group and those receiving the treatment.

A second linked problem is the impact on patient behaviour of the trial itself. By their nature, clinical trials are closely managed. This means that patient compliance with the treatment routine is likely to be high.

This need not hold in subsequent clinical use since ordinary patients are more likely to fail to follow treatment processes by, for example, failing to take medication exactly as prescribed. This means that actual patient outcomes may not be as good as the trial results.

Problems with Evidence Based Medicine: Limitations in Application

A further problem is that the most rigorous gold standard approaches dictated by evidence based medicine can only be applied in narrowly defined circumstances, leaving a range of medical approaches that have to be tested by less rigorous means.

This should not matter so long as the limitations are recognised. In practice, it risks introducing two distinct distortions into the medicine and the health system. The first is the risk that investigation may be biased towards those things that can be measured through more rigorous techniques, reducing thought and investigation in areas less amenable to measurement. The second related risk is that treatment itself may become biased.

At clinician level, this links back to my earlier point about perception bias. Doctors trained in evidence based medicine may, consciously or unconsciously, come to focus in treatment terms on those things that can be measured, ruling out other less easily measured options.

This tendency may be reinforced by actions from those managing or funding the provision of health care services who may refuse to allow/pay for certain types of services notwithstanding the views of individual clinicians.

Conclusion

These problems do not detract from the potential value of evidence based medicine, but they do illustrate the need for care in application or the cure may be worse than the disease.

Monday Forum - more on evidence based approaches

In yesterday’s post, Sunday snippets - problems with evidence based public policy, I expressed my reservations about the weight currently placed on evidence based public policy. I thought, therefore, that I might make it the subject of today’s Monday Forum.

Looking back over past posts, It seems that I first wrote on evidence based approaches at the start of 2007. I was especially interested in the application of evidence based approaches in medicine and their spread to other areas. I also found an early comment from 2tanners on the topic, so he has been consistent. It was such a nice story that I am repeating it again:

"Just on evidence based Scottish doctors, perhaps the most famous is Sir Arthur Conan Doyle, who used to emphasise the importance of observation and evidence in medicine as a medical lecturer.

One of his favoured tricks was to take a beaker of foul tasting fluid, dip his finger into it, taste it, pull a face and then ask the class to do the same. Only after they had all tasted the fluid did he take them to task for not noticing that he had dipped one finger, but tasted another. Unlike them. :)" 

I don’t have time to summarise that earlier material, but I pointed at the time to some of the problems that could arise from the blind application of the approach. Two quotes to illustrate:

"In Real world medicine a Scottish doctor expresses his concerns about the application of evidence based medicine in a UK context. In doing so, he makes a distinction between the measurable and immeasurable, suggesting that the focus on the measurable could blind.

He has a point. Part of the reason for the development of evidence based medicine lay in the need to challenge and test previously accepted medical nostrums. However its blind formalised application can distort practice to just the measurable. This holds especially where application is mandated through formalised Government rules".

And again:

"In his interview Professor Sutton defines evidence based management as the simple willingness to find the best evidence you can, and then act on it. But this is not always easy: It's hard to tell what's right and what's wrong, and anybody can be a management expert. It's a signal-to-noise ratio kind of problem: There's just too much stuff out there. And what sells best is by no means the best way to actually practice management. 

He goes on: People are attracted to brand-new ideas and things. The way most knowledge is developed is that people build on one idea, or on nothing at all. A consequence is that the same new things get discovered every six or seven years, and just relabeled. Think of business process reengineering, which is built on a whole lot of earlier ideas."

Taking these quotes as an entry point, I wondered about your own experience with the application of evidence based approaches. As always, feel free to wander!

Saturday morning musings - Heather Ewart and the Country Party, Mr Abbott and problems with what to do and how to do

The first part of Heather Ewart's A Country Road: the Nationals (ABC TV) really took me back as it did for others.. In a Facebook comment Paul Barratt, one of Doug's Deputy Secretaries during the Fraser year's,  recollected: 

Heather's Episode 1 really brought back some memories, memories like ringing the general store at New Brighton via Billinudgel (aka the Summer Capital) and leaving a message for Doug to ring me back. Some time later a call from a public phone in the store, with that familiar voice yelling "Paul? You wanted to talk to me?" etc. Another: Curly Nixon's steadfast refusal to have a phone installed at his beach house. Those were the days!

I had the same reaction to the same incidents. It may seem odd to focus on the refusal of senior ministers to interrupt their holidays, to reject 24/7 unless it was really necessary, but there was a certain balance to it. Doug ran the country from his caravan, and the country was no worse for that. I will write a proper interpretative memoir once the series has finished.

As an aside, I wandered across to Don Aitkin's place to see what his reactions might be to the program. He wrote a number of perceptive books on the Country Party and knew the players. Sadly, or at least I think it so, climate change has come to dominate Don's world. When he first started writing on climate change I was interested and found it a useful corrective to some of the almost theological writing that was and still is around on climate change. Theology goes two ways. I find now that I tune out unless there is something new.

 The negative reporting of the Abbott Government drags on after a somewhat chaotic week. Mark Kenny's  

Tony Abbott steers ship into storm of uncertainty is an example.Apparently "conservative" commentators are upset. I had to put conservative in inverted commas. It is far from clear to me that either Andrew Bolt or Janet Albrechtsen, for example, are conservative in the normal sense that word is used.

I seem to have been dragged into commenting into Australian political events as they proceed. Here I want to come back to the last sentence in my last post, How to make a mess: Mr Abbott’s confusion with objectives, strategy and tactics. There I ended: Meantime, things that are really important get lost in the confusion.

I write about both the what to do and the how to do. On the first, in the lead-up to the last Federal election I suggested that the campaign had become a policy free zone. We almost seem to be in that position now, despite the volume of reporting. And yet, a lot is happening.

Staying with politics, the Victorian elections are this weekend. The latest opinion polls show a small swing back to the Napthine Coalition government, with the most likely result a narrow Labor win. Blowed if I know, however. It all depends on individual seat results.

In Melbourne as in Canberra, we have the prospect of a one term government. In Melbourne as in Canberra, there is no sign that the opposition has regrouped, developed new ideas. The Victorian election has been marked by what one might variously call retail or supermarket politics. There is little new, just long lists of promised activities.

Regardless of whether I agree with Mr Abbott or not, there is a strong case for governments to have at least two terms. I am not talking mandated terms, just that governments need enough time to evolve, oppositions enough time to develop alternative positions in response. Because both sides get locked into stylized positions set within existing frames, we also need circuit breakers,. movements that will capture and force recognition and response to new needs. The Labor and Country Parties played this role,  as indeed (dare I say it?) did One Nation!

One Nation lanced a boil by giving recognition to feelings that existed but had become suppressed within the Australian electorate. By venting those feelings, One Nation allowed the country to move on.

I said that I was concerned with the how as well as the why. Just at the moment, I am probably more concerned with the how.

Interesting interview on ABC Radio National this morning. The link is not yet up. I will add it if I can.

I suppose the fist key point was that engineering studies taught engineering science. The question of engineering practice had been placed aside, something you learned on the job. That was fine when you had Government public works departments that provided a conservative but deep exposure to engineering practice. But now with outsourcing, nobody is providing access to engineering practice, The private sector in Australian and many other countries is effectively free-loading off previous practice knowledge. One outcome is a rise in cost over-runs.

Now there are all sorts of issues here. I am just recording it now for later analysis.  

Monday Forum – the changing face of work

In this forum, I want to pluck your brains.

I write a fair bit on changing management patterns. This includes the use of technology, the organisation of work and changes in perceptions, structures and roles.  One of the interesting questions here is just when particular approaches affected particular workplaces.

Remember the telex? While this dates back to 1926, it really exploded during the 1970s. Like the telex, the fax has a longish history, actually predating the telephone. In 1964, the Xerox Corporation developed the first commercial fax machine. By the late 1970s, a number of models were available. By the early 1980s, fax was fast replacing telex.

These are technology examples. If we look at organisational examples, consider the emergence of the executive, While the idea of boards or executive committees has a very long history indeed, the mass spread of the idea of the executive as a form of governance is quite recent, especially in the public sector. It links to what we now call corporatist models. 

I come now to my questions to you. If you look at your working life, what are the really big changes and when did you first become aware of them? Feel free to go in any direction you like. I’m just interested in what you have to say.

Making social media relevant to project management

A comment exchange with kvd on Prince2, PMBOK and all that project management stuff reminded me of an element from the course that go me musing,

We were talking about virtual teams. The question was which communications tools would you use to support the teams. It won’t come as a surprise that I mentioned social media. The reaction from the group was quite negative.

I need to give you a little background here. The small agency that I am supporting at present is part of a much bigger agency. The bigger agency has suddenly gone into social media, including installing yammer, a Facebook style system designed for enterprise use. It’s very underutilised at our place because people haven’t worked out how to use it to achieve their own ends.

My suggestion was that we might create a closed yammer group where each member of the team could access material as required, check threads as required. We might also post photos and other team material.  People just didn’t get it. They went for emails, a section on the intranet, video-conferencing. Video conferencing has the advantage of interactivity, but most of the other suggestions did not.

Thinking about the discussion. one of the threads coming through was a very negative reaction to Facebook. This actually had nothing to do with my suggestion since this was a controlled use to achieve defined ends, but it did affect responses.

That decided me to try an experiment. From Monday, I am going to use the Departmental yammer system in exactly the same way that I use Facebook and other forms of social media, if with a specific work focus.

This is actually a very rebellious thought. As a contractor, the worst that can happen to me is that they refuse to extend my contact. I never expected that my current contract would roll on to the point that I have become part of the folk memory of the place. Ask Jim, people say!

So, and because this is a structured experiment, I will report on progress. Can I change a mega-agency from the bottom? We will see. At the very least, I will make yammer relevant.  

Prince2, PMBOK and all that project management stuff

I have been on a two day advanced project management training course delivered by Patricia Healy from Marana Consulting. Some of my colleagues were surprised I went. They know that I started studying project management in the 1980s, that I have managed hundreds of projects and indeed delivered project management training. I went because I wanted a refresher. In this regard, I have to say that it was a very good course.

One of the difficulties for someone with my experience is that our knowledge is gained partly by doing. Some of the big things now were simply not invented when I started. The early days of Prince2 date to 1989, although it would be many years before the current version emerged. PMBOK started in 1996. All this creates a number of difficulties.

First, there are subtle shifts in jargon. While the concepts actually remain the same, their naming shifts. Take a simple example. When I started learning project management, we used the term parametric pricing to describe comparisons based on historical data and comparisons with other similar projects. Parametric because these comparisons set estimating parameters. However, the terms top down or analogous estimating appears to be the current use to describe the same process. Now the problem is that if you use the wrong jargon, people don’t know what you mean.

A second difficult is that we now live in a credentialed world. If you don’t have formal qualifications in project management and the capacity to talk learnedly about, say, Prince2, then your experience counts for little. Unless you can drill through and get people talking about specific things such as estimating problems, your knowledge will count for little.

To add to the problems, the actual project management capabilities in many areas of Government are limited. There are very particular reasons for this. A key one is the focus on form rather than function or purpose. You do this and that, create this and that, but it really doesn’t matter. You have an implementation plan, a communications plan, a risk management plan because you must have these things. The plan as an action device is neither here nor there. You just have to have the plan.

This may sound jaundiced and indeed it is. This was the area where Patricia was remarkably good. She showed us, and we were all project managers at different levels, how we might use the project management process to achieve our objectives. It became a tool for our use, not one imposed from on high. None of us were blind to the practical application issues, but we all saw how we might improve performance.

And then from a purely Jim perspective, she gave me some tools and techniques that I actually didn’t know, despite all my experience. One was the way to crash a project, the term used when you must shorten the time frame and provide advice on the costs of so doing. I thought that this technique was bloody brilliant.

Anybody involved with projects will have experienced either a project over-run or a demand to shorten time horizons. We all have techniques for managing in these circumstances. The technique that Patricia demonstrated centred on critical path analysis. It’s advantage from my perspective is ease of application in smaller projects; you don’t need a computer application, but can use a simple working sheet.

We all use project management techniques. It’s called planning. Project planning takes that a step further by creating a degree of rigour. When my daughters were at primary school, the curriculum included project management. In my naive way, I was surprised but thought that was good, that the techniques learned would carry through.

It didn’t quite work like that because the real world isn’t like that. Both girls are good planners when required, but I doubt that they use many of the techniques taught then. De Bono’s hats, a problem solving technique, would be an example. Part of the reason for non-use is that the the techniques were not integrated into the later curriculum. They lacked relevance as a consequence.

A second and more important reason lies in the very nature of project management. Project management takes time. It is a set of processes and tools whose relevance varies depending on what you need to do, If you blindly apply the whole process, use every tool, then planning comes to dominate doing. This is what often happens in government where the tick box approach becomes all. This doesn't work.

If you think of project management as a process and tool kit that you apply to the degree that it’s relevant when it’s relevant, then it becomes very powerful. That is the reason why Patricia’s course was perceived as good; it was personally useful.

Do I feel another Belshaw reform program coming on? Perhaps at a very micro level. 

A number of my colleagues have done or are doing either the certificate IV or diploma courses in project management. They find, will find, that much of what they learn simply sits on the shelf because its just too difficult and time consuming to apply. Maybe, just maybe, it might be possible to create an environment that will encourage selective use.

This environment will not come from on-high. The higher up the food chain people go, the more disconnected they become from working realities. They simply forget or acquire different priorities. Meantime, the worker bees have to get on with their jobs. Change might come if enough of the worker bees adopt the techniques on a selective basis as a way of both managing their own work and managing up. 

Postscript

This post was picked up by Greg Sealby’s Project Management News. Through that, I picked up this post by Ben Horsman, PRINCE2, Waterfall, Agile, and PMBOK. It’s not a bad summary.

Rear Vision - The Market for Higher Education

ABC Radio National’s Rear Vision series has just run a useful documentary on the market for education  especially in the US. You can listen to the program here. I do not know if is available to people outside Australia.

I mention it because of the recent discussion threads here linked to Minister Pyne’s proposals, the economics of education, the structuring of Government funding (if any) that should be provided to support education and especially higher education and, if so, how.

If you can access it, I think that it’s worth a listen. 

Postscript

Another story from this morning that I wanted simply to record at this  time: Working age Australians have become far less reliant on welfare payments, new figures show.

Postscript 2

kvd found this piece by Jill Lepore in the New Yorker: THE DISRUPTION MACHINE. It makes for depressing reading, at least for me. Over thirty years in senior management or as an adviser to management, I have seen so many management fashions. Some I have supported, others made me cautious because they seemed just too simplistic, too partial. I find it ironic now that a fair bit of my professional or semi-professional writing attempts to address the continuing dead hand of things I once supported!

Postscript 3

Evan pointed me to this response by Professor Clayton Christensen to the Lepore piece: Clayton Christensen Responds to New Yorker Takedown of 'Disruptive Innovation'’

:

Saturday Morning Musings - using time recording to help you refine your personal goals

While I am still partially off posting, I thought that I should give you a report on the personal tests that I have been running since it bears upon the discussion on personal goal setting and may actually be helpful to some.

While I write a lot on problems associated with goal setting and the current obsession with measurement, this doesn't mean that I am opposed to either in a general sense. You do need numbers, metrics, to identify issues and analyse performance. My gripe lies in the way in which it is presently done and the adverse results that can follow. You need to be clear on the reasons why and what you hope to gain.

Some years ago I was in a very high pressure Commonwealth job. I became dissatisfied with my apparent failure to deliver, frustrated with the pressure. I decided to keep a detailed time log, just recording the start time, activity, end time. I wasn't sure that this was sensible, it was another thing I had to do, but I persevered for three weeks until I had some decent results.

The results were actually quite reassuring. The results showed clearly both the range of issues that I was dealing with and the high output levels I was achieving. They also showed that the average time between interruptions was just five minutes. Uncomfortably, they also suggested that my pattern of follow up on delegated work was impulsive and hence variable. I actually wasn't putting enough time into managing my people. At the time, I had thirty seven staff, five direct reports plus a very high level of Departmental and Ministerial visibility. I found that that was driving me into things that I didn't need to do, that I was becoming reactive.

With the evidence in front of me, I was able to make a number of simple changes that reduced load and improved performance. While I maintained my open door policy, I also introduced a rule that if my door was shut I could not be interrupted. When work came in, I asked myself should I be doing this, or should I give it to someone else? I started grouping items for later discussion, disciplining myself so that I did not simply rush out to follow up as the impulse hit me. This proved remarkably hard, for I am by nature a reactive and impulsive person.

As an aside before going on, I feel just so sorry for current managers in my previous position. I had a fairly rigid rule that I would stop work at six if not before. In these days of email, blackberries or equivalent and on-line access, the working day has extended and extended. Has real output gone up? I think that the answer has to be a firm no. Still, that's another story.

Recently, I became very dissatisfied once again. This time my focus was on my writing performance and what this was meant to achieve.

Step one was to define a rolling publishing program linked to my personal objectives. I had done this before, but not linked to time recording and results measurement.  The results were not encouraging. I more or less kept to the publishing pattern, but I found that my output was below my expectations, that I only had so much time. I am not saying that my output was bad in objective terms, although it was lower than I had achieved before. Still, I averaged over a thousand words a day, over seven thousand word a week,but I wasn't completing my major writing targets.

The next stage was to withdraw from blogging as much as I could, focusing on my main writing targets. This stage was helped by hard writing deadlines, including a book chapter that I must finish.  Now I found a new issue, I was missing the interactivity that came from my public writing. This proved to be absolutely critical, for it lay at the heart of the reason why I write. That's the greatest fun!!

So in the last stage of my current test. I have set up a spreadsheet to simply record the interactivity. I want to see the pattern in a general sense and in the context of particular media since I write across a number of platforms.

I will report on this at a later point. For the moment, I simply note to my fellow bloggers who wish to write in a professional patten, and indeed all those who are dissatisfied with their personal performance, that measuring your time input is not a bad way to go in at least scooping the problem.       

Monday Management - introducing Common Management Problems

Some years ago I began a series of blog posts on the theme common management problems – and what to do about them. The posts drew from my practical experience as a manager and a consultant. I wrote them because I had found that many people who had acquired some form of management role actually didn’t know what to do in a practical sense.

The problem is especially acute in professional services because so much of the work is individual: individual in the way work is structured, individual in relation to clients, individual in the way performance is measured. There is limited scope to learn how to manage by actually doing. Problems exist even in those areas of professional services where projects and project management are the norm.

Project management is not the same as management, although many project management techniques are also management techniques. A professional discipline in its own right, project management centres on individual projects. These may be simple or complex. They may involve a single person or a much larger team of full and part time people. However, they do not necessarily provide you with the skills required to manage a team with functional responsibilities involving multiple activities with varying time horizons, where the focus is on the achievement of broader results rather than individual project outcomes.

Lack of management skills and experience may be an especial problem in professional services, but it is not limited to that sector. Management is a craft, not a science. It has to be learned through practice. Across sectors, the thinning of management structures has reduced the opportunities for people to grow into management through progressive learning by doing. The problem is compounded by the growing role of specialists in senior management roles and by modern management command and control structures that have reduced the authority and scope of responsibility of individual managers.

Interestingly, the widespread adoption of project based approaches within organisations has further compounded the problem. This may sound counter-intuitive. Surely these approaches play an important role in improving organisational effectiveness? Well yes, but many areas that have moved to project based approaches end by displaying many of the cultural features to be found in professional services. There is the same emphasis on individual matters or assignments; performance assessment becomes narrower; it becomes more difficult to attract attention too, or resources for, issues that fall outside current assignments.

Staff have always complained about managers and management. However, the volume of complaints seems to have risen significantly over the last two decades. Importantly, managers themselves complain about the difficulties they experience in managing properly, in simply coping with the demands placed upon them. A surprising number of those in apparently managerial roles express actual dislike for the management process. Their focus is on their personal performance, on the things that they have to do themselves. The need to deal with people, to be responsible for others with varying needs and personalities, is seen as an impediment to their own performance.

If project management is not the same as management, nor is administration. All managerial roles involve a degree of administration, another thing that many professionals complain about, but that is only part of the manager’s role. Administration focuses on rules and systems, while management focuses on the organisation of resources and especially people to set and achieve objectives and to undertake specific activities.

Finally, management is not the same as leadership, although a measure of leadership is a necessary part of management. A leader may or may not be a manager. Indeed, many of those who see themselves as leaders are in fact extremely bad managers! There is a certain irony in the parallel rise in emphasis on leadership and governance at a time when management as such has been in decline. These trends are connected, for governance is a control on leadership that has become more important as management has declined.

Can management be learned? Yes. Like any other craft, management skills can be learned, although management abilities (the final capacity to do) will vary between people. The keys are knowledge and practice. For that reason, I am drawing together and updating my previous posts to provide practical guidance to all those who wish to improve their management skills.

Postscript

I was asked about the current fad for activity based working and the associated idea of bossless teams. How do these fit with the type of argument I am mounting?

Later I will write something on both. However, for the present, activity based working is popular in certain larger professional services or certain financial firms where work is predominantly individual, or involves small variable teams. The term really relates to office design as much as it does to the organisation of work, although the first affects the second. It can be a productive approach.

Bossless teams rely on particular group dynamics and types of work and on the surrounding culture to be effective. I think that they actually first appeared in certain manufacturing activities, although I stand to be corrected there. Again they can work in particular circumstances, although in the end there is always a boss! 

In terms of the fit with my argument, the best way of organising work will always depend on the type of work. I don't think that either detract from my focus on the importance of management.

I would be interested in reactions from readers in terms of their own experience with these different forms.   

Sunday Essay - Australian politeness & the importance of manners

I was in the supermarket. The operator was obviously new. She was getting flustered, made all the worse by the partially concealed but increasing annoyance of the woman she was serving. My turn came and the girl, she was only a girl, apologised for the delay. "You are knew, aren't you?", I said. "Take your time. I'm not in a hurry." She gave me a beaming smile and shot me through without mistakes and with speed.

It got me thinking at two levels.

The first level is that Australians are a remarkably polite people. If you don't believe me, watch how many people thank the driver when they leave the bus, for example. It's something that overseas students comment on as a distinctively Australian trait.

The second level is the importance of manners. While I have overcome many aspects of the bullying experience at a certain point in my school life. it has left me with residual behaviour patterns that can kick in in sometimes unexpected and unfortunate circumstances. Its a kind of a flight or fight response. If triggered, I can get angry. More often, I find myself unable to handle the situation; at the extreme, I can be reduced to a stuttering wreck. It generally happens where I come under aggressive attack, experience anger or, sometimes, just impatience. It's not a good trait to have in managing personal relationships.

I know that I am not alone in this response. Here one of the things that I have tried to do in my management training is to teach people ways of managing the experience.

For managers, I try to get them to learn to slow down, to listen, to get the other person talking. This can be difficult. Some managers lack the emotional intelligence to measure impact. Others are themselves insecure and compensate through aggression. Sometimes, you just have to drive through regardless of the cost. Yet, as a general rule, a little less haste and impatience generally gives you faster and better results. Less haste, more speed.

For individuals, I try to teach them coping techniques, ways of managing up, as well as ways to minimise the damage that can be inflicted. I'm quite good at it. While I can't always manage my own advice at a personal level, my understanding of my own failures does make me a more effective trainer.

In all this, I come back to a key thing, the importance of politeness and of manners. It really is the grist that holds our society together. Throw in a dash of kindness, and you have the basis for an effectively working society.            

Use and abuse of modelling

In a comment on Navigating the economic forecasting mess, regular commenter kvd pointed me to this piece by Ross Gittins, Damned lies and economic modelling. kvd asked; "Are you moonlighting for Ross Gittens? .. When I read this I could almost swear I've read very similar things right here!"

Over at Winton Bates' place, a strongly favourable comment here on Ross Gittins' writing from another regular commenter, Evan, inspired Where is Ross Gittins coming from? It's kind of a Gittins phase at the moment!

kvd is right, of course, because I have been hammering away at the misuse of modelling as part of my analysis of just what's wrong with current approaches to management and public policy. The Gittins piece is worth reading because it's a very clear exploration of one aspect of the problem. 

We need a management revolution

In my column to be published in today's Armidale Express, I said in part:

One of the difficulties faced by those of us trying to improve management is that it’s almost impossible to break through. The mental and organisational lock-in is just too great.

Imagine, for a moment, just what response you might get if you tell a client or the organisation you work for that it must abolish or drastically restructure its existing performance management and reporting systems; do away with terms like key performance indicators, inputs, outputs or outcomes; and give its people greater real freedom to make decisions, to experiment, to take risks, to make errors.

To argue that we should abolish all the controls, targets and reporting systems so beloved by modern organisations may sound a bit extreme. Sometimes, cleaning the slate and then rebuilding is the only way to go.

In a post, Do family benefits provide a welfare pedestal?, Winton Bates mused about the concept of a welfare pedestal has been popularised by Noel Pearson.

A little while ago I did some work on different aspects of social housing rental policy. As part of this, I spent a lot of time looking at benefits schedules.  Consequently poor Winton attracted a far more detailed response than he might have expected for what was a muse.

Summarising the points I made to Winton:

1. Commonwealth benefits are very complex and also interact with other things such as the provision of social housing and social housing rental policy.
2. Australia is a very diverse country. Consequently, the on-ground impacts of uniform national or state policies are quite differential.
3. The combination means that the likely impact of particular welfare changes can only be understood by combining a detailed analysis of interacting benefits structures on one side, the likely differential on-ground effects on the other. As I so often say, there needs to be a point and counterpoint between the general and the particular if the proposed changes are to work.

As I write, Julia Gillard has just, and I quote from the Australian,  flagged new incentives to encourage two million Australians, including disability pensioners and part-time employees, to work more hours, as part of a dramatic bid to ease workforce shortages and secure the nation's economic future. Here she said in part:

We look with particular care and concern on the large number of working-age Australians, plausibly as many as two million, who stand outside the full-time labour force above and beyond those registered as unemployed

The PM is correct that Australia has a large number of underemployed workers over and beyond the formally unemployed. I think that she is correct in wishing to address this issue. Will it work? I don't think so. She has stitched together a series of different issues (carbon price, mining tax, underemployment, aging population) expressed in generalities, national universals.

It may that her hard working officials can give all this solid substance, but then those same officials are both siloed and  used to dealing in very specific but narrowly defined and universalised performance measures. Further, there are no quick fixes. 

In what will seem an apparently disconnected segue, I am presently working with my old friend and colleague Noric Dilanchian defining a new on-line services business. My assignment is to give the concept sufficiently defined form so that its viability can be tested and then, if it passes that test, carried through to a business entity.

The concept itself rests on two assumptions (conclusions?) directly relevant to our discussion.

The first is that the thinning out of management, the introduction of new forms of working arrangements, has now destroyed previous in-house knowledge to the point that managers can no longer do or advise on things that they once could. Further, organisations no longer have the discretionary resources they once had to do real development work, to address new or even recurring issues.

There is an important sub-text here in the way that modern information systems operate. In some ways, they have actually reduced organisations' capacity to access lessons or at least experience from the past. I just mention this one now.

The second assumption and/or conclusion lies in the impact of computing and communications technology on the pattern of work and on work flows. I suspect that most professional services advisers will smile in sympathy on this one.

We live in a just in time, 24/7 world. Time to think is limited. Issues that don't have to be dealt with now are pushed aside. However, when they are dealt with, they have to be dealt with quickly. The client sends a brief, partially thought out email, do this now.

The professional follows up, seeking the information required to provide proper advice. Clients quickly become impatient, especially when (as is often the case) they haven't actually done the preparation required. They just see a problem and want it fixed - now and at a low price.

A lot of professional services firms simply roll with this. They do just what, and only what, the client wants even though the professionals involved know that it's not going to work or could work better. Take the fees and run. This maximises fees.

The issue in our mind lies in the possibility of doing new things, of combining changes to organisational structure with changed working styles. Have we now reached the point where clients will actually pay for certain higher value added services because they really have no choice?

We are not talking here of things like document automation, things that automate and industrialise certain processes. Rather, the management of certain more complex matters and processes that are not, on the surface, easily amenable to on-line delivery because they still require customisation and individual attention.

Will this work? The short answer is that I don't know because it still requires people to commit a degree of time. Nevertheless, the issues involved are quite fascinating.

The concerns that I have about management, about public policy, about the conflict between the short and long term, are not academic nor are they based on economic generalities. They come from the fact that I have worked at high policy level, at high management level, but also at the coal face. then and now. They also come from my experience in working across sectors and functions.

As a framework person, I cannot help generalising and testing. I then really get very frustrated when I cannot get my conclusions across. There is little point and no pleasure in saying I told you so when some pretty obvious failures emerge.

We need a management revolution, something that will clean out the barnacles of the past without destroying the ship itself.

Postscript:

Lynne came up with a very quick comment that exactly captured the problems associated with universal rules. I quote:

Jim, I have been looking for some studies on the impact of the compulsory " volunteering " for over 55 year olds who are in receipt of Centrelink benefits. The client is required to 'volunteer' for 15 hours unpaid work a week. The Voluntary Organisations are restricted to Centrelink approved and do not include family activities such as care of grandchildren. Health doesn't seem to be taken into much consideration and nor does the expense of participating. The previous qualifications and achievements of the Client also seem to count for very little.

I am also interested in the flow on effects to the organisations which are often flooded with unwilling or untrained " volunteers' and on positions which would once have provided paid employment.

I have seen the polished versions of why this is a good scheme and of how well it is working but have not really seen that in the reality of the unemployed over 55 year old. Yrs Lynne.

Once again , just another musing.

Do you think that the things that Lynne talks about are a sensible result?

Saturday Morning Musings - problems with text tables

From time to time something happens that make me realise (again) that I do not fit in with some features of the current world. The latest example is the constant use of text tables in official documents as a way of summarising and presenting information and desired outputs or results.

I, too, use text tables and they have their place. However, their use is now so wide-spread that it has become quite pernicious.

I have the good fortune to be a very fast reader. I also have a reasonably good memory for the things I read. Often in my work I have had to provide briefing or advice on documents - reports, statistical releases, plans, articles, books -  in very short time. My reading speed allows me to skim a document to get the guts very quickly, thus allowing me to move quickly to writing at least an initial brief.

You can actually see this in some of my blogging. I think of some pieces as briefing notes to my readers who (in this context) have taken the place of ministers or senior management.

The approach that I follow depends a little on my purpose and the nature of the document. However, in broad terms I start by skimming the table of contents. This tells me what's in the book or report.

I then look at the introduction or executive summary. This tells me what the writer thinks is important. Depending on purpose and the nature of the document, I may look at the index, using this to focus on specific areas.

With history books in my areas of interest, I actually use the index to determine the value of the book to me. If the index does not contain items that I consider to be important, or if on checking back to the relevant page I find the content limited or bad, I don't bother any more.

With official reports, I browse the report after looking at the introduction or executive summary. My purpose here is to flesh out but also check the material in the introduction or executive summary. I also look for particular points of interest, noting the page number in case I want to go back.

In most cases, I can now write my brief. In some cases, I may want to read the whole document in more detail.

I cannot follow this approach with text tables. Where the text table is important, I have to stop and work through it column by column, row by row. This is actually quite a slow process. However, problems do not end here.

By its nature, the English in the text table is shortened form, sometimes cryptic. This comes about because of the mechanical limitations associated with the use of tables. Only so many columns will fit on a page, while there are also practical limitations on the amount of text that can be included in a cell. All this makes comprehension harder.

Quite often, there is at least some disconnect between the text table and the document. This comes about because of the importance of text tables in setting out key recommendations and measurable outputs or outcomes.

Towards the end of the writing process, many official documents go through a committee process in which people focus on the table itself. Changes are made, but these are not always reflected in changes in text in the main document. Quite subtle shifts in wording mean that the main document and text table diverge.

Problems continue.

We live in a measurement age. When looking at the text table, people want to add measures, aka outcomes. The very structure of the table means that individual items are considered in isolation from the broader picture.

One of my colleagues described me as a framework person. I like to work out a general structure and then dig down.

From a policy perspective, the more policy objectives that you have, the greater the likelihood of conflict between objectives. Further, you are always dealing with a hierarchy of objectives, a cascade down. If you want to achieve a, then there are a whole lot of things that you need to do to achieve a. But these are all secondary.

In a measurement world, the manifold outcomes set out in text tables acquire a life of their own. They become the things that measure success independent of relevance.

My experience with successful policy development - and this applies in a business context as well - is that implementation is messy, requiring constant changes. Things are never what you expected.

The problem with the combination of measurement and text tables is that they lock policy into a rigid structure determined in advance of implementation. Once set, they continue independent of diverging reality.

Text tables are a useful tool - the matrix approach that Bob Quiggin and I talk about is an example. However, once they become the key way of presenting things, the end result is nearly always a mess.       

Fashion and the first five fallacies in modern management - a note

My post Saturday Morning Musings - the first five fallacies in modern management drew a comment from Bob Embry who said in part:

Almost no institutions on the planet have practiced or are practicing Drucker's view of management. If they were it would be "positively" obvious in our day to day lives—as opposed to "negatively obvious."

In a broad sense, I think that Bob is right. I found Peter Drucker's writing enormously influential, still relevant today. The core problem I have is the way in which certain approaches have come to be applied in combination almost as holy writ.

Take planning as an example. At one level, the value of planning is self-evident - it sets out the how to do. However, the problem we now have lies in the blind application of planning techniques without proper understanding of either their role or limitations. I have discussed this at various times, most recently in Mechanistic Management - the sometimes fallacy of planning.

A particular difficulty lies in the role of fad and fashion in management thinking. Management is very prone to fashion. New ideas rise, become popular, then recede. Yet even in recession, they may still remain embedded in the organisational landscape like an ancient artifact
.
To illustrate what I mean, take the concept of the business plan. Originally conceived in a start-up context, this had morphed by the mid-eighties into something that all businesses should have. The idea became specially popular with Australian governments - state and Federal Governments provided funds to help smaller businesses develop their own business plans.

Most if not all of this money was wasted because thinking was actually mis-directed. Good businesses already knew what they were doing - they had techniques in place. Other businesses lacked the capacity to really benefit.

In all this, there was a mismatch between the assumptions held by those espousing business planning approaches almost as holey writ and the actual mechanics involved in small to medium size business. The most successful small businesses were too busy responding to changing market need, too busy trying new things, too busy improving what they did, to set up plans that they knew would be quickly invalidated by changing business conditions.

It is now more than fifteen years since the business planning ideal peaked. However, the concept remains embedded in the Australian institutional landscape.

Government is a particular problem here in that Government agencies have, to my mind inappropriately, institutionalised internal business planning techniques without proper reference to their original purpose. They have become, to use modern jargon, commonly accepted "best practice". Worse, they sometimes attempt to impose the approach on others in the name of good public policy.

Don't get me wrong. I do support business planning approaches, but only in the right place.

In all this, as a reformer and business improver I am as prone to fashion as anybody else. I have been involved in one way or another with most of the popular management movements over recent decades. So I will and have pleaded mea culpa at times (here, for example) for my own enthusiastic errors.

Now as I argue the need for new approaches I am conscious at times of the danger that I may simply be participating in the latest fashion wave.

We live in age of shortening organisational life cycles, of business destruction. Here I am not talking about the latest financial crisis, but of a longer term trend. I attribute this in part to mechanistic management, arguing that we need improve organisational resilience. I thought that this was a new idea, but it is in fact very popular just at present. So it too is a fashion

I have also argued for evidence based approaches, centered especially on the concept of a discipline of practice. Now here I was seriously concerned to learn that Australian Prime Minister Rudd is an exponent of evidence based approaches.

Why concerned? If we formalise evidence based approaches within current policy and institutional structures, then it risks becomes just another device within what I have called the current fallacy of measurement for enforcing central control.

Project Management, Creation and Use of Case Studies

Putting aside my personal hat for my professional one just for a moment, I wanted to mention two interesting series of posts on other blogs that I am involved with.

Many of us use case studies in our work. They are also used increasingly in a range of areas like regional and community development to explain why development does, does not happen. In this context, I have just put up three posts on the Managing the Professional Services Firm blog on the creation and use of case studies. The posts are:

• Creation & Use of Case Studies - Introduction
• Creation & Use of Case Studies - Training 1
• Creation & Use of Case Studies - Training 2

Project management is another area of common interest across management and the professions. Here we are putting up a series of posts on project management techniques on the Ndarala Group blog. So far we have put up two posts:

• Project Management for Professionals - Introduction
• Project Management for Professionals - What is a Project?

In both cases we are drawing from internal Ndarala Guides prepared for the use of members and clients. We have decided that we should make this material more broadly available.