History & personal papers - a story of personal stupidity

I am still tidying up and disposing of stuff. I doing so, I got very angry with myself. Let me explain why.

In 1980 I was Assistant Secretary Economic Analysis Branch in the Department of Industry and Commerce.  From 1983 to mid 1987, I was the key Commonwealth policy adviser providing advice on the development of Australia's electronics, aerospace and information industries. Our particular approach involved us in every aspect of Government policy that affected those industries.

In mid 1987 I put my personal money where my policy mouth had been and set up  a consulting, training and information services consultancy specialising in the high technology sector. The business grew very rapidly only to crash in the slump of the early 1990s. To this day, I regret that we did not have access to the professional advice that might have saved it. Nobody then knew - the internet was still some years ahead - that the things that we were doing would form the core of global information services businesses. I knew what we had, but simply couldn't get the message across.

As part of our work, we had a very large Government relations practice. Our clients included Telecom, Westpac, AWA, Rolls Royce, United Technologies, AWA, IBM and Toshiba as well as industry associations and  overseas Governments. We provide policy advice to more than a dozen Government departments or agencies in Australia.

When the business went down, Ferrier Hodgson as first administrators and then liquidators proposed to send all the company records to the tip. I hardly thought that our clients would like that, and personally moved thousands of files to our garage, Then I had to make decisions about what to do with those files and my own papers since we were leaving Armidale for Sydney.

I am a meticulous record keeper. I had to decide what to do. I looked at putting the collections into an archive protected by an embargo period, but with cut backs everywhere no one was interested. This is the record series and what happened to it:

• I had drop copies of every piece of paper that I had written as an official. Minutes to the minister, letters, memos etc. Shredded.
• There were copies of every policy release or official report relevant to my interests from the 1970s to the early 1990s. Recycled.
• Every contact my company had had (we had seventeen staff) with clients, officials, researchers, whatever, was meticulously recorded, along with our internal analysis. We practiced what we called triangulation, checking facts by asking contacts not about their what they thought, but about what others' thought. We also had a practice that we called penetration in depth, With one agency, I would deal at the higher levels, but my research officers and industry analysts would deal with people at their level. In some agencies we knew the views of people from secretaries down to class sevens or even lower. This material was shredded.
• We had thousands of our reports looking at every aspect of official policy, industry performance, firm performance. To give an indication of scale here, we had the annual reports, the official filings, of more than two hundred global telcos over multiple years, along with consolidated analysis on various performance aspects. All this was shredded or recycled.

By the time the company went down we had what was, in retrospect, a remarkable resource. I should add that we did not record personal details. Our focus was on policy issues and debate. We wanted to know what was likely to happen and why.

We were helped here by our detailed understanding of official process. We knew every stage in the policy process, allowing us to forecast just what might happen and in what time table.

Even today working alone in a home office with so many things changed, I can make pretty accurate judgements about likely outcomes.

I don't care how good the analyst or historian is, they will never be able to replicate the material or knowledge we once had.

Today when I want to write about some of these matters, I really miss my access to past material. Why didn't I save this stuff? That is why I say it is a story of personal stupidity. I really should have found a way to save the material.  

Postscript

I responded in comments to a comment from regular commenter KVD. He responded: "I think you should postscript your comment into your main post - because it provides an eloquent reasoning as to why your loss is important, significant." I said:

David, we need more books and analysis showing how decisions are made. Some of my commentors wonder about the process.

You see, we have lots of people doing their job. In all the material we collected, there was no evidence of improper conduct. There was just the way things worked.

There were heros and villains, but the villains were those who allowed immediate objectives, to override other things. For their part, many of the heroes are unknown, many in industry and the unions.

I saw union leaders who knew that change was inevitable prepared to take positions against the short term interests of their members in the hope of longer term gains. I saw industry leaders adopt similar positions. And I saw both destroyed by positions taken by others on political and ideological grounds.

There have been some very good US books on decision processes. I would argue that we need more such in Australia to overcome the superficiality of current analysis. It's actually exciting stuff if well written.

The problem I have with the material I destroyed is that it would have allowed a detailed analysis to be written of a slice of decision making. This includes the aspirations and dreams involved.

I thank KVD for the suggestion for my comment does amplify the reasons why I consider the example to be important. It's not just self-indulgence on my part.

Policy development is an interaction over time among many players with very varied interests. Much of the material that has been written, and some very good, has a political focus. It starts and ends there, when the reality is far more complex.