Wrestling with confirmation bias

In a comment on Somewhat depressing threads, Winton Bates wrote in part:

I suspect confirmation bias in the pattern you observe, but that doesn't mean you are wrong.

The pattern I see is about the consequences of growth of government. It displaces civil society, it makes people dependent, it responds to every problem with more regulation. It invites disrespect.

But that is a pattern I see everywhere!

I had to laugh. As Winton notes, I may be guilty of it, but in his dry way he is also accusing himself!

Confirmation bias, the tendency to selectively interpret material so that it reinforces our own views, is pretty common. We saw it in some of the reactions to Julia Gillard's now famous speech. We saw it, too, in the US Republican reactions to the polls and polling results casting doubt on their candidates chances. It actually became a circular dance as one news channel or commenter quoted another in support of their views.

We see this all the time on the modern internet as people drift towards sites that support and reinforce their own views. Arguably, this has become one source of deepening polarisation.

I claim to be a social analyst. How, then, do I avoid confirmation bias, something that I get accused of directly or indirectly from time to time?

At one level, life is just too short to bother. I have a bad habit at dinner parties or social functions of trying to explain, of being too objective. Bad Jim! Wrong place, wrong time. Just listen. Your social analyst hat is just inappropriate If people are having a lot of fun piling opinion on opinion.

At other times in similar circumstances if the mood takes me, I will deliberately throw in an alternative view, something that I know will be a red flag item. Done with discretion, this can be fun, but discretion is advisable. You must be able to stand back and not actually become personally involved in the issues.

Then, at other times, I may just want a chance to talk and express my opinions without having to worry about just what I say. Here confirmation bias can be a fine thing!

Yet when trying to write as a social analyst, I am concerned with what is, what it might mean. Sometimes I go beyond this, for I try to provide solutions, responses, to the problems as I have defined them. They are my solutions, I own them, I want to argue for them.

But when writing in a professional way, I use three different techniques to try to control my natural tendency to confirmation bias.

The first is by consciously reading, and its a real effort some time, opposing views. In political terms, this takes me to the blogs and commentariat of left and right.  I tune out the things that are standard opinions beyond noting patterns. Instead, I look for the kernels that might challenge or enhance my own views.

The second thing I do is what I call point and counterpoint. I start with the particular then test with the general and then back to the particular.

The thing that  I do is to try to test by evidence. In the US election this time, everybody started talking about the impact of demographic change. This one has been obvious to Blind Freddy for some time. If you look at my own writing on change in Australia, you will see that I have often quoted demographic or other forms of statistics.

Some of my writing is not easily amenable to statistical test without special surveys. I argue, for example, that changes in management styles in public and private sectors have increased overhead and slowed decision making.

I can show through simple arithmetic that decision times slow and decision costs increase as the number of decision points rises. I can show by logical analysis the way in which the obsession with KPIs and with the measurable can twist decision processes. I can use simple mud map analysis to show how decisions such as those made by the NSW Government on the length of time required to get a driver's license have imposed costs and had a range of adverse outcomes.

What I cannot do is provide an analysis that measures the overall costs of all these things in a way that would grab attention and test my case. The data presently isn't there. I have to rely on point and counterpoint, generalising and then testing with specific examples. But it remains qualitative analysis.

Having said this, I wonder if there is more that I could do to test. But that is a matter for a later discussion. I really need to cook tea, not chat to you!

Tools for the home researcher 2 - Using your scanner

This post continues my new irregular series on the things that I have learned that may help the home researcher.

This post is going to sound a bit like the current habit of plagiarism that universities are trying to stamp out and indeed it does start with something of the same technique. However, the results are different.

Just at the moment my scanner is broken. I am actually having to key things in. This approach will work much better if you can scan into Word.

I am presently writing some stuff on Aboriginal social structures. I have decide to use John Mulvaney & Johan Kamminga, Prehistory of Australia, Allen & Unwin, Crows Nest, 1999 as a base. I am not trying to steal their ideas. I want to make sure that I acknowledge their contribution.

I have copied the relevant section almost verbatim. Then at the end of each paragraph I have added my own comments as footnotes pointing to areas where I agree, disagree or which require further work. In doing so, I have been careful to record the original page number for each paragraph.

I now have a document that marks the first point in a dialogue with the thoughts and words of the authors, one that I can print, read and think about.

As I do further reading I will amend the text. I have written on some of the ideas already. Here I will compare the ideas and amend. In other cases, I know of other sources. New material can be added as either footnotes or in the main body, again recording source.

In writing, one of the key things that I want to do is to compare the general Australia wide statements of the authors with the on-ground position in New England. Is there a variation between the two? If so, what does it mean? Here I am particularly concerned at the way central and north Australian examples are generalised, given variation in Aboriginal cultures across the country.

The document evolves as a working document, a continuing dialogue.

I do not want to steal the original ideas. So, depending on the nature of the changes, I will acknowledge the original contribution in either footnotes or the text itself. I also save the first copy as V1 and then save major changes later as V2, V3 etc. That way I can check wording and ideas as to sources.

Simple isn't it, and I think quite fair. But does it breach copyright?

I don't think so. Copying a very small section of a work for private review would seem to me to be fair dealing.

The approach actually works best with small sections. The problem with large scale scissors and paste, say bringing together a number of documents from the web and then trying to meld them in some way, is not just the almost certain risk of plagiarism in the end result, but the time and difficulty involved in the process.

Return to first post in this series.