This paper slightly amended first appeared in the Autumn 2008 issue of the 'Skeptical Adversaria' (the Newsletter of ASKE, the Association for Skeptical Enquiry), pp. 1-3.
Much of this issue is about risk and requires a reasonable working knowledge of the concept of probability. This is not difficult to acquire in my opinion (otherwise I wouldn't have managed it myself), and is a useful resource for anyone who surveys the world of information through a sceptical lens.
So, let's think a bit about probability and what it means when we say 'X has a such-and-such probability (or chance or likelihood) of happening'.
I think it was last year that the public were shocked and dismayed by a particularly horrific murder that occurred on the victim's own doorstep. Several men were convicted and it was announced that one of them, Mr X, had only just been released from prison (I believe on parole), despite, according to the press, being assessed as '80% likely to re-offend violently'.
What does it mean to say that 'Mr X is 80% likely to re-offend violently'? Is this some property that the person possesses that you can detect on examination, like observing, for example, that he has 20% loss of power in his right arm or that he has first-degree burns to 30% of his skin? Hardly.
In his book Taking Chances: Winning with Probability (Oxford University Press, 2003) John Haigh considers various approaches to the question 'What is probability?' and then has this to say:
'To think of probability as describing a degree of belief reconciles all these approaches. The stronger the degree of belief in the event, the larger the probability associated with it' (page 2). One of the examples he uses to illustrate this is the statement 'The probability that Shakespeare wrote Macbeth is 80%'.
Mr Haigh is a Reader in Mathematics and Statistics at one of our prestigious universities and I hesitate to make even a token gesture of disagreement. However, I do not find his explanation helpful in some of the tasks I undertake in my professional work.
I find it more useful to treat a probability, such as a percentage, for what it is - a proportion of something. So in response to the statement 'Mr X has an 80% chance of re-offending' I would retort '80% of what?' Viewed in this way, clearly the statement doesn't make any sense.
In fact the original statement will not have simply reflected someone's subjective belief based on years of experience in these matters. It is highly likely that somebody, probably a probation officer, undertook a risk assessment on Mr X for the Parole Board that came up with the figure 80%.
But what does this 80% mean; what is it 80% of?
The answer is that that it is the proportion of offenders scoring the same or higher than Mr X on the risk assessment protocol who later commit a violent crime (sometimes a time period is specified).
The most common forensic risk assessment procedures consist of lists of factors that have been demonstrated to correlate with re-offending. In brief, the assessor scores the person on each of these factors (e.g. on a scale from 0 to 2) and adds up the scores. The authors of the instrument provide tables giving the re-offending rates of people whose total score is equal to that of the offender being assessed.
Many of the known risk factors are 'static' or 'historical'; that is, they cannot change or are unlikely to (and if they do it may be for the worse). Examples are age of first offence, number of violent offences, number of failed relationships and a history of drug or alcohol misuse. There are also 'dynamic' factors (which may change) such as evidence of genuine remorse, responsiveness to treatment for any mental health problems, and realistic plans for future management.
Thus, there is an evidence base for forensic risk assessment but the correct way of expressing the outcome is not to say, for example, 'Mr X is 80% likely to re-offend violently'. Strictly one should say '80% of people with Mr X's risk profile go on to re-offend'.
It is difficult to know whether Mr X will be amongst the 80% who do re-offend or amongst the 20% who don't. There is a myriad of 'idiosyncratic' factors, many difficult to quantify, that may push him into the 20% rather than the 80% group.
So, to take an example from another sphere of life, namely the weather, what does it mean when the weather forecaster says, 'There is an 80% chance of rain'? She might be attempting to say something about her confidence in her own belief about whether it will rain or not. However, if she is being more objective, her real meaning will be something like 'Whenever these particular meteorological conditions are in place, rain occurs on 80% of occasions'.
Active symptoms of a major mental illness are a known risk factor for violent re-offending. Whether directly or indirectly, the media have a habit of regularly conveying the impression that increasing numbers of mentally ill people are wandering the streets attacking and murdering innocent people (see note 1). Whenever such a murder happens there is an official enquiry, shortcomings in the relevant services and personnel are noted, and recommendations for improvements made. Thus media attention to individual cases of homicide by mentally ill people are likely to be given much more prominence than other killings. Invariably the commentators resort to the usual cut-and-paste clichés - 'This is yet another case where care in the community has gone wrong', etc.
'Care in the community' is a concept of which few journalists manifest any real understanding. But more importantly, far from there being an increase in homicides by mentally ill people, in England and Wales there has been a steady fall since the 1970s. This is set against a steady rise in murders by people who are not mentally ill.
These findings are published in the British Journal of Psychiatry, 2008, 193, 130-133 (see tinyurl.com/5tphtt). According to the authors, 'The introduction and increasing use of antipsychotic medication, the greater awareness of the treatment of psychosis by primary care providers after deinstitutionalisation, and the creation of regional health authorities with responsibility for defined populations, may all have contributed to the observed decline in abnormal homicide since the 1970s'.
One commonly used risk assessment procedure for violent offending is the Revised Psychopathy Check-List (the PCL-R) due to Robert Hare, Professor of Psychology at the University of British Columbia. This is a 20-item instrument that in forensic psychology and psychiatry is recognised in North America and the UK as the gold standard for determining whether someone is a psychopath. This decision is based on a single score (although there are at least two main composite factors that can be rated separately). Particularly in North America, if you are a criminal then your score on this instrument can significantly influence decisions affecting your freedom or incarceration. Serving convicts have taken psychologists to court to contest, item-by-item, their PCL-R assessments - a sure sign that something is wrong with the system.
The PCL-R also measures risk of violent re-offending. However, all the items are static or tend to be so: they do not change or change little over the person's lifetime. So a convict's score, say, at the start of his sentence is likely to be the same when he is due for parole or release. More popular risk assessments incorporate dynamic factors but their total scores still tend to be heavily weighted by static items.
In short, these instruments are elaborate methods for quantifying one of the fundamental laws of psychology, namely that past behaviour predicts future behaviour. This is not a bad way of summing up the essence of 'psychopathy': metaphorically speaking, a leopard can't change its spots.
Notwithstanding all of the foregoing, formal, custom-made risk assessment scales are the best methods we have of quantifying risk of violence and are superior to personality tests and 'clinical judgement', whether by an experienced forensic psychiatrist or the lady who cleans his office - it may not make much difference.
I was not therefore too impressed by the following announcement which ASKE member John Birchall drew to my attention recently.
It seems that asking a person a single question can help determine whether he is a psychopath. The test has been used by 'a famous American psychologist' who has found that serial killers often gave a certain answer. The question is as follows:
'A woman, while at the funeral of her own mother, met a guy whom she did not know. She thought this guy was amazing. She believed him to be her dream guy so much, that she fell in love with him right there, but never asked for his number and could not find him. A few days later she killed her sister. What was her motive for killing her sister?'
See note 2 for the answer.
1. Recently the media, including BBC's Radio 4 morning news, reported a 'shock horror' story about the 'horrific' number of dangerous mentally ill patients who escape from secure hospitals. According to Radio 4 there are more such escapes than from prison. In fact there are no escapes from high secure hospitals at all, only from medium and low secure unite. Moreover, most of the 'escapes' recorded are in fact 'absconsions' by patients who are on leave (escorted or unescorted). Leave (for example a few hours spent shopping) is an essential part of the rehabilitation of these patients and is only granted to those who have attained mental stability and have been carefully risk assessed. Very occasionally a patient will overextend his or her leave period, visit relatives, go to the pub, etc. but they are nearly always returned to the unit without much delay or incident. Their leave entitlement is then withdrawn until they are again deemed ready to go out. The risk to any particular member of the public of harm from a sectioned patient who is out on leave, legitimately or absconding, is finite but miniscule - I imagine less than being struck by lightning.
Incidentally, a few years ago the secure unit in which I work was enclosed within a metal fence over 5 metres high, despite the fact that nobody had ever escaped or made any serious attempt to escape from the building. This was a requirement of hastily convened government requirements. This fence cost the National Health Service one million pounds while hundreds of people were dying because of infections picked up in general hospitals because of unhygienic conditions there.
2. According to the author of this puzzle, you are a psychopath if you say that the woman killed her sister so that there would be another family funeral at which the mysterious man might show up again! If you thought of this answer please don't take any of this seriously.
Skellett, A.-M., Millington, G.W.M. & Levell, N.J. (2008) Sudden whitening of the hair: An historical fiction? Journal of the Royal Society of Medicine, 101, 574-576.