Oxford University Press's
Academic Insights for the Thinking World

When probability is not enough

While out driving one afternoon, I notice a bus speeding down the road towards me. As it approaches, the bus drifts into my lane, forcing me to swerve and strike a parked car. The bus doesn’t stop and, while I glimpse some corporate logo on the side, I’m shaken and I don’t manage to make it out. Later, when I’ve recovered myself, I do some investigating and find out that of all the buses operating in the area on the day in question, 90% were owned by one company – the ‘Blue-Bus’ company – with the other 10% owned by rival companies. Should I take the Blue-Bus company to court and demand compensation for the damage caused? If you were presiding over the case, would you make them pay? When people were presented with cases like this in a series of psychological studies, the majority thought that the company should not be held liable just because they happened to own the majority of buses in the area. And judges and lawyers seem to agree – this example is based upon a real, unsuccessful, lawsuit.

But there is something curious about this. After all, my evidence against the Blue-Bus company seems very strong – it makes it 90% probable that the bus involved was a Blue-Bus bus. Why isn’t that enough? Surely we don’t have to be 100% certain before we make the Blue-Bus company pay up – we’re rarely 100% certain of anything. Also, it seems that we would be willing to hold the company liable on the strength of other kinds of uncertain evidence.

Suppose I don’t find any general information about the buses in the area, but I do find another witness to the incident who tells me that the bus was a Blue-Bus bus. Under current legal practice, the Blue-Bus company could well be found liable on the basis of testimony like this. But witnesses don’t always speak the truth – it’s still not certain it was a Blue-Bus bus. The witness might have been hallucinating, or might have a false memory, or could just be making up a story to smear the company. The testimony undoubtedly makes it probable that it was a Blue-Bus bus that forced me to swerve but, plausibly, it doesn’t make quite as probable as 90%. Eyewitnesses are not that reliable. Viewed in one way, the situation seems very puzzling. We’re happy for the Blue-Bus company to be found liable on the basis of the testimonial evidence, but not on the basis of the statistical evidence, even though the statistical evidence makes it more likely that the bus involved was a Blue-Bus bus.

Though it has been widely discussed in legal theory, this puzzle isn’t just a legal one – it goes deeper. Suppose you’re not involved in any legal proceedings against the Blue-Bus company, but you’re just trying to figure out for yourself what happened. If your only evidence is that 90% of the buses operating in the area were Blue-Bus buses then you shouldn’t believe that the bus involved was a Blue-Bus bus – that would be jumping to conclusions. And yet, if a witness says that the bus involved was a Blue-Bus bus, then you would be justified in believing this. So much of what we believe is based on taking others at their word. But how could I be more justified in believing that the bus was a Blue-Bus bus on the basis of testimony, when the statistical evidence makes this more likely to be true?

But how could I be more justified in believing that the bus was a Blue-Bus bus on the basis of testimony, when the statistical evidence makes this more likely to be true?

One possibility is that our instincts about this example, and others like it, are just wrong. Maybe we should believe that the bus involved was a Blue-Bus bus on the basis of the statistical evidence, and we’re being irrational if we refuse to believe this. And maybe the Blue-Bus company should be found liable on the strength of this evidence too, and our legal practices need to be reformed.

Then again, maybe our instincts about the example are absolutely right – and they only seem puzzling because we’re trying to understand them within a probabilistic framework. Thinking in terms of probabilities is very natural and useful, but is it the only way to look at a piece of evidence? Here is something different to consider: If a witness tells me that the bus involved was a Blue-Bus bus, and it wasn’t, then there has to be some explanation as to how this came about. Some possible explanations have been mentioned above – maybe the witness suffered a hallucination or a false memory or was lying etc. Whatever the case, there has to be more to the story – it can’t ‘just so happen’ that the bus wasn’t a Blue-Bus bus, even though the witness said that it was. But it could ‘just so happen’ that the bus wasn’t a Blue-Bus bus even though 90% of the buses operating in the area were Blue-Bus buses. While this would in a way be surprising, given the proportions involved, no special explanation would be needed.

If you believe that the bus was a Blue-Bus bus, based on testimony then, while your belief is not guaranteed to be true, something is guaranteed: if your belief is false, then this must be due to some interfering factor. This is a feature of the testimonial evidence that has nothing to do with probabilities – and it’s not shared by the statistical evidence. If you believe that the bus was a Blue-Bus bus based on the fact that 90% of the buses in the area were Blue-Bus buses then this could turn out to be simply false, and there’s nothing more to it – you took a gamble and you lost.

Is this what makes it OK to believe things on the basis of testimonial, but not statistical, evidence? Is this what makes it reasonable to find companies liable on the basis of testimonial but not statistical evidence? Maybe, maybe not. But, at the very least, it shows us that probabilities are not the whole story when it comes to understanding our evidence.

Featured image credit: Photo by Dave Gough. CC-BY-2.0 via Flickr.

Recent Comments

  1. Garrett

    Usually testimony alone isn’t enough to convict, there needs to be corroborating evidence along with it (the capabilities of prosecution and the defense along with the temperament of the judge and jury notwithstanding). Though I get that this is a civil and not a criminal case so I guess testimony alone may be enough for the jury. The reason why that particular probability alone wouldn’t be something to present in this case is because it doesn’t make definite assertions. Lie detectors and DNA tests make specific assertions about specific instances that can be wrong, and in the US we exclude the former, whereas in this case the probability only makes an assertion about the makeup of buses in the area but not about the probability of that bus in particular. It is a technicality but in general we err on the side of caution in western law and, for the most part, follow the dictum: better a thousand guilty men go free than a single innocent be punished. If this were reversed I presume a different standard would be in place. I guess that would be one reason as to why we wouldn’t go with that alone. I may be wrong as I haven’t read the case files and this was just a guess.

    Though I don’t know why you didn’t get the bus routes and cross reference them with the time the incident occurred, or at least supeanad the bus companies to provide the trackers that practically all public transit keep on their buses for a variety of efficiency driven purposes.

  2. Max Sinclair

    Don’t forget the motivation to prove or avoid liability, and the resulting investigations of fact. Don’t forget the law’s evolution of practices to test the reliability of evidence. Was the witness recently fired by Blue Bus? Cross-examination finds out.

Leave a Comment

Your email address will not be published. Required fields are marked *