When probability is not enough

While out driving one afternoon, I notice a bus speeding down the road towards me. As it approaches, the bus drifts into my lane, forcing me to swerve and strike a parked car. The bus doesn’t stop and, while I glimpse some corporate logo on the side, I’m shaken and I don’t manage to make it out. Later, when I’ve recovered myself, I do some investigating and find out that of all the buses operating in the area on the day in question, 90% were owned by one company – the ‘Blue-Bus’ company – with the other 10% owned by rival companies. Should I take the Blue-Bus company to court and demand compensation for the damage caused? If you were presiding over the case, would you make them pay? When people were presented with cases like this in a series of psychological studies, the majority thought that the company should not be held liable just because they happened to own the majority of buses in the area. And judges and lawyers seem to agree – this example is based upon a real, unsuccessful, lawsuit.

But there is something curious about this. After all, my evidence against the Blue-Bus company seems very strong – it makes it 90% probable that the bus involved was a Blue-Bus bus. Why isn’t that enough? Surely we don’t have to be 100% certain before we make the Blue-Bus company pay up – we’re rarely 100% certain of anything. Also, it seems that we would be willing to hold the company liable on the strength of other kinds of uncertain evidence.

Suppose I don’t find any general information about the buses in the area, but I do find another witness to the incident who tells me that the bus was a Blue-Bus bus. Under current legal practice, the Blue-Bus company could well be found liable on the basis of testimony like this. But witnesses don’t always speak the truth – it’s still not certain it was a Blue-Bus bus. The witness might have been hallucinating, or might have a false memory, or could just be making up a story to smear the company. The testimony undoubtedly makes it probable that it was a Blue-Bus bus that forced me to swerve but, plausibly, it doesn’t make quite as probable as 90%. Eyewitnesses are not that reliable. Viewed in one way, the situation seems very puzzling. We’re happy for the Blue-Bus company to be found liable on the basis of the testimonial evidence, but not on the basis of the statistical evidence, even though the statistical evidence makes it more likely that the bus involved was a Blue-Bus bus.

Though it has been widely discussed in legal theory, this puzzle isn’t just a legal one – it goes deeper. Suppose you’re not involved in any legal proceedings against the Blue-Bus company, but you’re just trying to figure out for yourself what happened. If your only evidence is that 90% of the buses operating in the area were Blue-Bus buses then you shouldn’t believe that the bus involved was a Blue-Bus bus – that would be jumping to conclusions. And yet, if a witness says that the bus involved was a Blue-Bus bus, then you would be justified in believing this. So much of what we believe is based on taking others at their word. But how could I be more justified in believing that the bus was a Blue-Bus bus on the basis of testimony, when the statistical evidence makes this more likely to be true?

One possibility is that our instincts about this example, and others like it, are just wrong. Maybe we should believe that the bus involved was a Blue-Bus bus on the basis of the statistical evidence, and we’re being irrational if we refuse to believe this. And maybe the Blue-Bus company should be found liable on the strength of this evidence too, and our legal practices need to be reformed.

Then again, maybe our instincts about the example are absolutely right – and they only seem puzzling because we’re trying to understand them within a probabilistic framework. Thinking in terms of probabilities is very natural and useful, but is it the only way to look at a piece of evidence? Here is something different to consider: If a witness tells me that the bus involved was a Blue-Bus bus, and it wasn’t, then there has to be some explanation as to how this came about. Some possible explanations have been mentioned above – maybe the witness suffered a hallucination or a false memory or was lying etc. Whatever the case, there has to be more to the story – it can’t ‘just so happen’ that the bus wasn’t a Blue-Bus bus, even though the witness said that it was. But it could ‘just so happen’ that the bus wasn’t a Blue-Bus bus even though 90% of the buses operating in the area were Blue-Bus buses. While this would in a way be surprising, given the proportions involved, no special explanation would be needed.

If you believe that the bus was a Blue-Bus bus, based on testimony then, while your belief is not guaranteed to be true, something is guaranteed: if your belief is false, then this must be due to some interfering factor. This is a feature of the testimonial evidence that has nothing to do with probabilities – and it’s not shared by the statistical evidence. If you believe that the bus was a Blue-Bus bus based on the fact that 90% of the buses in the area were Blue-Bus buses then this could turn out to be simply false, and there’s nothing more to it – you took a gamble and you lost.

Is this what makes it OK to believe things on the basis of testimonial, but not statistical, evidence? Is this what makes it reasonable to find companies liable on the basis of testimonial but not statistical evidence? Maybe, maybe not. But, at the very least, it shows us that probabilities are not the whole story when it comes to understanding our evidence.

Featured image credit: Photo by Dave Gough. CC-BY-2.0 via Flickr.