Oxford University Press's
Academic Insights for the Thinking World

Why we need “mystery shoppers” directly observing health care

Considering the well documented problems of medical error, it’s remarkable that it’s rarely observed. Of course there is much scrutiny of the data that is generated during the health care encounter, but that is not the same thing. For instance, while quality measures track data on how well blood pressure is managed, there are not measures of whether blood pressure is actually measured accurately. To know if it’s measured correctly, you’d have to be there, know what to look for, and see if it’s taken correctly. If the blood pressure cuff is the wrong size, or the patient has not been resting for at least 3 minutes, or their arm is not at the level of their heart and their feet on the ground, the measurement upon which everything else is based may not be accurate. This is true for much of the data upon which we rely when assessing quality. Unfortunately, the evidence is that a good deal of what is recorded in the medical record is inaccurate.

How is this known? Through undercover direct observation of care: A relatively small number of researchers have trained actors to present as real patients, tell a narrative that will prompt certain questions and tests, and then observe what happens next – a kind of “mystery shopper” who knows what to look for in the health care setting. The technical term for these individuals is “unannounced standardized patient” (USP) because they adhere to a script, allowing comparison of how different doctors manage the same patient. Such studies are eye opening: At a top tier medical center 33% of physical exam maneuvers recorded in the medical record were never conducted. Conversely about 16% of preventive care services rendered are not documented. In one study, a USP showed up at many primary care doctors’ offices saying she was going to be working as a life guard and needed a pre-work physical. The actor was an 18 year old fair skinned woman. One third of the doctors provided no skin protective counselling, not even recommendations to use a sunscreen. An audit of the medical record, however, showed perfectly well conducted new patient visits, regardless of whether the patient was counselled about how to prevent sun burns. There are many such published examples.

Given the plethora of evidence that much is not well (besides the patient) behind the closed door to an exam room, why is no one doing anything about it? After all, “mystery shoppers” are widely used in retail, including hotels, restaurants, and shopping centers to assess the performance of employers. Aren’t sick people at least as important as folks looking for a place to eat or spend the night?

Image Credit: Doctor recording notes on patient chart by Wesley Wilson. CC0 via Pexels.
Doctor recording notes on patient chart by Wesley Wilson. CC0 via Pexels.

There are several reasons: First, it’s more complex to send an uncover observer of care (i.e. a USP) into a health care setting undetected than it is to send a secret shopper into a mall. We’ve sent hundreds, though, in many practice settings, so it’s doable. Others have as well and detection rates are in the range of 10-15%. USPs makes it possible to collect authentic data on how care is actually delivered with no “Hawthorne Effect.” Such an approach has yielded the remarkable findings noted above, illustrating the chasm between what we think patients are getting and what care really looks like.

Second, among those who know about USPs, there is skepticism that the method could ever be scalable. Given the thousands of clinical settings around the country, how could one possibly deploy fake patients to all of them on a regular basis? To answer that question one would need to know how much money could be saved by correcting all these hidden deficits in care, and whether better outcomes for patients brought additional value. If the hospitality industry thinks it’s worth the expense, the health care industry might arrive at a similar conclusion.

Finally, the third reason USPs are underemployed is that they are scary. Who wants to endorse being spied on? In fact, a few years ago the American Medical Association tabled a recommendation by its own ethics committee to endorse the use of “secret shopper” patients in health care. The way to solve this problem, though, is to make the process safe. This is done by treating it like “peer review,” in which doctors receive feedback on their performance but that data cannot be used for disciplinary purposes or to cut pay. And the data cannot be subpoenaed.

Those of us who work with USPs to directly observe care often feel like we have not only a secret shopper but also a secret weapon to addressing much of what ails health care. What will it take for USPs to become as common in health care as they are in retail? It may require a combination of consumer and payer expectation. Anyone who has been or will be a patient needs to demand that quality assessment reaches into the exam room, so that they can be sure that the care they are receiving is in fact quality care. Payers, such as Medicare or Blue Cross Blue Shield, will need to insist that if they are paying for that diabetic foot exam, they need to know that the doctor didn’t just document they did one, but that they actually did do one and did it correctly (it turns out that this is not always the case more than 50% of the time).

It’s human nature to focus attention on what is assessed. Right now we are measuring how well doctors and other health care providers document what they do, rather than what they actually do. The result is a medical record that gets the best of care, and patients who often get a lower quality of service than the medical record indicates. Isn’t it time to pay as much attention to the patients as their charts?

Featured image credit: Stethoscope by Wesley Wilson. CC0 via Pexels.

Recent Comments

There are currently no comments.