Oxford University Press's
Academic Insights for the Thinking World

The problem with overqualified research

Not all research findings turn out to be true. Of those that are tested, some will need to be amplified, others refined or circumscribed, and some even rejected. Practicing researchers learn quickly to qualify their claims, taking into account the possibility of improved measurements, more stringent analyses, new interpretations, and, in the extreme, experimental or theoretical error.

Qualification is routinely achieved by changing the form of the verb. Take this example from a physics article: “This lack of correlation may be due to the increased inhomogeneity expected for the MG method.” The authors’ use of “may” allows for the lack of correlation to be due to something else.

The softening of any statement with a word or words is called hedging. It can be accomplished in many ways. One has already been indicated—inserting an auxiliary modal verb, “may,” “might,” “can,” or “could.” Another common way is to add a mitigating phrase or clause such as “It is possible that” or “This suggests that.”

Unhappily, the temptation for an author is to keep accumulating hedges to secure progressively more protection against potential criticism. As a result, it becomes correspondingly harder for the reader to see what the author is actually  claiming.

In these two sentences from a molecular biology article, there’s a doubling of hedges, here in italics: “it is possible that … G9a-mediated H3K9 and H3K27 monomethylation might serve as stimulating factors” and “Possible candidates might comprise Jarid2 and Snail1.” Both instances of “might” can be eliminated without affecting the provisional status of the claims.

Altogether this article contained 44 instances of “might,” “can,” “could,” “likely,” and varieties of “possibly,” but none of “may.” Because of its potential for ambiguity, some authorities advise authors not to use “may” when what’s being signalled is possibility instead of permission.

This next example is from an oncology article. It has five hedges in one sentence: “Although, on balance, testosterone might possibly be beneficial to the cardiovascular system, testosterone might also have some detrimental effects.” Two of the hedges can be safely eliminated; for example, “Although testosterone might be beneficial to the cardiovascular system, testosterone might also have some detrimental effects.” Yet with two opposing uses of “might,” the sense of equivocation remains, despite the restrictive “some.” Given that “might” is seen as being more tentative than “may,” a less equivocal version is “Although testosterone may be beneficial to the cardiovascular system, it might also have some detrimental effects.” Whether this edit is valid, though, needs expert judgement. In all, this article contained 54 instances of “might.”

There can be very good reasons for using “might” so often—but it’s unusual.  I estimated its general frequency of occurrence from an analysis of a sample of about 1000 articles from the physical, life, and behavioural sciences, mathematics, engineering, and medicine. Across the sample, 50% of articles containing “might” used it just once or twice and only 1% used it as many as 21 times or more.

In a more detailed analysis, I also found that as the frequency of the modals “may,” “might,” “can,” and “could”  increased in an article, the frequency of pairs of hedges such as “could indicate” and “possibly suggests” increased still more. The implication is that multiple hedges within a sentence are a response to the prevailing rate of protection within the article—rather like having to talk more loudly in a noisy restaurant.

Whatever the cause, hedging can clearly escalate. But there’s a countervailing force: the need to deliver impact, usually in the abstract or conclusion of an article—something to be remembered by the reader, a take-home message. Since impact is lessened by hedging, summary statements may succumb to under-qualification and go beyond what the data can safely support. At worst, they can lead to public correction and retraction of the article.

How, then, to choose the right amount of hedging?  Every claim is unique to its particular context and it’s difficult to generalize. Even so, there’s some strategic advice, popularly attributed to Albert Einstein, that may be helpful. Applied to hedging, it’s this: Hedging should be as simple as possible—but never simpler.  Although finding the right balance is another challenge, it at least encourages restraint.

Featured Image Credits:  Julian Schöll on Unsplash 

Recent Comments

  1. Gio Wiederhold

    If exceptions or ranges of validity can be stated, that is yet better than any hedging. I have seen hedging actually used to convey what a smart scientist one is, recognizing limits of the work, when the scope was actually clear.

  2. Ben Craven

    I wonder about the necessity for “some” in the oncology sentence. What possibilities does it exclude that its omission wouldn’t also exclude? The first part of the sentence has already acknowledged that the effects of testosterone may not be *all* detrimental.

  3. Susana Chung

    This is a very thought-provoking article. Indeed there is a need to balance between overqualifying one’s results, and making big claims that are not solidly supported by data. I will definitely pay more attention to hedging in my own writing and when editing papers.

  4. Dimitris Mylonas

    Bringing certainty in an uncertain area of research is the main goal of a scientific publication but in our times where the big effects are rare it is indeed difficult to make strong claims.

  5. adam reeves

    Researchers should not hedge. Either prior knowledge, current data and careful reasoning together force a conclusion, or they do not.

  6. Jan Morovic

    This may well be the best exposition of this topic that one could ever hope to find. 😊 On a more serious note, I think that the point about an imbalance between the level of hedging in an abstract and the body of an article is particularly relevant. As Prof. Foster says though, hedging is a tricky question and it is about its appropriate application rather than wholesale omission.

  7. Chris Dainty

    I’d say that over-qualification is more prevalent in the life sciences, because the interpretation of measurements and data is nearly always much more difficult than the physical sciences. It is pity that authors are effectively barred from saying “we cannot explain these measurements” or similar phrases; over-qualification allows them to waffle a bit, and often enables them to get accepted in a higher impact journal. Life scientists seem obsessed with impact factor and journal status … there’s nothing wrong with the journals that your peers publish in.

  8. Charles Leek

    Absolutely on the mark. On the one hand, some appropriate degree of ‘Albertian’ hedging – where this helps completeness of evaluation by the scientific community, is, of course, appropriate. Mature scientists (should ) have the confidence to defend their claims – but also to acknowledge appropriate (feasible) limitations. Possibly a reflection of exterior pressures to publish or peril.

  9. Wuliang Yin

    A thought-provoking article. The hedging in publications is on the increase and the hedging in research proposals is in the opposite direction. Do people promise more and deliver less in general?

  10. jasna martinovic

    It would be important to more explicitly educate not just the scientists but also the wider public about the role of hedging in scientific writing. Perhaps the variation in between articles stems from the fact that a lot of people learn it implicitly, through reading articles and accommodating to reviewers’ requests. If scientists are educated about the way hedging should be done, writing would get better. If the public is educated about the reasons why hedging is done, it might help them understand science better – i have recently been told by a software engineer that science should not get so much funding as it is mostly a waste of time – when he looks at papers, very often he sees that “A could influence B if C”, when he would expect it to describe how A actually influences B under a myrad of conditions. So at least some members of the public have expectations of research programmes with a very large scope and degree of accuracy, while the reality of what scientists aim (or are able to) to achieve and publish in one unit of output is much more modest – and thus requires hedging.

  11. Gaoyang

    It is the first time actually I understand what does ‘hedging’ mean in English?
    If translated literally, it means ‘ride on the fence’ in Chinese.
    I often found being ambiguous is not a good idea in scientific writing. Author often doesn’t explain words such as ‘may’.

    But normally it is true that in abstract you find some idea is over-stretching and in main body the tone is suddenly softer.

  12. Vien Cheung

    Surely the same issue occurs in non-verbal arguments, for example, confidence intervals in statistics or uncertainty in metrology. Is this really any difference? The way to resolve this is to propose how future research might resolve the uncertainties that have been identified.

  13. C. Alejandro Parraga

    Thanks David for such an excellent article. I do believe that one of the fundamental tenets of science is acknowledging that one does not hold the ultimate truth. No matter how good the data or the theory is, it is likely to be superseded by a better theory and/or better data, and we might not forget that. However, you are right that in some cases “hedging” might simply be too much. If I have to chose, I would prefer to err towards the “too much” side.

  14. Subhasish Chakraborty

    Interesting stuff and a well-written article. I suppose over-qualification is everywhere – what we generally call political correctness. It is necessary but the real craftsmanship is when things are written in a bold language with a robust argument. But would reviewers like that?

  15. Xavier Otazu

    May be my comment is too simple, but I see a relationship between hedging and the previous blog post about “Future work”. From my point of view, hedging should be only used when talking about possible future work, about the possible explanations of some new findings in the present work that have to be explored in the future. Science is both firmly grounded results and conclusions (with no hedging) and some new answers and perspectives to be explored in the future (here hedging is allowed).

Comments are closed.