Research Impact · Neuroscience

The eighteen-month window

Early detection research promises diagnostic lead time. A closer look at what that lead time is worth in routine clinical practice — and at the validation flaw that inflated a decade of estimates.

Reassessment of published lead-time estimates required re-examining the validation design of prior work.
Reassessment of published lead-time estimates required re-examining the validation design of prior work.

The arithmetic of early detection is seductive. If a model can identify degenerative change before symptoms present, the diagnostic window opens, interventions become available that were previously foreclosed, and the disease — clinically speaking — becomes a different condition.

The literature has reported such lead times for over a decade, often with impressive figures. A reassessment published in 2024 established that a substantial portion of those figures were artefacts of validation design.

The mechanism is subtle. Longitudinal cohorts are expensive, so researchers reuse them, and the standard practice of cross-validation splits such cohorts by participant rather than by time. A model trained on later observations from some participants and tested on earlier observations from others has, in effect, been shown the future. The resulting performance estimate is not fraudulent. It is simply answering a different question than the one the paper claims to answer.

Correcting for this — using strictly forward-looking splits, as a clinical deployment would face — produces materially lower figures. In the reassessment, mean lead time fell to around nineteen months from published estimates roughly twice that. Nineteen months is still clinically significant. It is not what the field had been telling itself.

What lead time is worth. Here the evidence thins considerably. A diagnosis arriving nineteen months earlier is valuable if something can be done in those nineteen months, and the answer depends on the condition, the availability of intervention, and the willingness of a health system to act on a probabilistic finding in an asymptomatic patient. These are not modelling questions.

The prospective studies now underway in several groups are the first attempt to answer them directly: not whether a model can predict, but whether prediction changes a care pathway. Early indications suggest the bottleneck is institutional rather than technical. Clinicians presented with a probabilistic early finding in an asymptomatic patient have, in many systems, no protocol to follow.

That is a solvable problem, but it is not solved by better models. It is solved by the considerably less glamorous work of writing protocols, which is where a portion of the field's attention has now sensibly turned.

Share

Corrections? Contact the desk.

Continue reading

More on neuroscience

All writing
The Aeternum Dispatch

New profiles and research writing, once a month

A short editorial letter listing newly published profiles and the research pieces accompanying them. No advertising, no third-party sharing.

By subscribing you agree to our privacy notice.