What Performance Data Leaves Out

Talent identification asks who deserves further observation or support. Selection asks who receives a particular place now. Those questions need different evidence and different tolerance for error. An exploratory system may reasonably produce a wide group for review. A selection process must explain why its evidence is relevant to a specific role, competition and development period.

Start with an operational question: which athletes might benefit from an additional observation session? That is more defensible than asking a database to reveal “future stars.” Future achievement depends on changing circumstances, including coaching, health, access and personal priorities. A record describes an encounter with those circumstances, not an athlete’s fixed ceiling.

Decide what would count as useful evidence before collecting it. Otherwise, an available metric can quietly become the definition of talent simply because it is easy to rank.

A sprint result depends on the start procedure, timing technology, distance, surface and recovery interval. A hand-timed effort and an electronic-gate effort should not be treated as interchangeable. Even identical devices can produce different records when placed or configured differently. Preserve the protocol alongside the result, including calibration checks and known interruptions.

World Athletics technical rules show how carefully formal performances are defined. FIFA’s Quality Programme illustrates a related principle for football technology: measurement systems need evaluation under specified conditions. Neither reference makes a local academy’s dataset automatically comparable with elite competition data. Local collection still requires a documented procedure and staff who follow it.

Reliability and relevance are separate tests. A device can measure something consistently while that measure remains only weakly connected to the scouting question. Do not confuse a precise output with an appropriate target.

Physical output is inseparable from opportunity. A substitute has fewer minutes to accumulate actions; a defender in a low block faces a different task from a full-back asked to advance repeatedly. Video can clarify those conditions, although camera position and annotation quality introduce their own limitations.

Developmental context matters especially when young people are involved. Relative age and biological maturation can affect current physical performance without settling long-term development. Maturity estimates are themselves uncertain and should not become another unquestioned ranking. Medical or sensitive developmental information needs a specific, proportionate justification and suitable protection.

The International Olympic Committee’s consensus work on youth athletic development and the Australian Institute of Sport’s development resources provide useful starting points for a broader view. They support asking about development environments, not treating an early advantage as a guaranteed destination.

Missingness is often systematic. Well-resourced clubs may have more cameras, wearable devices and analysts. Rural athletes may travel farther for testing. Someone recovering from injury may have fewer recent records. A dataset can therefore reward visibility while appearing to reward ability.

Document why values are absent where possible. Do not replace every missing score with an average and assume the problem has disappeared. That substitution can conceal the very access barrier the discovery programme should address. Keep missingness indicators available for audit, but test whether a model uses them as proxies for privilege.

Coverage should be reviewed against the programme’s intended population. Sport England participation research can help frame questions about access in England, but population participation statistics cannot establish the representativeness of a particular scouting sample. That requires examining the actual routes into the dataset.

A useful evidence record includes the source, date, protocol, conditions, athlete role, permissions and known limitations. Preserve corrections rather than silently overwriting history. Record how long information is needed, who can access it and how an athlete or representative can ask about an error.

The UK Information Commissioner’s Office explains data protection principles including accuracy, minimisation and storage limitation. Those principles are practical safeguards, not just administrative paperwork. Collecting everything “in case it becomes useful” increases risk and makes it harder to explain a decision.

Before modelling, ask an independent colleague to trace one record from collection to interpretation. Can they reproduce the meaning without asking the original observer? If not, improve the evidence trail first. More sophisticated analysis will not repair an undocumented measurement.