What counts as clean data in a messy study

Clean data is one of those phrases that sounds obvious until you try to define it in a real study.

Pristine? Consistent? Complete? In an ideal world, maybe. But digital trials, real-world studies, and nutritional interventions aren't ideal worlds. Participants miss days. They submit entries late. They correct something they logged incorrectly. The data reflects life, and life is messy.

So here's a more useful definition: clean data is data you can trust, track, and explain. Not perfect data. Trustworthy data.

That framing isn't just a house preference. A widely cited working paper from the IMIA Primary Health Care Informatics Working Group set out to define exactly what makes routinely collected data fit for research use, and landed on six concepts: data quality as the overarching question of fitness for purpose, provenance (how the data came to be, including its lineage), extraction and processing errors, traceability (whether you can identify the origin of any single data point in your final analysis), and curation. Two of those, provenance and traceability, are doing almost all the work in the example below, and they're a more useful lens than "is this number correct" on its own.

A concrete example

A participant records their daily symptom score on days 1 through 5. Day 6: nothing. Day 7: they log two entries, one for today and one to catch up on yesterday.

Is that dataset clean? It depends entirely on what the system captured. If the platform recorded accurate timestamps for both entries, flagged the back-entry clearly, and included it in the audit trail as a late submission, then yes. The data is messy in behaviour, but clean in documentation, which is precisely what provenance and traceability describe: not whether the data arrived on a tidy schedule, but whether its origin and history can still be reconstructed afterwards.

What makes it dirty is different: the second entry silently overwriting the first with no version history, a monitor editing a date to make the submission appear on-time, or no way to tell which entry was original and which was corrected. The mess isn't the problem. The invisibility of the mess is.

What clean actually looks like in practice

In digital and real-world studies, trustworthy data has a few consistent characteristics. It reflects what actually happened, including gaps, late entries, and participant-led corrections, rather than a version tidied up after the fact. The context travels with it: timestamps, submission method, who entered it and when. Changes are visible, not buried, meaning edits are versioned rather than overwritten. And ambiguity is acknowledged rather than cleaned away by someone trying to make the dataset look tidier than it actually was.

That last point is particularly relevant for observational or nutritional studies, where variability is often part of the signal. Aggressively normalising messy data doesn't improve it. It erases the texture that makes it useful, and it's a step the framework above would class as an unrecorded processing error rather than legitimate cleaning.

The questions that matter for audit

If the data is clean, you should be able to answer these:

1. What did the participant actually do?
2. When did they do it?
3. Was anything changed after initial entry, and if so, by whom and why?
4. Can you stand behind this number if an auditor asks about it?

If yes, you're in good shape, even if the number arrived late, was revised twice, and came with a coordinator note explaining the circumstances. That's not a mess. That's a study that ran as they all do, in the real world and not the protocol document.

Clean data isn't a standard you achieve at the end of a study. It's an infrastructure you build at the start, so that by the time you reach analysis, the messy parts are documented rather than hidden.