The 10 biggest data collection challenges and frustrations: a researcher's guide

We spoke with researchers from academia and industry across a wide range of disciplines: nutrition, public health, neurodegeneration, cancer, immunology, microbiome research, and others. Many were vocal about the specific frustrations that make data collection harder than it should be. These are the ten that came up most consistently.

1. "Not another portal"

Digital tools can improve productivity, but a proliferation of separate logins creates its own problems. Usernames and passwords scattered across many platforms, with humans managing all of them, is a genuine security risk. Some researchers admitted storing passwords on sticky notes near their workstations because there were simply too many to remember.

The solution is integration. Single sign-on (SSO) and API connectivity between platforms reduce the number of entry points and lower the risk of a breach through a forgotten or reused credential. All-in-one solutions that replace multiple separate tools are worth the consolidation effort.

2. Systems that do not talk to each other

Even when platforms are individually good, disconnected systems create a hidden workload. Researchers described copying and restructuring data tables between tools manually, which introduces formatting errors, data loss risk, and significant staff time costs.

API connectivity between services is no longer a nice-to-have. A platform that cannot share data with the rest of your infrastructure creates friction at every stage of the study.

3. Platforms that are too hard to use

Usability directly affects compliance. When end users (staff and participants alike) struggle with a platform, they avoid using it, use it incorrectly, or give up. Cloud-based, browser-native tools have a significant advantage here: they are not tied to a specific operating system, scale across devices, and can be updated without requiring individual installations.

Good documentation and accessible customer support matter too, particularly for teams without a dedicated technology lead.

4. No support and no humans to talk to

Out-of-the-box software often comes with a manual. That is not the same as support. For teams without in-house IT capability, setting up complex data collection systems requires a knowledgeable point of contact at the vendor. Without it, the default is to appoint an internal "digital champion" who ends up doing technical work on top of their existing role.

5. IT teams required to operate the platforms

Academic researchers in particular highlighted this as a significant barrier. Many open-source platforms are free to license but require substantial IT infrastructure to install, host, and maintain. For organisations that cannot get in-house IT support, that effectively makes the platform unusable.

Cloud-hosted platforms shift these responsibilities to the provider. The trade-off is that the provider needs to be transparent about their security protocols and compliant with relevant regulatory frameworks. Check that before signing anything.

6. Data, encryption, and security concerns

Sensitive health data attracts significant regulatory obligations and attracts malicious attention. When evaluating any platform, ask:

• Is the login process sufficiently secure?
• Are password policies, two-factor authentication, and SSO supported?
• Are credentials encrypted?
• Can you prove who accessed what, and when?
• Is data encrypted in transit and at rest?
• Are data backed up, and how?
• Can you export your data as needed?
• Are full audit logs available?
• Does all of the above comply with the relevant regulatory frameworks?

Remote and decentralised working adds another layer: are participants accessing your systems securely? A link that anyone could follow without authentication is not a secure data collection method.

7. Regulatory compliance uncertainty

Most clinical research staff will have GCP training. But the software they use needs to meet the same standards. Is your platform provider aware of MHRA, EMA, and FDA requirements? Can they demonstrate compliance? If your organisation is self-hosting, does your IT infrastructure support the security level needed?

For a worst-case scenario, you want to know that participant safety was prioritised and that all protocols were followed. A software provider that cannot speak clearly about regulatory compliance is a liability.

8. Nothing does everything you need

Research requirements are varied: image and video data, cognitive assessments, food diary integrations, multilingual support, wearable connectivity. No single platform covers all of these for all studies. The question is whether your platform can be extended through bespoke development or API integration, or whether it is a closed system.

A good software partner is willing to work with you on features that go beyond their standard offering. Rejection without flexibility means more platforms, more logins, and more risk.

9. Budget constraints

Software always has a cost. Even free tools carry invisible costs in staff time, IT support, and training. The most accurate comparison is total cost of ownership: what does this system actually cost across the life of the study when all time and resources are accounted for?

Open source or free platforms:

• No licence fee
• Often no support available
• Setup, maintenance, and security are your responsibility
• May not be regularly updated
• May create a false sense that responsibilities are reduced

Paid or subscription platforms:

• Usually include support and regular updates
• Responsibility is shared or shifted to the provider
• More likely to include integrations and configurable features
• Can sometimes be worked with to develop bespoke solutions
• Licence costs can be a barrier for academic researchers with restricted grant budgets

10. Unreliable or untrustworthy systems

Many researchers described their current setup as "not perfect, but close enough." That is not a comfortable position when participant safety and data integrity are at stake.

The access barrier in particular is worth taking seriously rather than assuming smartphones have solved it. A study surveying 202 patients in a diverse, multilingual urban safety-net healthcare system found strong interest in remote digital tools (65.3% were interested in trying a video visit), but only 54% actually managed to complete one successfully. Internet or mobile data access, not unfamiliarity or reluctance, was the single most common barrier, cited by nearly a quarter of participants. Interest in digital participation and the practical ability to follow through on it are two different things, and a study design that only measures the first will systematically miss why some participants quietly disengage.

Traditional approaches come with familiar problems:

• Post is slow, unreliable, and sometimes lost
• Handwriting is increasingly illegible as people write less by hand
• Phone calls are often not answered, particularly from unfamiliar numbers
• Physical documents are a burden for participants to manage

Digital approaches have their own failure modes:

• Email is unreliable and frequently filtered as spam
• Not everyone has a smartphone or sufficient storage for apps
• Poorly designed systems deter participation from both staff and participants
• Password reuse across platforms increases breach risk significantly

The pattern across all ten points is the same: the cost of poor data collection infrastructure is real, recurring, and largely invisible until something goes wrong.