Multimodal Data Platform in Life Sciences: Why Structured Data Alone Is No Longer Enough

Multimodal Data Platform in Life Sciences: Why Structured Data Alone Is No Longer Enough

Published: Oct 22nd, 2026

Most of the clinical information that explains why a patient responds to a therapy never appears in a structured field. Industry analysts have estimated that as much as 80 percent of patient information is unstructured (Healthcare IT News), and other commonly cited figures place the share between 60 and 80 percent (MedCity News). Estimates vary and some are dated, but they consistently point the same way. That information sits in clinical notes, imaging archives, pathology reports, genomic files, and device streams. Structured data, meaning coded diagnoses, laboratory values, claims, and EDC entries, records what was captured in a fixed format but it rarely records why.

The consequences reach well beyond data management. Only around 14.3 percent of investigational drugs that enter Phase 1 trials go on to reach regulatory registration (Schuhmacher et al., Drug Discovery Today, 2025), and that attrition is often tied to questions that incomplete data cannot answer: whether a biomarker generalises, whether the right patient subgroup was selected, whether a safety signal was visible early enough to act on. Analytics built only on structured records are, by design, blind to much of the evidence that bears on those questions.

The field is responding. An analysis of 8,532 registered AI-related clinical studies found that the share classified as multimodal rose from 27.8 percent in 2020 to 36.4 percent in 2025, and that new multimodal registrations grew from 79 a year in 2015 to 448 in 2025 (arXiv, 2026). Capturing that opportunity is not a matter of adding data sources. It requires a multimodal data platform able to ingest, harmonise, link, and govern data of very different kinds in a way that stands up to regulatory scrutiny.

This article sets out where structured data falls short, what multimodal approaches change across drug development, where the evidence is still maturing, and what a healthcare data platform needs in order to deliver.

What multimodal data covers in life sciences

Multimodal data is information of different types and formats that describes the same patient, disease, or population. Each modality answers a different question, and the value lies in analysing them together rather than in isolation.

Structured clinical and trial data

EHR fields, claims, laboratory results, and EDC data remain the foundation of most analytics because they are consistent and queryable. They are also limited to what someone chose to code, which is why they work best as the spine of a patient record rather than the whole of it.

Clinical notes and narrative text

Physician notes, discharge summaries, and radiology and pathology reports hold disease severity, symptom progression, the reasoning behind a change in treatment, and social context that rarely reaches a structured field. Reaching it requires natural language processing built for clinical text, with the output mapped back to standard terminologies.

Imaging and digital pathology

CT, MRI, X-ray, and digital pathology slides carry phenotypic information that no diagnosis code can describe. They are large, stored in specialised formats such as DICOM, and typically managed outside the EHR in dedicated imaging systems, which is a large part of why they are so often left out of analysis.

Genomic and molecular data

Genomic, transcriptomic, and proteomic profiles explain biological heterogeneity, including why patients with the same diagnosis respond differently to the same therapy. They are also among the hardest data to de-identify, which has direct implications for governance.

Wearable, device, and patient-reported data

Continuous monitoring and ePRO data capture what happens between clinic visits. They tend to register changes in function, adherence, or symptoms earlier than visit-based records, provided the data is validated and linked to the right participant.

Where structured data alone falls short

Structured data became the default because it is easy to query, not because it is complete. Four limitations recur across life sciences programmes.

Missing clinical context. A diagnosis code does not convey severity, symptom trajectory, or why a clinician changed a regimen. That reasoning is normally recorded in free text, so cohorts built from codes alone can group clinically different patients together.

Delayed signals. Structured fields are frequently populated after the encounter, sometimes weeks later through coding and abstraction. Wearable and patient-reported streams can show deterioration or non-adherence much earlier.

Limited biological insight. Without imaging and molecular data, it is difficult to explain why a therapy works in one subgroup and not another. That is the question that decides whether a biomarker holds up in a wider population.

Selection bias in analysis. When only well-coded data is usable, patients with sparse or inconsistently coded records are underrepresented in the results. The cohort that is easiest to analyse is not necessarily the cohort that reflects the intended treatment population.

The outcome is a healthcare data intelligence effort that is technically sound but clinically shallow: accurate about what was recorded, and silent on much of what mattered.

Where a multimodal data platform delivers value

A multimodal data platform ingests, harmonises, and links these sources so that teams can analyse them together. The practical benefits show up at several points in the development pathway, though the strength of evidence differs from one to the next.

Target and biomarker discovery

Biomarkers decide who is eligible for a trial, how efficacy is measured, and eventually what appears on a prescribing label, so errors made here propagate through every later stage. Linking genomic, imaging, and longitudinal clinical outcome data allows candidate biomarkers and responder subgroups to be examined against several kinds of evidence at once, rather than inferred from a single source.

Research models are already moving in this direction. MADRIGAL, a multimodal model from Harvard Medical School's Zitnik Lab, learns from structural, pathway, cell viability, and transcriptomic data to predict drug combination effects across 953 clinical outcomes and 21,842 compounds (Zitnik Lab, 2025). It is a preprint under review, so it illustrates the direction of travel rather than demonstrating clinical impact.

Patient stratification and trial feasibility

Genomics, electronic health records, imaging, and claims data can be combined to model which subgroups are likely to respond to a given mechanism, and whether a region has enough eligible patients to meet enrolment timelines. Multimodal models that correlate genetic variants with clinical biomarkers are one example of how stratification for clinical trials can be refined.

Natural language processing applied to clinical notes adds a further layer, since eligibility criteria such as prior therapy, disease stage, and comorbidity are often documented in free text rather than in coded fields. These are also the inputs behind enrolment velocity and screen failure rate, two of the trial metrics that most often signal early whether a study is on course.

Real-world evidence generation

Combining unstructured clinical narratives with structured records produces more complete cohorts, because patients whose key attributes live only in notes are no longer dropped from the analysis. More complete cohorts with traceable provenance make the resulting evidence more defensible when it is used to support external comparators, label expansions, or payer discussions.

Trial monitoring and early safety signals

Device and ePRO streams provide continuous data between visits. When linked to site-level and visit-level records, they can support earlier detection of adherence problems and emerging safety signals, which is the point at which a protocol can still be adjusted.

Deployment and lifecycle management

A model that performs well in a research environment and one that runs reliably inside a pharmaceutical company's production infrastructure are governed by different requirements. Moving from one to the other takes MLOps discipline: version control, performance monitoring, defined retraining triggers, and controls built around existing data governance and regulatory obligations, not bolted onto a generic deployment stack.

This matters more with multimodal inputs. A change in any one upstream source, such as a new imaging protocol or a revised coding standard, can alter model behaviour without any change to the model itself.

Why more data has not automatically meant better decisions

Does multimodal AI in life sciences improve decisions, or does it add complexity? The honest answer is that the evidence is still developing, and it is worth stating plainly rather than rounding towards optimism or scepticism. Activity is clearly rising, as the trial registry data above shows. Demonstrated, reproducible clinical impact is harder to find, and results from retrospective analyses on curated datasets do not automatically transfer to the messier data of live programmes.

The constraint is rarely the model architecture. It is the data underneath. Not every patient has every modality, so models must handle missing imaging or genomic data without silently biasing results. Linking records across systems introduces matching errors that compound as modalities are added. Differences in scanners, assay platforms, and documentation practice across sites create variation that a model can mistake for biology.

A 2026 review in Genes & Diseases reaches a similar conclusion from the development side: AI now touches stages from early lead screening through clinical trial implementation, but data reliability, verification of model predictions, and regulatory supervision are still what limits wider adoption (Genes & Diseases, 2026). The practical implication is that investment belongs first in the foundation, meaning standardised, linked, governed, and traceable data, and only then in the models built on top of it. Reversing that order often means the data problems surface after a model has already been put in front of a clinical team.

What a multimodal data platform requires

Not every system that stores several data types is a multimodal data platform. Holding files side by side is not the same as making them analysable together. The capabilities below separate a usable healthcare data platform from a collection of disconnected repositories.

Healthcare data intelligence, the practice of turning integrated clinical, operational, and patient-generated data into actionable insight, is only as good as this layer. A dashboard or model built on unharmonised, unlinked data inherits every inconsistency beneath it.

The regulatory and governance landscape

Linking more kinds of data raises the compliance bar, and the landscape is still forming. Five developments shape how a multimodal programme should be designed.

Privacy comes first. Under HIPAA, de-identification proceeds through either the Safe Harbor method or Expert Determination, and combining modalities makes both harder. Imaging files can carry identifiers in their headers or burned into the image, and genomic data is difficult to de-identify at all. De-identification and access controls therefore need to be designed into the platform, not applied to extracts afterwards.

The FDA has issued draft guidance, not final policy. The January 2025 draft guidance, “Considerations for the Use of Artificial Intelligence To Support Regulatory Decision-Making for Drug and Biological Products” [FDA-2024-D-4689], sets out a risk-based credibility assessment framework built around a defined context of use, with seven steps for planning, gathering, and documenting evidence of model credibility. It applies where an AI model generates information intended to support a regulatory decision on safety, effectiveness, or quality, and it explicitly excludes AI used in drug discovery and for operational efficiency. For a multimodal platform that feeds eligibility, endpoint, or safety analyses, data provenance and transformation history become part of the credibility evidence a sponsor needs to document.

The EMA takes a wider view. Its “Reflection Paper on the Use of Artificial Intelligence in the Medicinal Product Lifecycle”, adopted by the CHMP and CVMP on 30 September 2024, covers AI and machine learning from discovery through post-authorisation pharmacovigilance. It recommends early regulatory engagement where a system may influence a product's benefit-risk balance, and it stresses continuous monitoring for model drift as the population a model was trained on changes.

The European Health Data Space is changing access to data. Regulation (EU) 2025/327 entered into force on 26 March 2025. Most of its secondary use provisions, under which health data holders must make defined categories of electronic health data available for research and innovation through health data access bodies, apply from March 2029, with categories including clinical trial data and human genetic data following from March 2031 (McCann FitzGerald; Sidley). Individuals will also have the right to opt out of secondary use. Organisations designing a multimodal data platform now should assume that the data they hold may eventually need to be discoverable and shareable under standard permits.

Existing validation obligations still apply. Where an AI model supports a GxP-regulated activity, GAMP 5 Second Edition (2022), which added guidance on machine learning systems, and 21 CFR Part 11 audit trail and record integrity controls still apply. A model that keeps updating itself raises change control questions that a locked model never did, and those belong in validation planning, not after go-live.

Why platform scope matters more than any single model

A more capable model fed by an unchanged, fragmented data estate does not move the number that matters, which is time to a confident, defensible decision. The programmes seeing returns from multimodal AI in life sciences treat data integration as continuous infrastructure that supports discovery, trial design, execution, and evidence generation, rather than as a one-off project attached to a single use case.

That is also where implementation is hardest. Building a good model on a clean research dataset is, at this point, a fairly well-understood task. Linking and harmonising real-world data across modalities, validating outputs against clinical reality, integrating with existing pharmaceutical data infrastructure, and earning the trust of the clinical teams who must act on the results is slower, less standardised work, and it depends as much on clinical fluency as on engineering.

Life sciences work that targets this gap, including combining literature and multimodal data mining for target and biomarker validation, applying Neural ODE modelling to longitudinal clinical data, and building MLOps infrastructure that lets pharmaceutical teams run several production models without replacing existing systems, gives a sense of where the discipline is heading. Reveal HealthTech's life sciences practice is one example of a team working in this space, pairing clinical and product expertise with hands-on AI and cloud engineering on HIPAA-compliant infrastructure.

Frequently Asked Questions