Opportunity | Key Issues with Compiling Healthcare Provider (HCP) Data
Our client approached Reveal HealthTech to address the differing needs of their Commercial and Medical teams, both of which sought deeper insights into healthcare providers (HCPs) within their therapeutic areas. However, the challenges were centered around aggregating diverse data sources—both public, such as PubMed and Citeline, and internal proprietary datasets—to create a unified and holistic view of HCPs. Additionally, the solution required scalability & extensibility to accommodate new data sources, expand across various therapeutic areas, and meet the evolving needs of different teams within the organization.
Approach | The AWS-Driven Solution Blueprint
Reveal HealthTech’s approach began with a deep engagement with our client’s Commercial and Medical leaders to fully understand their distinct needs. This collaborative process enabled the creation of a comprehensive product roadmap designed to meet the current and future demands of the organization. By carefully defining the requirements for each functional and therapeutic area, our team was able to ensure that the solution was customized and aligned with the company’s overall goals.
At the core of this approach was a robust, scalable data platform. The platform was designed to seamlessly integrate with new data sources while accommodating various data formats, including CSV, relational databases, JSON, and images. By connecting to systems like S3, SFTP, and RDBMS, the platform facilitated flexible data ingestion and management. Automated data ingestion pipelines, coupled with rigorous data quality checks, ensured that the data used was accurate and reliable. The entire platform was built on scalable, open-source technologies, hosted in a cloud environment for maximum flexibility and growth potential.
Additionally, Reveal’s engineers developed user-facing applications that were specifically tailored to individual therapy areas and functions, whether for Commercial or Medical teams. The application platform allowed for rapid profiling, data ingestion, and experimentation, providing the flexibility to adapt to new data sources or user interfaces quickly. These tailored applications drove business value by empowering users to derive actionable insights efficiently, ultimately enhancing the organization’s ability to meet its strategic objectives.
Reveal leveraged advanced AWS-based infrastructure to build a robust, scalable, cloud-based platform. Key components of the solution included:
- Data Aggregation Layer: AWS Glue was employed for efficient ETL (extract, transform, load) processes, enabling the seamless integration of diverse data sources. Data was securely stored in Amazon S3, while AWS Lake Formation managed data access policies to ensure secure, scalable, and compliant data integration.
- Data Access Control Layer: AWS Lake Formation enforced strict access controls, while AWS CloudTrail provided auditing capabilities. Metadata related to file management was stored in DynamoDB, and operational logs were tracked through CloudTrail to maintain visibility and compliance across data handling processes.
- Application Integration Layer: AWS Athena was used for fast, serverless querying of the aggregated data, while AWS API Gateway enabled secure, scalable access to user-facing applications, ensuring smooth interaction between the data platform and the end-user applications.
Features | Compliant and Scalable Data
- Scalable Data Platform: AWS Glue and S3 allow the data platform to easily integrate additional data sources and formats. This ensures the solution can grow with the client’s evolving needs while maintaining flexibility and efficiency.
- Robust Data Access and Compliance Controls: With AWS Lake Formation managing data access, DynamoDB handling metadata, and AWS CloudTrail providing detailed audit logs, the design ensures secure, transparent, and compliant handling of sensitive healthcare data. This enabled the client to maintain control over data access while adhering to regulatory standards.
- Tailored User Applications with Seamless Integration: By leveraging AWS Athena for efficient data querying and AWS API Gateway for secure application access, the solution supports multiple user applications developed for diverse therapeutic area teams. Additionally, automatic updates to a relational database ensure the aggregated, clean data is ready for AI and ML experimentation, allowing the client to easily launch AI/ML initiatives with minimal upfront investment when they are ready.
Outcomes | Impact and Improvements
- User-facing Products for Separate Therapeutic Areas: Leveraging the solid foundation of the data platform, two applications tailored to individual therapeutic areas were quickly developed, ensuring that the teams could easily access and utilize the enriched HCP data relevant to their focus. This streamlined development process highlighted the scalability and flexibility of the platform, enabling rapid deployment of user-facing solutions.
- Integration with existing systems: The cleaned, normalized data and insights were integrated with the client’s decision support systems (DSS) downstream, unlocking additional potential of their data.
Conclusion
The investment in building a scalable and extensible data platform delivered significant value to the client, enabling the rapid development of two successful applications for different therapeutic areas. This foundation not only meets the current needs of the organization but is also poised to support future growth as new data sources and emerging technologies can be seamlessly integrated into the platform. As the client continues to innovate, their data platform will remain a critical asset, driving business value and enabling AI and ML experimentation.
To learn how Reveal HealthTech can help your company, contact our team.