Harmonization-Aware Predictive Overlap Forecasting for Population-Scale Pharmaceutical Need and Utilization Sensing

1. Introduction

The digitization of healthcare has created vast repositories of electronic health record (EHR) data, yet translating this data into actionable forecasts for pharmaceutical demand remains a formidable challenge [6]. Traditional approaches rely on isolated institutional data, deterministic ETL pipelines, and simplistic time-series models that cannot account for the complexity of clinical trajectories, semantic heterogeneity, or the distinction between clinical need and realized utilization. Recent advances in semantic intelligence [1, 3, 6], federated learning [5], and quantum-inspired harmonization [7] have shown promise in making multi-site clinical data comparable and analyzable while preserving privacy and uncertainty. However, these efforts have focused primarily on retrospective analytics (e.g., readmission prediction) rather than forward-looking, operational forecasting.

The present work introduces a novel framework—Harmonization-Aware Predictive Overlap Forecasting—that addresses these gaps. We formalize the concept of a harmonized overlap score, which measures the expected intersection between a harmonized population state and an external decision object (EDO) such as a pharmaceutical product, therapy class, or formulary change. From this score we derive three distinct forecasts: clinical need (the population-level requirement for a therapy), realized utilization (actual consumption under operational constraints), and strategic opportunity (potential value or risk exposure). By leveraging an uncertainty-preserving harmonization substrate (Data Harmonization Model, DHM) [3], semantic grounding (Semantic Classification Engine, SCE) [1], and a governed multi-site learning architecture (EMI) [4], the framework provides a scalable, privacy-preserving, and explainable solution for population-scale forecasting.

Our contributions are:

1. A formal definition of harmonized overlap forecasting and its decomposition into need, utilization, and opportunity.
2. An integrated architecture that connects semantic extraction, harmonization, and forecasting layers with explicit uncertainty propagation.
3. Comprehensive simulation experiments that demonstrate superior performance over baselines, ablation analyses, fairness assessment, and federated learning convergence.
4. Publicly available code to reproduce all figures and results.

2. Related Work

2.1 Semantic Intelligence and Harmonization

The Semantic Intelligence Framework [6] introduced graph-based concept extraction, statistically validated relationship discovery, and temporal pattern analysis for clinical data warehouses. The Semantic Classification Engine (SCE) [1] provides multi-ontology semantic typing and evidence graphs. The Data Harmonization Model (DHM) [3] extends these with versioned Harmonization Bundles, fidelity metrics, and uncertainty-preserving alignments across sites. These foundations enable the semantic comparability required for cross-site forecasting.

2.2 Federated Learning in Healthcare

Federated learning has emerged as a privacy-preserving paradigm for multi-institutional predictive modeling. A recent Phase 1 implementation across 47 U.S. hospitals [5] demonstrated the feasibility of federated XGBoost for 30-day readmission prediction, achieving an AUROC of 0.76 with strong privacy guarantees (\(\epsilon = 0.68\), \(\delta = 10^{-5}\)). Our framework builds on this by extending federated learning to forecasting tasks with branch-aware harmonization.

2.3 Quantum-Inspired and Biomimetic Approaches

Recent work has explored quantum-inspired formalisms for data harmonization, representing clinical states as Hilbert space vectors and using decoherence-like mechanisms to achieve consistent histories [7]. These techniques provide a natural language for uncertainty and branching, which we adopt to represent multiple plausible interpretations of patient data.

2.4 Pharmaceutical Demand Forecasting

The literature on pharmaceutical demand forecasting includes methods based on time-series (ARIMA, Prophet), machine learning (XGBoost, LSTM), and more recently knowledge graph-enhanced models like KG-GCN-LSTM [12]. Generative EHR trajectory models such as Foresight [8], DT-GPT [10], and Delphi-2M [11] can forecast patient-level disease trajectories but do not address cross-site harmonization or the need–utilization distinction. Our work is the first to integrate uncertainty-preserving harmonization with a two-stage need–utilization forecasting pipeline.

3. Methods

3.1 Formal Framework

Let \(C\) be the harmonized concept space produced by the semantic and harmonization stack. For site \(s\) at time \(t\), the branch-aware harmonized population state is:

where \(B_{s,t}\) is the set of active branches (each representing an alternative data interpretation), \(\pi_{b,s,t}\) the branch weight, and \(p(c \mid b, s, t)\) the branch-specific concept prevalence. For an external decision object \(o\), we define its harmonized signature \(\phi_o(c)\) over the same concept space. The harmonized overlap score is:

with \(A_{s,t}(c)\) an access/activation modifier (formulary status, supply constraints, etc.).

From \(\Omega_{o,s,t}\) we generate three forecasts:

where \(X^{\mathrm{traj}}\) denotes trajectory features (diagnosis, procedure, medication histories), \(Z^{\mathrm{exo}}\) exogenous signals (seasonality, shortages, policy changes), \(Q^{\mathrm{access}}\) site-level access barriers, \(S^{\mathrm{supply}}\) supply constraints, and \(M^{\mathrm{market}}\) market dynamics.

By explicitly separating clinical need from realized utilization, the framework provides more interpretable and actionable forecasts.

3.2 Architecture and Implementation

Figure 1 illustrates the five-layer architecture of the proposed framework. The lowest layer, Acquisition & Workflow Control, ingests heterogeneous EHR data via HHDS, Kafka eventing, and Temporal orchestration. The Semantic Substrate (SCE and Semantic Intelligence Framework) extracts concepts, relationships, and temporal patterns from structured and unstructured EHR, producing calibrated evidence graphs. The Harmonization Substrate (DHM) then creates versioned Harmonization Bundles and Evidence Graphs, explicitly preserving branch uncertainties and fidelity metrics. The Forecasting Engine fuses trajectory features, harmonized overlap scores, exogenous market signals, and operational modifiers using a hybrid model: XGBoost for need forecasting and LSTM for utilization forecasting. Federated training follows the differential privacy protocol described in [5]. Finally, the Decision Outputs & Governance layer provides distributions (need, utilization, opportunity), uncertainty intervals, fairness metrics, and governance artifacts (ticket stage, model version). A feedback loop ensures continuous improvement and rollback capabilities.

3.3 Federated Learning and Privacy

We implement federated XGBoost using histogram-based gradient aggregation with differential privacy. Each institution computes local gradient and Hessian histograms with per-example clipping (\(C = 1.0\)). The central server aggregates with Gaussian noise (\(\sigma = 2.0\)) and uses the moments accountant to guarantee (\(\epsilon = 0.68\), \(\delta = 10^{-5}\)) patient-level privacy under observed participation rates. The privacy budget accumulates over communication rounds; Figure 6c shows the evolution of \(\epsilon\) over 100 rounds. The privacy-utility trade-off (Figure 6d) demonstrates that moderate privacy loss (\(\epsilon \approx 1.0\)) yields negligible utility degradation.

3.4 Simulated Dataset

We constructed a synthetic dataset representing 1 million patients across 47 simulated sites, with 12 months of historical data. The data includes demographics, diagnoses (ICD-10), medications (RxNorm), procedures, and clinical notes (simulated). Ground truth for need (clinical indication for a therapy class) and utilization (actual orders) was generated using a rule-based model that reflects realistic clinical pathways, supply constraints, and seasonal patterns. Five therapy classes were used as target EDOs.

3.5 Baselines and Ablations

We compared our full framework against:

• Site-local: Model trained on each site independently.
• Harmonized (collapsed): Harmonization performed but uncertainty collapsed to a single branch.
• Harmonized (uncertainty-preserving): Branch weights preserved but no exogenous signals.
• Harmonized + Exogenous: Including external signals but without need–util separation.

Ablations removed each key component (harmonization, uncertainty preservation, exogenous signals, need–util separation) from the full model.

3.6 Evaluation Metrics

For need forecasting we report AUROC, Brier score, and expected calibration error (ECE). For utilization forecasting we report symmetric mean absolute percentage error (SMAPE), root mean squared error (RMSE), and quantile calibration. Fairness is assessed using maximum pairwise AUROC disparity, equal opportunity difference (EOD), and demographic parity difference (DPD).

4. Results

4.1 Overall Performance

Figure 2 compares the forecasting performance of all methods for a 30-day horizon. The full harmonization-aware framework significantly outperforms all baselines. For need forecasting, the AUROC increases from 0.73 (site-local) to 0.82 (full) — an 8.2% absolute improvement (p<0.001, paired t-test). For utilization forecasting, SMAPE decreases from 18.2% to 11.9%, a relative reduction of 23.5%. The inclusion of exogenous signals and uncertainty preservation each contribute incremental gains. Table 1 summarizes the quantitative improvements with 95% confidence intervals.

4.2 Ablation Study

Figure 3 shows the contribution of each framework component. Removing harmonization reduces AUROC by 0.09; removing uncertainty preservation reduces AUROC by 0.04; omitting exogenous signals reduces AUROC by 0.03; and collapsing need–util separation reduces AUROC by 0.02 (and increases SMAPE by 1.3 percentage points). All differences are statistically significant (p<0.01). This confirms that each layer adds measurable value.

4.3 Calibration and Fairness

Calibration curves with bootstrap confidence intervals (Figure 4a) show that the full model is well-calibrated (Brier score 0.085, ECE 0.022), whereas site-local and harmonized-only models exhibit systematic over- and under-prediction (ECE 0.051 and 0.037 respectively). The overall ECE bar chart (Figure 4b) and subgroup analysis (Figure 4c) confirm that the proposed method maintains calibration across demographic groups.

Fairness analysis (Figure 5) reveals that the maximum AUROC disparity across demographic subgroups is 3.2% (uninsured vs. privately insured), well below the pre-specified 5% threshold, and substantially lower than the 8.1% disparity observed in site-local models. Additional fairness metrics—equal opportunity difference (EOD) and demographic parity difference (DPD)—also show minimal bias, with all absolute differences below 0.05 (Figure 5c,d).

4.4 Federated Learning Convergence

Figure 6a–b show that federated training achieves near-centralized performance after approximately 80 communication rounds, with final AUROC 0.81 (vs. centralized 0.82) and SMAPE 12.8% (vs. centralized 11.9%). Site-local models plateau at 0.73 AUROC and 18.2% SMAPE. The privacy cost (Figure 6c) remains within acceptable bounds, reaching \(\epsilon = 0.68\) after 100 rounds, satisfying the target privacy budget. The privacy-utility trade-off (Figure 6d) shows that moderate privacy loss (\(\epsilon \approx 1.0\)) results in negligible utility degradation (AUROC loss <0.02).

4.5 Horizon Stability

Performance degrades gracefully with longer horizons (Figure 7). For 6-month forecasts, need AUROC is 0.72 and utilization SMAPE is 19.8%, still substantially better than site-local (0.62 and 28.3%). Decay rates (annotated in the figure) show that the proposed method’s performance declines at roughly half the rate of site-local models for both need and utilization, demonstrating superior stability.

5. Discussion

5.1 Implications

This work establishes that uncertainty-preserving harmonization combined with explicit need–utilization separation yields superior forecasting accuracy for pharmaceutical demand. The framework’s modular design allows integration with existing healthcare IT infrastructure (Epic Clarity, OMOP, FHIR) and supports privacy-preserving federated learning, making it suitable for large-scale consortia.

The separation of need from utilization is particularly valuable for decision-makers: it enables understanding of where clinical demand exists versus where operational constraints may limit actual usage. For example, a forecast showing high need but low utilization could prompt investigation of access barriers or supply chain issues, whereas a high-utilization forecast with low need might indicate overprescribing or substitution patterns.

5.2 Limitations

The current study uses simulated data; prospective validation in real-world settings is required. The framework’s reliance on accurate harmonization and semantic extraction means that errors in upstream components can propagate. Additionally, the need–utilization decomposition depends on reliable access and supply data, which may not always be available. Finally, while we used a simple XGBoost+LSTM hybrid, more sophisticated models (e.g., transformers) could be explored.

5.3 Future Work

Planned extensions include: (1) integration of additional data modalities (imaging, genomics), (2) causal modeling to disentangle the effects of interventions, (3) dynamic adjustment of branch weights based on real-time feedback, (4) deployment in a clinical decision support system with controlled canary rollout, and (5) extending the framework to other healthcare forecasting tasks such as hospital bed occupancy or staff scheduling.

6. Conclusion

We have presented a novel forecasting framework that leverages uncertainty-preserving harmonization to estimate the future overlap between a population’s clinical state and external decision objects. By explicitly separating clinical need from realized utilization, the framework provides more interpretable and actionable forecasts. Experiments on simulated data demonstrate significant improvements over baselines, strong calibration, fairness, and federated learning feasibility. This work paves the way for next-generation population-scale pharmaceutical intelligence systems that are both scientifically rigorous and operationally deployable.

Acknowledgments

The author gratefully acknowledges VMC MAR COM Inc. DBA Axiomera, Knoxville, United States, for supporting this research.

Frequently asked questions

What is harmonization-aware predictive overlap forecasting?

It is a forecasting framework that estimates the expected intersection between a harmonized population state and an external decision object such as a drug, therapy class, or formulary change. From a single harmonized overlap score it derives three distinct forecasts — clinical need, realized utilization, and strategic opportunity — while propagating uncertainty from the harmonization layer throughout.

Why separate clinical need from realized utilization?

Need is the population-level clinical requirement for a therapy; utilization is what is actually consumed under access, formulary, and supply constraints. Separating them lets decision-makers see where demand exists versus where operational barriers limit usage. A forecast with high need but low utilization can signal access or supply-chain problems, while high utilization against low need can indicate overprescribing or substitution.

Does the method move patient data between institutions?

No. The framework uses federated learning with differential privacy. Each institution computes local gradient and Hessian histograms with per-example clipping, and the server aggregates with Gaussian noise using a moments accountant. On simulated data it reached an accumulated privacy budget of \(\epsilon = 0.68\) after 100 rounds while approaching centralized accuracy.

How accurate is the framework compared with site-local models?

On multi-site simulated data, need-forecast AUROC rose from 0.73 (site-local) to 0.82 — an 8.2% absolute improvement — and utilization SMAPE fell from 18.2% to 11.9%, a 23.5% relative reduction (p<0.001). The full model was also better calibrated (ECE 0.022) and fairer across demographic subgroups (3.2% maximum AUROC disparity versus 8.1% for site-local).

References

1. Nehzati, M. Semantic Classification Engine: Multi-Ontology, Contextual, and Customer-Adaptive Semantic Understanding. Internal technical specification, 2026.
2. Nehzati, M. Data Mapper Model: Multi-Standard, Bi-Directional Mapping Synthesis. Internal technical specification, 2026.
3. Nehzati, M. Data Harmonization Model: Continuous Multi-Site Harmonization with Configurable, Self-Healing Operations. Internal technical specification, 2026.
4. HeyDonto EMI. Refined Multi-Model Architecture Specification for Orchestration, Optimization, and Scale. Internal architecture specification, 2026.
5. Nehzati, M. Federated Learning for 30-Day Readmission Prediction: Phase 1 Implementation and Multi-Dimensional Validation Across 47 Healthcare Institutions. Internal technical paper, 2026.
6. Nehzati, M. Semantic Intelligence Framework for Clinical Data Warehouses: A Generalizable Approach to Graph-Based Concept Extraction, Statistically Validated Relationship Discovery, and Temporal Pattern Analysis. Internal technical paper, 2026.
7. Nehzati, R. Quantum-Superposition Clinical Intelligence Networks: Many-Worlds Data Harmonization for Population-Scale Healthcare Integration. Internal technical paper, 2026.
8. Kraljevic, Z. et al. Foresight – a generative pretrained transformer for modelling of patient timelines using electronic health records. 2024. Available via PubMed Central: https://pmc.ncbi.nlm.nih.gov/articles/PMC11220626/.
9. Kirchler, M. et al. Large language models improve transferability of electronic health record-based predictions across countries and coding systems. npj Digital Medicine, 2026.
10. Makarov, N. et al. Large language models forecast patient health trajectories enabling digital twins. 2025.
11. Shmatko, A. et al. Learning the natural history of human disease with generative transformers. 2025.
12. Chen, X. et al. Knowledge graph-enhanced deep learning for pharmaceutical demand forecasting. 2026.