Semantic Intelligence Framework for Epic Clarity Data Warehouses

1. Purpose and goals

Build and evaluate a semantic intelligence layer on Epic Clarity to unify structured and unstructured EHR data, discover clinically meaningful associations and longitudinal patterns, and deliver near–real-time clinical decision support (CDS) within Epic workflows (Abstract; Fig. 1 p. 5).

Specific goals: improve diagnostic coding accuracy and comorbidity capture; extract and normalize concepts from notes to standards; validate statistically robust associations; surface temporal care patterns; and integrate recommendations into Epic via Interconnect / CDS Hooks (Sec. 2; Sec. 3).

2. What was achieved (outcomes)

• Concept extraction: 834,267 notes processed; 8,347 unique concepts recognized and mapped to SNOMED CT / ICD-10-CM / RxNorm / LOINC (Fig. 5, pp. 28–30).
• Association discovery: 34,847,131 candidate pairs → 2,847 (co-occurrence ≥ 25) → 247 statistically significant after BH-FDR at \(q = 0.05\) → 156 clinically validated (Fig. 6, pp. 30–31).
• Temporal patterns: 389 candidates at support \(\sigma = 0.05\) → 89 significant (\(p < 0.001\)); next-event prediction accuracy up to 0.784 (30-day horizon) with bi-LSTM + attention (Fig. 7, p. 32).
• CDS in Epic: per-encounter recommendation generation 2.3 ± 0.7 s; end-to-end analytics under load 28.9 ± 2.8 s (Abstract; Figs. 4 & 8).
• Clinical impact (multisite): diagnostic accuracy +8.3 pp; comorbidity detection +12.3 pp; CDS relevance +15.4 pp; CDS acceptance rate 67.3% (Fig. 8, pp. 33–34).
• Scalability & reliability: validated to 50 concurrent users (JMeter 5.4.1), ∼19% latency rise from 10 → 50 users; max CPU 78%, memory 42.1 GB; availability 99.2%, MTTR 4.2 ± 1.8 min; system “stable and consistent in production” (Fig. 9, p. 35).

A semantic layer can operate read-only, refresh incrementally, and still meet near–real-time CDS latency in Epic environments.

3. Data inventory and volume

3.1 Scope and cohort

Three healthcare systems; 12-month period; 127,834 patients; 492,542 encounters (Sec. 3.1, p. 25).

3.2 Sources and types

Epic Clarity (SQL) for batch analytics and Epic Interconnect (FHIR R4) for near–real-time clinical context / CDS (Sec. 2.2, pp. 7–8). Structured relational data (encounters, diagnoses, medications, labs) and unstructured clinical notes with NLP; concepts mapped to SNOMED CT, ICD-10-CM, RxNorm, LOINC (Sec. 2.3, pp. 8–9). Knowledge / graph model \(S = (E, R, T, C)\) with weighted edges (temporal, hierarchical, associative, contraindicated) (Sec. 2.1; Fig. 1 p. 5).

3.3 Table-level volumes and quality

Data quality metrics. Completeness \(C(D) = N_{\text{complete}}/N_{\text{total}}\) and a Jaccard-style cross-table consistency score; both reported per table (Eqs. 1–2; Table 1).

Privacy. HIPAA Safe Harbor de-identification plus date shifting, geographic generalization to state level, and k-anonymity ≥ 5; no PHI used for ML training (Sec. 3.4, p. 27).

4. Data transition and processing flow

1. Ingestion: Read-only connection to Epic Clarity (SQL) and Interconnect (FHIR R4). Incremental loading via timestamps; routine refresh every 4–6 hours for analytics; CDS updated near–real-time. Processing scheduled in maintenance windows (typically 02:00–06:00) to avoid clinical impact (Sec. 2.1–2.2, pp. 6–7).
2. Quality checks: automated completeness and consistency validation across integrated datasets (Eqs. 1–2; Table 1).
3. NLP concept extraction: normalization → tokenization → rule-based + ClinicalBERT NER → probabilistic confidence scoring; keep entities with confidence \(> \theta\); map to standard vocabularies (Algorithm 1; Eqs. 3–4; Sec. 2.3).
4. Association discovery: co-occurrence threshold ≥ 25 (power analysis); Fisher’s exact test; Cramér’s \(V > 0.1\) for practical significance; Benjamini–Hochberg FDR at \(q = 0.05\); blinded expert validation with consensus and \(\kappa \ge 0.60\) (Algorithms 2–3; Secs. 2.4–2.5).
5. Temporal analysis: sequence alignment and imputation; sequential pattern mining with support \(\sigma = 0.05\); z-test vs. null (report \(p < 0.001\)); DTW similarity and bi-LSTM + attention modeling (Algorithm 4; Sec. 2.6).
6. CDS delivery: real-time recommendations via HL7 CDS Hooks v1.1 through Epic Interconnect (FHIR R4), prioritized by evidence and urgency (Sec. 2.9).
7. Monitoring: precision / recall / F1 by clinical domain, SPC-style drift detection (control limits \(\mu \pm 3\sigma/\sqrt{n}\)), and a composite reliability metric; QA tracked over 12 months (Sec. 2.10; Fig. 9).

5. Technologies and architecture

• Integration / standards: Epic Clarity (SQL), Interconnect (FHIR R4), HL7 CDS Hooks v1.1 (Secs. 2.1–2.2, 2.9).
• Analytics stack: ClinicalBERT + rules for NER; Fisher’s exact, Benjamini–Hochberg, Cramér’s \(V\); DTW; bi-LSTM + attention; bootstrap CIs; SPC monitoring (Secs. 2.3–2.6, 2.8, 2.10).
• Platform: containerized microservices (Docker 20.10.8) orchestrated by Kubernetes with horizontal pod autoscaling, intelligent caching, composite indexing, and parallelization (Secs. 2.1, 2.7; Fig. 3).
• Security / compliance: TLS 1.3 in transit, AES-256 at rest; SAML 2.0 / OAuth 2.0 SSO; RBAC; audit logging; Business Associate Agreement-aligned controls; read-only Clarity integration; validated on Epic 2018 and 2020 (Sec. 2.1).
• Infra tested: Dell PowerEdge R440; Intel Xeon Silver 4210R; 64 GB RAM per server; load testing with Apache JMeter 5.4.1 (Sec. 3.2, p. 26; Fig. 9).

6. Key insights and findings

• A semantic layer can operate read-only, refresh incrementally, and still meet near–real-time CDS latency in Epic environments (Secs. 2.1–2.2; Figs. 4 & 8).
• Combining NLP, FDR-controlled statistics, and blinded expert review reduces false positives and yields clinically credible signals at health-system scale (Fig. 6).
• Temporal intelligence adds interpretable value for pathway optimization and next-event prediction beyond single-task ML (Fig. 7).
• Production-grade SRE practices (autoscaling, caching, SPC QA) delivered high availability and stable performance over a year (Fig. 9).

7. Evidence of production / near-production conditions

• Maintenance-window scheduling (02:00–06:00), incremental loads, read-only Clarity access (Sec. 2.1–2.2).
• Enterprise security (TLS 1.3 / AES-256, SSO / RBAC) and HIPAA-aligned de-identification (k ≥ 5), with audit logs (Sec. 2.1; Sec. 3.4).
• Measured concurrency / latency under load, availability 99.2%, and explicit statement of production stability (Fig. 9, p. 35).

8. Notes and limitations

• Storage volume in GB/TB is not reported; scale is conveyed via row counts, throughput, and latency (Table 1; Figs. 4–9).
• Reported associations and temporal patterns are observational / non-causal; expert validation enforces clinical plausibility (Figs. 6–7).

9. Appendix: at-a-glance metrics

• Sites / timeframe: 3 sites; 12 months (Sec. 3.1).
• Cohort / events: 127,834 patients; 492,542 encounters (Sec. 3.1).
• Key tables (rows): PAT_ENC 3.85M; HNO_NOTE_TEXT 0.83M; CLARITY_TDL 0.85M; ORDER_MED 1.92M; LAB_RESULTS 12.35M (Table 1).
• NLP: 834k notes; 8,347 unique concepts (Fig. 5).
• Associations: 34.85M → 2,847 → 247 → 156 validated (Fig. 6).
• Temporal: 389 candidates → 89 significant; bi-LSTM accuracy 0.784 (30d) / 0.716 (90d) (Fig. 7).
• CDS: 2.3 ± 0.7 s per encounter; 28.9 ± 2.8 s end-to-end; 67.3% acceptance (Fig. 8).
• Scale / reliability: 50 concurrent users; +18.9% RT at 5× load; max CPU 78%, mem 42.1 GB; availability 99.2% (Fig. 9).

Frequently asked questions

What is the Semantic Intelligence Framework for Epic Clarity?

It is a read-only semantic layer built on the Epic Clarity SQL warehouse and Epic Interconnect (FHIR R4). It unifies structured relational data with unstructured clinical notes, maps concepts to SNOMED CT, ICD-10-CM, RxNorm, and LOINC, discovers statistically validated associations and longitudinal patterns, and returns clinical decision support inside Epic workflows via HL7 CDS Hooks v1.1.

Does the framework move patient data out of the hospital?

No. The integration with Epic Clarity is read-only, and no protected health information was used for model training. The pilot applied HIPAA Safe Harbor de-identification with date shifting, geographic generalization to state level, and k-anonymity of at least 5, with TLS 1.3 in transit and AES-256 at rest.

How accurate was the temporal next-event prediction?

A bi-LSTM with attention reached next-event prediction accuracy up to 0.784 at a 30-day horizon and 0.716 at a 90-day horizon. These followed sequential pattern mining that surfaced 89 statistically significant temporal patterns (p < 0.001) from 389 candidates.

How were the statistical associations validated?

Candidate pairs required a co-occurrence count of at least 25, Fisher's exact test, and Cramér's V above 0.1 for practical significance, with Benjamini–Hochberg FDR control at q = 0.05. Surviving associations then went through blinded expert review with consensus and inter-rater agreement of κ ≥ 0.60, leaving 156 clinically validated associations.

Source

1. HeyDonto Science Department (2025). Data Volume & Transition Report — Semantic Intelligence Framework for Epic Clarity Data Warehouses. November 5, 2025.
2. Source paper: Semantic Intelligence Framework for Epic Clarity Data Warehouses: Graph-Based Concept Extraction, Statistical Relationship Discovery, and Temporal Pattern Analysis. HeyDonto Science Department.