- Sending API (BHT-EMR-API) - Health facility EMR system
- Receiving API (MOH Reporting API) - Central data repository & analytics
- MOH Reports Gem
- Sending API (BHT-EMR-API) - Health facility EMR system
- Receiving API (MOH Reporting API) - Central data repository & analytics
- Ruby 3.2.0 +
- MySQL 5.6 +
- PostgreSQL 14 +
- Unix Environment / Docker Environment
The EMR Streaming Service is a comprehensive data synchronization and analytics system designed for the Malawi Health Information System (HIS). It enables real-time streaming of patient visit data from health facility EMR systems to a centralized data repository (CDR), which then powers analytics, reporting, and business intelligence across the health system.
The system consists of two interconnected components working in a coordinated pipeline:
-
Sending API (BHT-EMR-API) - Source System
- Health facility EMR system that captures patient care data (visits, encounters, treatments)
- Streams patient visit data to the central data repository
- Uses SolidQueue (database-backed job queue) for reliable asynchronous data transmission
- Supports the ART (Antiretroviral Therapy) module with planned expansion to other programs
-
Receiving API (MOH Reporting API) - Central Data Repository
- Centralized system that receives, processes, and stores streamed data
- Generates analytics and HIV program reports from aggregated data
- Supports read-only operations (reporting and analytics, not data entry)
- Uses dual-database architecture for operational and analytical data separation
┌─────────────────────────────────────────────────────────────────────┐
│ SENDING API (BHT-EMR-API) │
│ Health Facility EMR System │
└──────────────────────────────────┬──────────────────────────────────┘
│
┌──────────────▼──────────────┐
│ SolidQueue Job Queue │
│ (Database-backed Queue) │
└──────────────┬──────────────┘
│
┌──────────────▼──────────────┐
│ StreamingService │
│ - Builds data payloads │
│ - Compresses with gzip │
│ - Transmits via REST API │
└──────────────┬──────────────┘
│
┌───────────────▼────────────────┐
│ REST API POST Request │
│ /api/v1/stream │
│ (Complete Patient Visit Data)│
└───────────────┬────────────────┘
│
┌──────────────────────────────────▼──────────────────────────────────┐
│ RECEIVING API (MOH Reporting API) │
│ Central Data Repository & Analytics Engine │
└──────────────────────────────────┬──────────────────────────────────┘
│
┌──────────────▼──────────────┐
│ StreamingEngine │
│ - Validates incoming data │
│ - Queues cache & IDS jobs │
│ - Routes to program service │
└──────────────┬──────────────┘
│
┌──────────────▼──────────────┐
│ Program-Specific Service │
│ (e.g., ArtStreamingService)│
│ - Extracts demographics │
│ - Stores encounters │
│ - Saves clinical data │
└──────────────┬──────────────┘
│
┌─────────────────┴─────────────────┐
│ │
┌────────▼────────┐ ┌────────────▼─────┐
│ MySQL Database │ │ PostgreSQL (IDS) │
│ (Operational) │ │ (Analytics) │
└────────────────┘ └──────────────────┘
│
┌──────────────▼──────────────┐
│ Reporting Engine │
│ - Generates reports │
│ - Supports analytics │
│ - Provides dashboards │
└──────────────────────────────┘
- Data Capture: Health facilities use the Sending API (BHT-EMR-API) to record patient visits and encounters in the ART module
- Job Enqueueing: When visits are completed, the system automatically enqueues streaming jobs using SolidQueue
- Reliable Transmission: SolidQueue ensures jobs are processed reliably with automatic retry and error tracking
- Data Streaming: StreamingService compiles patient data and sends it to the Receiving API via REST API
- Data Reception: The Receiving API validates incoming payloads and routes them to program-specific services
- Data Processing: Program services extract and store data in both operational (MySQL) and analytical (PostgreSQL) databases
- Reporting: The analytical database powers real-time reports, dashboards, and business intelligence
- Reliable Delivery: Database-backed queue persists jobs across service restarts
- Automatic Scheduling: Daily jobs identify and stream incomplete and missed visits
- Asynchronous Processing: Jobs processed in background without blocking user operations
- Error Tracking: Failed jobs logged and accessible for investigation and retry
- Real-time Monitoring: Web-based dashboards for job status and performance tracking
- Multi-Database Architecture: Separates operational data from analytical data for performance optimization
- Program-Specific Processing: Currently supports ART module with architecture ready for expansion