Custom Integration Tutorial

This tutorial will guide you through creating custom integrations using the Healthcare Integration Platform's flexible transformation and connection capabilities.

Overview

Custom integrations allow you to connect systems that don't follow standard healthcare protocols or require specialized data processing. Our platform provides enterprise-grade capabilities including:

Healthcare Protocols: HL7 v2.x, FHIR R4/R5, X12 EDI, DICOM, CDA
Custom Protocols: REST APIs, GraphQL, SOAP, WebSocket, NATS messaging
Data Formats: JSON, XML, CSV, fixed-width, proprietary formats with AI-powered detection
Transformation Engine: JavaScript, JQ expressions, AI-powered mapping with healthcare ontologies
Custom Validation: Business rules, data quality checks, HIPAA compliance validation
Event Processing: Real-time WebSocket streams, NATS event bus, webhooks
Enterprise Features: Multi-tenancy, circuit breakers, distributed tracing, auto-scaling

Prerequisites

Healthcare Integration Platform access
Target system API documentation or specifications
Sample data files for testing
Authentication credentials for target systems

Step 1: Create Custom Connection

REST API Connection

curl -X POST "https://your-platform.com/api/v1/connections" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Custom EHR API",
       "connection_type": "rest_api",
       "configuration": {
         "base_url": "https://api.custom-ehr.com/v2",
         "timeout": 30,
         "verify_ssl": true,
         "headers": {
           "User-Agent": "Healthcare-Integration-Platform/1.0",
           "Accept": "application/json"
         },
         "auth": {
           "type": "api_key",
           "api_key_header": "X-API-Key",
           "api_key": "your-api-key-here"
         },
         "rate_limiting": {
           "requests_per_minute": 120,
           "burst_limit": 10
         }
       }
     }'

Database Connection

curl -X POST "https://your-platform.com/api/v1/connections" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Legacy Database",
       "connection_type": "database",
       "configuration": {
         "driver": "postgresql",
         "host": "legacy-db.hospital.com",
         "port": 5432,
         "database": "patient_records",
         "username": "integration_user",
         "password": "secure_password",
         "ssl_mode": "require",
         "connection_pool": {
           "min_connections": 2,
           "max_connections": 10,
           "timeout": 30
         }
       }
     }'

SFTP File Connection

curl -X POST "https://your-platform.com/api/v1/connections" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Lab Results SFTP",
       "connection_type": "sftp",
       "configuration": {
         "host": "sftp.lab-system.com",
         "port": 22,
         "username": "integration",
         "private_key": "-----BEGIN PRIVATE KEY-----\\n...",
         "remote_directory": "/incoming/lab_results",
         "file_pattern": "*.csv",
         "archive_directory": "/processed",
         "polling_interval": 300
       }
     }'

WebSocket Real-time Integration

# The platform includes a dedicated WebSocket service
# Connect to real-time data streams
ws_url="wss://your-platform.com/api/v1/websocket/ws"

# WebSocket configuration for real-time monitoring
curl -X POST "https://your-platform.com/api/v1/connections" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Real-time Patient Monitoring",
       "connection_type": "websocket",
       "configuration": {
         "endpoint": "/ws/patient-vitals",
         "channels": ["patient_alerts", "vital_signs"],
         "rate_limiting": {
           "messages_per_second": 100,
           "burst_limit": 20
         },
         "heartbeat": {
           "enabled": true,
           "interval": 30
         }
       }
     }'

Step 2: Test Custom Connection

Test REST API Connection

curl -X POST "https://your-platform.com/api/v1/connections/{connection_id}/test" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "test_endpoint": "/health",
       "expected_response_code": 200
     }'

Expected Response:

{
  "status": "success",
  "response_time_ms": 245,
  "endpoint_status": "healthy",
  "api_version": "2.1.0",
  "rate_limit_remaining": 115,
  "authentication": "valid",
  "ssl_certificate": {
    "valid": true,
    "expires": "2025-12-31T23:59:59Z"
  }
}

Step 3: Understanding Custom Data Formats

JSON API Response

{
  "patient_data": {
    "id": "PT12345",
    "demographics": {
      "first_name": "John",
      "last_name": "Doe",
      "dob": "1985-03-15",
      "gender": "M",
      "ssn": "123-45-6789"
    },
    "contact": {
      "primary_phone": "555-123-4567",
      "email": "john.doe@email.com",
      "address": {
        "street": "123 Main St",
        "city": "Anytown",
        "state": "ST",
        "zip": "12345"
      }
    },
    "insurance": {
      "primary": {
        "carrier": "BlueCross",
        "policy_number": "BC123456789",
        "group": "CORP001"
      }
    },
    "visits": [
      {
        "visit_id": "V789",
        "date": "2025-01-15",
        "type": "office_visit",
        "provider": "Dr. Smith",
        "diagnosis_codes": ["Z00.00", "I10"],
        "charges": [
          {
            "code": "99213",
            "amount": 150.00,
            "units": 1
          }
        ]
      }
    ]
  }
}

CSV Lab Results Format

PatientID,TestDate,TestCode,TestName,Result,Units,ReferenceRange,Status
PT12345,2025-01-15,CBC,Complete Blood Count,5.2,10^3/uL,4.5-11.0,Normal
PT12345,2025-01-15,GLU,Glucose,95,mg/dL,70-100,Normal
PT12346,2025-01-15,CHOL,Total Cholesterol,220,mg/dL,<200,High

Fixed-Width Format

PT12345DOE       JOHN     M19850315555123456712345
PT12346SMITH     JANE     F19920722555987654154321

Position mapping:

Positions 1-7: Patient ID
Positions 8-17: Last Name (padded)
Positions 18-27: First Name (padded)
Position 28: Gender
Positions 29-36: Date of Birth (YYYYMMDD)
Positions 37-47: Phone Number
Positions 48-52: ZIP Code

Step 4: Create Custom Integration with Transformations

JSON to FHIR Patient Transformation

curl -X POST "https://your-platform.com/api/v1/integrations" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Custom EHR to FHIR Patient Sync",
       "description": "Transform custom JSON patient data to FHIR format",
       "source_connection_id": "custom-ehr-api-conn",
       "target_connection_id": "fhir-server-conn",
       "transformation_rules": {
         "mappings": {
           "resourceType": "\"Patient\"",
           "id": "patient_data.id",
           "identifier[0].value": "patient_data.id",
           "identifier[0].system": "\"http://custom-ehr.com/patient-ids\"",
           "name[0].family": "patient_data.demographics.last_name",
           "name[0].given[0]": "patient_data.demographics.first_name",
           "gender": "transform_gender(patient_data.demographics.gender)",
           "birthDate": "format_date(patient_data.demographics.dob)",
           "telecom[0].system": "\"phone\"",
           "telecom[0].value": "patient_data.contact.primary_phone",
           "telecom[1].system": "\"email\"",
           "telecom[1].value": "patient_data.contact.email",
           "address[0].line[0]": "patient_data.contact.address.street",
           "address[0].city": "patient_data.contact.address.city",
           "address[0].state": "patient_data.contact.address.state",
           "address[0].postalCode": "patient_data.contact.address.zip"
         },
         "custom_functions": {
           "transform_gender": {
             "type": "javascript",
             "code": "function transform_gender(gender) { return gender === \"M\" ? \"male\" : gender === \"F\" ? \"female\" : \"unknown\"; }"
           },
           "format_date": {
             "type": "javascript",
             "code": "function format_date(date) { return date.replace(/\\//g, \"-\"); }"
           }
         },
         "transformations": {
           "exclude_empty_fields": true,
           "validate_output": true
         }
       },
       "source_configuration": {
         "endpoint": "/api/patients",
         "method": "GET",
         "pagination": {
           "type": "offset",
           "limit_param": "limit",
           "offset_param": "offset",
           "page_size": 50
         }
       },
       "schedule": "0 */4 * * *"
     }'

CSV Lab Results to FHIR Observation

curl -X POST "https://your-platform.com/api/v1/integrations" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Lab CSV to FHIR Observations",
       "description": "Convert CSV lab results to FHIR Observation resources",
       "source_connection_id": "lab-sftp-conn",
       "target_connection_id": "fhir-server-conn",
       "transformation_rules": {
         "csv_configuration": {
           "delimiter": ",",
           "header_row": true,
           "skip_empty_rows": true,
           "column_mapping": {
             "PatientID": "patient_id",
             "TestDate": "test_date",
             "TestCode": "test_code",
             "TestName": "test_name",
             "Result": "result_value",
             "Units": "units",
             "ReferenceRange": "reference_range",
             "Status": "status"
           }
         },
         "mappings": {
           "resourceType": "\"Observation\"",
           "id": "generate_uuid()",
           "status": "map_status(status)",
           "category[0].coding[0].system": "\"http://terminology.hl7.org/CodeSystem/observation-category\"",
           "category[0].coding[0].code": "\"laboratory\"",
           "code.coding[0].code": "test_code",
           "code.coding[0].display": "test_name",
           "code.coding[0].system": "\"http://loinc.org\"",
           "subject.reference": "concat(\"Patient/\", patient_id)",
           "effectiveDateTime": "parse_date(test_date)",
           "valueQuantity.value": "parse_number(result_value)",
           "valueQuantity.unit": "units",
           "interpretation[0].coding[0].code": "interpret_result(status)"
         },
         "custom_functions": {
           "map_status": {
             "type": "javascript",
             "code": "function map_status(status) { return status.toLowerCase() === \"preliminary\" ? \"preliminary\" : \"final\"; }"
           },
           "interpret_result": {
             "type": "javascript",
             "code": "function interpret_result(status) { return status === \"High\" ? \"H\" : status === \"Low\" ? \"L\" : \"N\"; }"
           },
           "parse_date": {
             "type": "javascript",
             "code": "function parse_date(date) { return new Date(date).toISOString(); }"
           }
         }
       },
       "data_filters": {
         "exclude_rows_where": {
           "result_value": "null",
           "patient_id": "empty"
         },
         "file_pattern": "lab_results_*.csv"
       }
     }'

Step 5: Advanced Custom Transformations

JavaScript-Based Complex Transformation

curl -X POST "https://your-platform.com/api/v1/integrations" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Complex Data Transformation",
       "description": "Advanced transformation with custom business logic",
       "source_connection_id": "complex-source-conn",
       "target_connection_id": "target-system-conn",
       "transformation_rules": {
         "custom_transformation": {
           "type": "javascript",
           "code": "function transform(input) {\\n  const output = {\\n    patient: {\\n      id: input.patient_data.id,\\n      name: {\\n        full: input.patient_data.demographics.first_name + \" \" + input.patient_data.demographics.last_name,\\n        parts: {\\n          first: input.patient_data.demographics.first_name,\\n          last: input.patient_data.demographics.last_name\\n        }\\n      },\\n      age: calculateAge(input.patient_data.demographics.dob),\\n      risk_score: calculateRiskScore(input.patient_data.visits)\\n    },\\n    summary: {\\n      total_visits: input.patient_data.visits.length,\\n      total_charges: input.patient_data.visits.reduce((sum, visit) => {\\n        return sum + visit.charges.reduce((chargeSum, charge) => chargeSum + charge.amount, 0);\\n      }, 0),\\n      primary_diagnoses: extractPrimaryDiagnoses(input.patient_data.visits)\\n    }\\n  };\\n  return output;\\n}\\n\\nfunction calculateAge(dob) {\\n  const today = new Date();\\n  const birthDate = new Date(dob);\\n  let age = today.getFullYear() - birthDate.getFullYear();\\n  const monthDiff = today.getMonth() - birthDate.getMonth();\\n  if (monthDiff < 0 || (monthDiff === 0 && today.getDate() < birthDate.getDate())) {\\n    age--;\\n  }\\n  return age;\\n}\\n\\nfunction calculateRiskScore(visits) {\\n  // Custom risk calculation based on diagnosis codes\\n  let score = 0;\\n  visits.forEach(visit => {\\n    visit.diagnosis_codes.forEach(code => {\\n      if (code.startsWith(\"I\")) score += 3; // Cardiovascular\\n      if (code.startsWith(\"E\")) score += 2; // Endocrine\\n      if (code.startsWith(\"C\")) score += 5; // Cancer\\n    });\\n  });\\n  return Math.min(score, 100);\\n}\\n\\nfunction extractPrimaryDiagnoses(visits) {\\n  const diagnoses = new Set();\\n  visits.forEach(visit => {\\n    if (visit.diagnosis_codes.length > 0) {\\n      diagnoses.add(visit.diagnosis_codes[0]);\\n    }\\n  });\\n  return Array.from(diagnoses);\\n}"
         }
       }
     }'

AI-Powered Data Mapping

# The platform includes advanced AI transformation capabilities
curl -X POST "https://your-platform.com/api/v1/transformation/ai-transform" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "input_data": "your healthcare data",
       "source_format": "auto-detect",
       "target_format": "fhir",
       "ai_provider": "anthropic",
       "model": "claude-3-sonnet",
       "transformation_options": {
         "ontology_mapping": {
           "enabled": true,
           "systems": ["ICD-11", "SNOMED-CT", "LOINC", "CPT", "RxNorm"]
         },
         "field_mapping": {
           "confidence_threshold": 0.8,
           "suggest_mappings": true
         },
         "format_detection": {
           "auto_detect": true,
           "supported_formats": ["HL7", "FHIR", "X12", "CDA", "DICOM"]
         }
       }
     }'

# Field mapping suggestions based on ML analysis
curl -X POST "https://your-platform.com/api/v1/transformation/suggest-mappings" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "source_schema": {...},
       "target_schema": {...},
       "healthcare_domain": "patient_demographics"
     }'

Step 6: Custom Validation Rules

Business Rule Validation

curl -X POST "https://your-platform.com/api/v1/integrations/{integration_id}/validation" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "validation_rules": [
         {
           "name": "valid_patient_age",
           "type": "javascript",
           "code": "function validate(data) {\\n  const age = data.patient.age;\\n  return age >= 0 && age <= 150;\\n}",
           "error_message": "Patient age must be between 0 and 150 years",
           "severity": "error"
         },
         {
           "name": "required_insurance",
           "type": "javascript",
           "code": "function validate(data) {\\n  return data.patient.insurance && data.patient.insurance.primary;\\n}",
           "error_message": "Primary insurance information is required",
           "severity": "warning"
         },
         {
           "name": "valid_phone_format",
           "type": "regex",
           "pattern": "^\\\\d{3}-\\\\d{3}-\\\\d{4}$",
           "field": "patient.contact.primary_phone",
           "error_message": "Phone number must be in format XXX-XXX-XXXX",
           "severity": "error"
         }
       ]
     }'

Data Quality Checks

curl -X POST "https://your-platform.com/api/v1/integrations/{integration_id}/quality-rules" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "quality_checks": {
         "completeness": {
           "required_fields": ["patient_id", "name", "dob"],
           "threshold": 95
         },
         "consistency": {
           "cross_field_checks": [
             {
               "name": "gender_consistency",
               "fields": ["gender", "title"],
               "rule": "if gender is M, title should be Mr. or Dr."
             }
           ]
         },
         "accuracy": {
           "format_checks": {
             "email": "email_format",
             "phone": "phone_format",
             "ssn": "ssn_format"
           }
         },
         "timeliness": {
           "max_age_days": 30,
           "date_field": "last_updated"
         }
       },
       "actions": {
         "on_failure": "quarantine",
         "notification": true,
         "retry_policy": {
           "max_retries": 3,
           "delay_seconds": 300
         }
       }
     }'

Step 7: Real-time Custom Integrations

Webhook Integration

curl -X POST "https://your-platform.com/api/v1/integrations" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Real-time Webhook Integration",
       "description": "Process incoming webhook events in real-time",
       "source_connection_id": "webhook-source-conn",
       "target_connection_id": "processing-target-conn",
       "trigger": {
         "type": "webhook",
         "endpoint": "/webhooks/patient-updates",
         "method": "POST",
         "authentication": {
           "type": "signature",
           "secret": "webhook-secret-key",
           "header": "X-Signature-256"
         }
       },
       "transformation_rules": {
         "event_routing": {
           "patient.created": "create_patient_flow",
           "patient.updated": "update_patient_flow",
           "patient.deleted": "deactivate_patient_flow"
         },
         "mappings": {
           "event_type": "event.type",
           "patient_id": "event.data.patient.id",
           "timestamp": "event.timestamp",
           "changes": "event.data.changes"
         }
       },
       "processing": {
         "async": true,
         "timeout": 30,
         "retry_policy": {
           "max_retries": 3,
           "exponential_backoff": true
         }
       }
     }'

NATS Message Queue Integration

# The platform uses NATS for event-driven architecture
# Configure NATS messaging for real-time integration
curl -X POST "https://your-platform.com/api/v1/integrations" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "NATS Event Processing",
       "description": "Process healthcare events via NATS messaging",
       "source_connection_id": "nats-source-conn",
       "target_connection_id": "processing-target-conn",
       "nats_configuration": {
         "subject": "healthcare.events.*",
         "queue_group": "integration-workers",
         "durable_name": "healthcare-processor",
         "ack_policy": "explicit",
         "max_deliver": 3
       },
       "event_routing": {
         "patient.created": "process_new_patient",
         "patient.updated": "update_patient_data",
         "lab.result": "process_lab_result",
         "appointment.scheduled": "sync_appointment"
       },
       "processing": {
         "async": true,
         "correlation_headers": true,
         "service_communication": {
           "discovery": true,
           "health_checks": true
         }
       }
     }'

Step 8: Monitoring Custom Integrations

Custom Metrics and Alerts

curl -X POST "https://your-platform.com/api/v1/integrations/{integration_id}/monitoring" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "custom_metrics": [
         {
           "name": "data_quality_score",
           "type": "gauge",
           "calculation": "javascript",
           "code": "function calculate(processedRecords, errors) {\\n  return ((processedRecords - errors) / processedRecords) * 100;\\n}",
           "alert_threshold": 95
         },
         {
           "name": "processing_lag",
           "type": "histogram",
           "buckets": [100, 500, 1000, 5000, 10000],
           "alert_threshold": 5000
         }
       ],
       "alerts": [
         {
           "name": "high_error_rate",
           "condition": "error_rate > 5%",
           "severity": "critical",
           "notification": {
             "email": ["admin@hospital.com"],
             "slack": "#alerts"
           }
         },
         {
           "name": "data_quality_degradation",
           "condition": "data_quality_score < 90",
           "severity": "warning",
           "notification": {
             "email": ["data-team@hospital.com"]
           }
         }
       ]
     }'

Step 9: Error Handling and Recovery

Custom Error Handling

curl -X POST "https://your-platform.com/api/v1/integrations/{integration_id}/error-handling" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "error_strategies": {
         "data_validation_error": {
           "action": "quarantine",
           "quarantine_queue": "data_quality_review",
           "notification": true,
           "retry": false
         },
         "connection_timeout": {
           "action": "retry",
           "max_retries": 3,
           "backoff_strategy": "exponential",
           "base_delay": 30,
           "max_delay": 300
         },
         "authentication_error": {
           "action": "alert_and_pause",
           "notification": {
             "priority": "high",
             "recipients": ["ops-team@hospital.com"]
           }
         },
         "unknown_error": {
           "action": "log_and_continue",
           "fallback_processing": true
         }
       },
       "recovery_procedures": {
         "automatic_retry": true,
         "manual_intervention_required": ["authentication_error"],
         "health_check_interval": 300
       }
     }'

Best Practices

Development and Testing

Start Small: Begin with simple transformations and gradually add complexity
Test Thoroughly: Use sample data to test all transformation scenarios
Version Control: Maintain versions of transformation rules and configurations
Documentation: Document custom functions and business logic clearly

Performance Optimization

Batch Processing: Process data in optimal batch sizes for your systems
Caching: Cache frequently accessed reference data and lookups
Parallel Processing: Use parallel execution for independent transformations
Memory Management: Monitor memory usage for large data transformations

Security and Compliance

Data Encryption: Encrypt sensitive data in transit and at rest
Access Control: Implement proper authentication and authorization
Audit Logging: Log all data access and transformation activities
Data Minimization: Only process and store necessary data

Maintainability

Modular Design: Create reusable transformation components
Error Handling: Implement comprehensive error handling and recovery
Monitoring: Set up detailed monitoring and alerting
Documentation: Maintain up-to-date integration documentation

Advanced Features

Pipeline Workflows with Step Runners

# The platform includes sophisticated pipeline processing
# Create multi-step data processing workflows
curl -X POST "https://your-platform.com/api/v1/pipelines" \
     -H "Authorization: Bearer YOUR_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Healthcare Data Processing Pipeline",
       "description": "Multi-step patient data processing with AI enhancement",
       "steps": [
         {
           "name": "data_extraction",
           "type": "extraction",
           "runner": "extraction_runner",
           "config": {
             "source_type": "database",
             "query": "SELECT * FROM patients WHERE updated_at > ?"
           }
         },
         {
           "name": "ai_validation",
           "type": "prompter",
           "runner": "prompter_runner",
           "config": {
             "ai_provider": "anthropic",
             "prompt": "Validate and enhance this patient data for FHIR compliance",
             "model": "claude-3-sonnet"
           }
         },
         {
           "name": "transformation",
           "type": "transform",
           "runner": "transform_runner",
           "config": {
             "mappings": {...},
             "custom_functions": {...}
           }
         },
         {
           "name": "quality_validation",
           "type": "validation",
           "runner": "validation_runner",
           "config": {
             "rules": "healthcare_quality_rules",
             "severity_threshold": "warning"
           }
         },
         {
           "name": "fhir_output",
           "type": "output",
           "runner": "output_runner",
           "config": {
             "target_format": "fhir_r4",
             "endpoint": "https://your-platform.com/api/v1/fhir"
           }
         }
       ],
       "execution": {
         "parallel_processing": true,
         "context_propagation": true,
         "error_handling": "continue_on_warning"
       }
     }'

Next Steps

← FHIR Integration Tutorial ← HL7 Integration Tutorial ← X12 Integration Tutorial