Lab Download Integration
Preface
These are instructions on how to set up electronic data transfer (EDT) for downloading your labs in HL7 format from each individual laboratory, and uploading them to your OSCAR EMR. This is a generic method of getting any electronic lab data to your EMR without the need for OntarioMD certification.
Your vendor may have alternate methods of downloading lab data such as Excelleris (check with Open OSP Service Cooperative) for their solution.
Document Version History
- v1.0 – Initial public release to oscarmanual.org – Jan 12, 2016
- v1.1 – Initial OSCAR 15 release – Mar 5, 2016
- v2.0 – Ported over to oscargalaxy.org – 11-Oct-2022 (KC Lai)
- v2.1 – Added instructions on setting up lab download from lab vendors (KC Lai)
- v2.2 – Updated and documented Linux and Excelleris approach – Aug 2, 2024
- © 2016-2024 by Peter Hutten-Czapski MD under the Creative Commons Attribution-Share Alike 3.0 Unported License
Prerequisites
- You are familiar with the Command line, dealing with source code and Java build tools.
- You have a dedicated server (or VM) that can stay on 24/7.
- You have a static IP for your clinic system that runs the lab downloader. Some labs restrict lab download to known static IP’s of authorized clinics/physicians. You need to contact your internet provider and pay a little extra for a static IP. If you static IP changes (upgrade services or new provider), remember to update the labs of your new static IP.
Overview
As a physician, you may order labs and patients may go to one of several laboratory vendors. The general process is this: Each lab will have a SFTP or API service that you can download HL7 lab files (electronically coded lab data) to a folder on your dedicated lab download computer or virtual machine. An HL7 transport utility (Mule) will upload the HL7 lab files from the folders to your EMR via a secure channel with private-public keypair in OSCAR for each lab vendor. The EMR will then process the HL7 lab data and file to the corresponding patients, and will show up in your InBox for review.
These instructions are mainly for a Windows and Linux systems.
Get Electronic Data Transfer (EDT) accounts
Any lab report in digital HL7 formats can be downloaded but to have them properly display depends on the presence of a lab “channel” in OSCAR that can handle the labs Hl7 “dialect”. Supported dialects can be viewed from Inbox > Upload HL7 labs. At the time of this writing we have support for the following.
- ALPHA aka AlphaLabs
- CML
- EPSILON/MHL
- ExcellerisON aka Excelleris (Ontario) XML wrapped HL7 v2.3.1
- PATHL7
- EXCELLERIS aka Excelleris (Ontario) XML wrapped HL7 v2.3.0
- GDML aka Dynacare
- HHSEMR
- HRM XML
- ICL
- IHAPOI
- MDS/Lifelabs
- MEDVUE
- OLIS HL7
- OSCAR_TO_OSCAR_HL7_V2
- PFHT
- SIOUX
- TDIS
- Spire
- OUL_R21
- ORU_R01
- BioTest
- CLS aka Calgary Lab Services
- CDL
- TRUENORTH
- MEDITECH
- Other
If your lab is not on the list you will need to speak to your laboratory provider’s technical people as to the version of HL7 they are using and to provide any test files to check against existing channels to find one that you can use or use a copy after light modification.
For the following channels their labs work out of the box and you just need to speak to the labs technical people to give you an account and instructions on how to download the lab results.
Lifelabs Excelleris
- Contact a LifeLabs Client Products Specialist by emailing ITHelpDesk@lifelabs.com and tell them that you are “a medical doctor that is self-hosting your own OSCAR EMR server” and that you need to be set up with the “Excelleris (ON)” and be given a “PCX file” and login credentials to download the labs.
- Install the PCX file into your browser and you will be able to download manually for testing
- Follow the ExcellerisDownload.sh instructions below for automation.
LifeLabs LRD (older approach) “MDS”
- Contact a LifeLabs Client Products Specialist by emailing ITHelpDesk@lifelabs.com and tell them that you are “a medical doctor that is self-hosting your own OSCAR EMR server” and that you need to be set up with the “LifeLabs Results Distribution System (LRDS)” and be given a “four digit route” and be given SFTP credentials (if possible) to download the labs.
- You need to get the form “Electronic Results Request Form” to fill out and return to LifeLabs.
- Vendor: specify your clinic name
- Contact: specify yourself
- PC/Server Operating System: Windows Pro or Mac (unfortunately, there is no Linux version)
- EMR Software Version: OSCAR McMaster / Community Edition v.19 (or whatever you are using)
- Switch (Winblast to SFTP): leave this blank
- New Install: checkmark this
- Client information: fill this section out with your physicians that need lab download to your system (you can add multiple physicians now or later to your account)
- LifeLabs will ask you for your OSCAR EMR vendor, and you will have to reply to them again and clarify that you are your own self-hosted vendor and need to be able to download the HL7 files yourself to upload to your EMR. Remind them that you are a MD medical doctor (not some random IT person), that you do not have OLIS, and that you need to be able to download labs to do your doctor work.
- LifeLabs will not actually give you your SFTP credentials, but will pre-package a WinSCP-type program (“proprietary SFTP client”) for you to install that has your unique ID and password and some custom scripts to be able to access their secure SFTP server to download HL7 lab files from your account to your computer. This is a rather ancient and cumbersome method, but LifeLabs has no incentive to pay developers to update this original EDT system when they have OLIS or Excelleris.
- You may also need to provide LifeLabs with your static IP address from where you are downloading the lab to.
- LifeLabs will send you a one time download link via a secure email. Do not click this link yet! Proceed with this next steps in the document to set up your dedicated lab download workstation or virtual machine, before you continue to set up the LifeLabs downloader. The software download link is a one-time access, and it may be time limited, so you can consider setting up your lab download system first before requesting the link. If the link expires, you can contact your LifeLabs Client Products Specialist to send it to you again.
Dynacare
- Contact Gamma Dynacare Medical Laboratories and tell them that you are a medical doctor, and that you are requesting to be set up with an “eResults SFTP account to download HL7 lab files for importing to your EMR“. Dynacare likes physicians to use their web-based Contact Us form (choose the option for Healthcare Providers and Hospitals). They don’t have a specific general email to contact. But once you get the email of one specialist, you can build a rapport with them and they can be quite helpful.
- If you are asked which system you are interested in being set up, tell them that you need the “Version 2.0 Batch Service (EMR) Users“. You can also request and account for the web based Online Transaction Service User (v 3.x), but this is more for downloading individual lab results for one patient at a time (not what you need for EMR).
- You will be asked to complete and sign a Dynacare eResults – Terms of Use Agreement.
- Organization / Clinic: your clinic name
- User Name: give yourself a unique download username
- Email: your email
- Results to be: choose “Downloaded into your EMR”
- EMR Provider: self-hosted
- Name of software: OSCAR EMR
- Phone: your phone for tech contact
- You will be given a SFTP username, password, and the website of their SFTP server. Please keep this credentials private and confidential!
- You may also need to give them your fixed IP address from where you will be doing the batch lab download from.
AlphaLabs
- Contact AlphaLabs and ask them for an SFTP lab download account.
- You will be given an SFTP username, password, and the website of their SFTP server. Please keep this credentials private and confidential!
Create a Virtual Machine to download and process labs
Lifelabs “LRD” service requires a Windows PC or a Mac to be the lab downloader. For this reason most lab downloader instructions are based on Windows however the recent option of using the newer Excelleris “Rover” API for Lifelabs lets you run the service on Linux.
An even better solution, would be to set up virtual machine on your server. Follow the same instructions on setting up a virtual machine for OSCAR, but instead of installing OSCAR, install your own copy of Windows Pro. We recommend Pro versions because it allows you to remote desktop to the Windows and manage the downloader files easily. This lab download method has been tested to work with Windows XP Pro. It has not been tested by us to work with newer Windows 10 / 11, check with Lifelabs technical people if their solution works with the Windows version you are installing. Do not allow anything else or anyone else to use this machine to limit user induced security vulnerabilities.
- Set up a Windows workstation or virtual machine to be the dedicated lab downloader.
- If you don’t know how to set up the downloaders to run as a Service, you could set up Windows to automatically log-in to an account on startup, and then immediately Lock Screen (to switch out but stay logged in)
Setup the Lab Downloaders
Lifelabs Excelleris
- A script can be obtained free from Peter Hutten-Czapski to download on a linux machine the Excelleris files from their “Rover” API.
- Place the supplied ExcellerisDownload.sh script into a DIRECTORY in your server and make it executable
- Copy the Excelleris PFX file for Clinic to a subdirectory with structure similar to
<SCRIPT DIR>/excelleris_download/cert/QA Peter Hutten Czapski MPC.pfx - Create a config_inc.txt file in the top <SCRIPT DIR> with the following contents:
CONTEXT="<Whatever you like to identify this set>" USERNAME="<Excelleris supplied User ID>" PASSWORD="<Excelleris supplied Password>" PFX="<Excelleris supplied PFX file>" CERT_PASS="<Excelleris supplied certificate passphrase>" HL7URLPATH="<test or production URI for the given province as supplied by Excelleris>" MULE_HOME="<path to mule-1.3.3 directory that you installed (see Mule Uploader below)>" MULE_HL7PATH="<path to downloaded lifelabs reports from your Mule configuration>" MULE_ERRORS="<path you configured/will configure for reports that fail to upload to OSCAR via Mule from your Mule configuration>" MULE_DONE="<path you configured/will configure for uploaded files>" MULE_LOG="<path to Mule log file>" FILE_LOG="<path to Downloading script log file>" EMAIL="<email address for error report>" SLEEP=<no quotes just the typical number of seconds needed for OSCAR to process upload>
- Example settings
CONTEXT="OSCAR 19 Production" USERNAME="hfhtclinic" PASSWORD="password" PFX="QA Peter Hutten Czapski MPC.pfx" CERT_PASS="passphrase" HL7URLPATH="https://api.on.excelleris.com/hl7pull.asp" MULE_HOME="/home/peter/hl7_file_management/mule-1.3.3" MULE_HL7PATH="/home/peter/HL7Data/incomingHL7Dir" MULE_ERRORS="/home/peter/HL7Data/errorHL7dir" MULE_DONE="/home/peter/HL7Data/completedHL7dir" MULE_LOG="${MULE_HOME}/logs/mule.log" FILE_LOG="excelleris.log" EMAIL="peter@hfhtclinic.net" SLEEP=300
LifeLabs LRD (older)
- In your Windows or Mac dedicated lab download workstation or virtual machine, use the link in the LifeLabs email to download the LRDS software.
- Install the LRDS software on the machine.
- The installer will create the following folders
- C:\lifelabs
- C:\DATA\LIFELABS
- The program to run to download labs is “C:\lifelabs\HL7Main.exe”
- Create a “Scheduled Task” in Windows or Mac to regularly run this program “C:\lifelabs\HL7Main.exe” every hour (or however often you want to check for new labs). Remember to allow this Task to run as administrator.
- The HL7 files will be dropped in to the folder “C:\DATA\LIFELABS”
- The LifeLabs downloader will check for new labs, download the HL7 files, and then generate their “CURHST” file which they use to validate whether a lab was successfully downloaded before deleting it off their server.
- A script can be obtained from Ian Pun to use Linux to download LRD.
Dynacare
- Create a directory “C:\DATA\GDML” to store your log files and incoming HL7 files from Dynacare
- Create a directory “C:\home\GammaDynacareScript” to hold the custom lab download scripts
- Use the custom Windows batch file script (created by KC Lai) and the Python script (created by Ian Pun) to automatically download from Dynacare. Join OSCAR Ontario Facebook Group to contact Kevin or Ian for the custom scripts.
- Edit the batch file and the Python script to match your instance (Dynacare SFTP username and password)
- user=”username”
- pw=”password”
- Create a “Scheduled Task” in Windows or Mac to regularly run KC Lai’s batch file (for logging the execution of Ian Pun’s Python script) on an hourly schedule.
AlphaLabs
- Create a directory “C:\DATA\ALPHA” for log files and incoming HL7 lab files from AlphaLabs
- Create a directory “C:\home\AlphaLabScript” for the custom lab download script
- Install WinSCP so you can run a custom download script from SFTP servers
- Open WinSCP and make an FTP connection to AlphaLabs SFTP server with your username and password (given by AlphaLabs)
- Copy down the SSH host key fingerprint to the authorized AlphaLab’s SFTP server. You will need this to put in to the custom script to automatically connect without manually confirming an authorized server each time the script runs.
- Create a WinSCP script in “C:\home\AlphaLabScript\AlphaLabDownloadScript.txt” with the following text (be sure to edit the username, password, domain name of the SFTP server, and the hostkey fingerprint details to suit your instance). This text file must be kept secure (as it contains plaintext passwords) and should not be accessible by anyone except yourself or a trusted IT person.
# Connect to AlphaLabs and Download labs # by Kevin Lai (2-July-2019) # Accept any hostkey: # open -hostkey=* sftp://username:password@alphalaboratories.ca # Accept verified hostkey: open -hostkey="ssh-rsa 1024 e3:4f:5h:63:g3:5h:6j:6g:2g:k8" sftp://username:password@alphalaboratories.ca # Change directory cd /outbox # Download files and delete from source when downloaded get -delete "*.*" "c:\DATA\Alpha\Alpha%TIMESTAMP#yyyymmddhhnnss%-*.*" # Download files but leave on server #get "*.*" "c:\DATA\Alpha\Alpha%TIMESTAMP#yyyymmddhhnnss%-*.*" # Logoff close # Quit WinSCP exit
- Create a Windows batch file and save it to “C:\home\AlphaLabScript\AlphaLabDownloadBatch.bat” with the following text:
REM Batch file for downloading AlphaLabs HL7 labs REM by Kevin Lai (2019) REM Logs output with timestamp echo off REM Determining the datestamp for backup folder: for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a" set "YY=%dt:~2,2%" & set "YYYY=%dt:~0,4%" & set "MM=%dt:~4,2%" & set "DD=%dt:~6,2%" set "HH=%dt:~8,2%" & set "Min=%dt:~10,2%" & set "Sec=%dt:~12,2%" set "datestamp=%YYYY%%MM%%DD%" & set "timestamp=%HH%:%Min%:%Sec%" set "fullstamp=%YYYY%-%MM%-%DD%_%HH%%Min%%Sec%" echo on echo datestamp: "%datestamp%" echo timestamp: "%timestamp%" echo fullstamp: "%fullstamp%" echo Running AlphaLabDownloadBatch.bat on... "%fullstamp%" >> c:\DATA\Alpha\Alpha-log.txt "c:\Program Files (x86)\WinSCP\winscp.com" /ini=nul /script=AlphaLabDownloadScript.txt /log=winscp-log.txt /loglevel=0 >> c:\DATA\Alpha\Alpha-log.txt
- Create a “Scheduled Task” in Windows or Mac to regularly run KC Lai’s script “C:\home\AlphaLabScript\AlphaLabDownloadBatch.bat” on an hourly schedule.
Next Steps
You can now take the HL7 lab files downloaded with the above scripts and manually upload them in to your OSCAR EMR via “InBox -> HL7 Lab Upload” and select the correct Lab type depending on which lab vendor the file is from
All that said we recommend you install Mule which automatically takes the HL7 files in a specific directory and uploads it to your EMR.
Credits
- Dr. Peter Hutten-Czapski
- Marc Dumontier
- Dr. Kevin Lai
- Dr. Ian Pun