Lab Download Integration
These are instructions on how to set up electronic data transfer (EDT) for downloading your labs in HL7 format from each individual major laboratory, and uploading them to your OSCAR EMR. This is the original method of getting electronic lab data to your EMR without the need for OntarioMD certification. The other newer methods are with OLIS (if you have an OntarioMD certified EMR), which the open source OSCAR Community Edition is not eligible. You vendor may have other newer methods of downloading lab data such as Excelleris (check with Open OSP Service Cooperative) for their solution.
Document Version History
- v1.0 – Initial public release to oscarmanual.org – Jan 12, 2016
© 2016 by Peter Hutten-Czapski MD under the Creative Commons Attribution-Share Alike 3.0 Unported License
- v1.1 – Initial OSCAR 15 release – Mar 5, 2016
- v2.0 – Ported over to oscargalaxy.org – 11-Oct-2022 (KC Lai)
- v2.1 – Added instructions on setting up lab download from lab vendors (KC Lai)
- You are familiar with Windows Command line, dealing with source code and Java build tools, and administrative access to a server or Windows workstation.
- You are familiar with how to setup a Windows virtual machine for the dedicated lab downloader. Or, you have a dedicated secure Windows or Mac workstation that can be kept on 24/7.
- You have a static IP for your clinic system that runs the lab downloader. Some labs restrict lab download to known static IP’s of authorized clinics/physicians. You need to contact your internet provider and pay a little extra for a static IP. If you static IP changes (upgrade services or new provider), remember to update the labs of your new static IP.
As a physician, you may order labs and patients may go to one of three major laboratory vendors in Ontario: Lifelabs (CML/MDS), Dynacare (Gamma Dynacare Medical Laboratory), and AlphaLabs. There are smaller labs, but it may not be worthwhile to get set up with their lab download accounts (if they have any in the first place).
The general process is this: Each lab will have a SFTP service that you can download HL7 lab files (electronically coded lab data) to a folder on your dedicated lab download computer or virtual machine. An HL7 transport utility (Mule) will upload the HL7 lab files from the folders to your EMR via a secure channel with private-public keypair in OSCAR for each lab vendor. The EMR will then process the HL7 lab data and file to the corresponding patients, and will show up in your InBox for review.
These instructions are mainly for a Windows system, but you can try to adapt the instructions for Mac or Linux.
Get Electronic Data Transfer (EDT) accounts
- Contact a LifeLabs Client Products Specialist by emailing ITHelpDesk@lifelabs.com and tell them that you are “a medical doctor that is self-hosting your own OSCAR EMR server” and that you need to be set up with the “LifeLabs Results Distribution System (LRDS)” and be given a “four digit route” and be given SFTP credentials (if possible) to download the labs.
- You need to get the form “Electronic Results Request Form” to fill out and return to LifeLabs.
- Vendor: specify your clinic name
- Contact: specify yourself
- PC/Server Operating System: Windows Pro or Mac (unfortunately, there is no Linux version)
- EMR Software Version: OSCAR McMaster / Community Edition v.19 (or whatever you are using)
- Switch (Winblast to SFTP): leave this blank
- New Install: checkmark this
- Client information: fill this section out with your physicians that need lab download to your system (you can add multiple physicians now or later to your account)
- LifeLabs will ask you for your OSCAR EMR vendor, and you will have to reply to them again and clarify that you are your own self-hosted vendor and need to be able to download the HL7 files yourself to upload to your EMR. Remind them that you are a MD medical doctor (not some random IT person), that you do not have OLIS, and that you need to be able to download labs to do your doctor work.
- LifeLabs will not actually give you your SFTP credentials, but will pre-package a WinSCP-type program (“proprietary SFTP client”) for you to install that has your unique ID and password and some custom scripts to be able to access their secure SFTP server to download HL7 lab files from your account to your computer. This is a rather ancient and cumbersome method, but LifeLabs has no incentive to pay developers to update this original EDT system when they have OLIS or Excelleris.
- You may also need to provide LifeLabs with your static IP address from where you are downloading the lab to.
- LifeLabs will send you a one time download link via a secure email. Do not click this link yet! Proceed with this next steps in the document to set up your dedicated lab download workstation or virtual machine, before you continue to set up the LifeLabs downloader. The software download link is a one-time access, and it may be time limited, so you can consider setting up your lab download system first before requesting the link. If the link expires, you can contact your LifeLabs Client Products Specialist to send it to you again.
- Contact Gamma Dynacare Medical Laboratories and tell them that you are a medical doctor, and that you are requesting to be set up with an “eResults SFTP account to download HL7 lab files for importing to your EMR“. Dynacare likes physicians to use their web-based Contact Us form (choose the option for Healthcare Providers and Hospitals). They don’t have a specific general email to contact. But once you get the email of one specialist, you can build a rapport with them and they can be quite helpful.
- If you are asked which system you are interested in being set up, tell them that you need the “Version 2.0 Batch Service (EMR) Users“. You can also request and account for the web based Online Transaction Service User (v 3.x), but this is more for downloading individual lab results for one patient at a time (not what you need for EMR).
- Organization / Clinic: your clinic name
- User Name: give yourself a unique download username
- Email: your email
- Results to be: choose “Downloaded into your EMR”
- EMR Provider: self-hosted
- Name of software: OSCAR EMR
- Phone: your phone for tech contact
- You will be given a SFTP username, password, and the website of their SFTP server. Please keep this credentials private and confidential!
- You may also need to give them your fixed IP address from where you will be doing the batch lab download from.
- Contact AlphaLabs and ask them for an SFTP lab download account.
- You will be given an SFTP username, password, and the website of their SFTP server. Please keep this credentials private and confidential!
Create a Virtual Machine to download and process labs
Although you can set up any Windows PC or a Mac to be the lab downloader, we would advise that you set up a dedicate machine for this task. This computer would need to be secured in a physical location, and would need to be running 24/7.
An even better solution, would be to set up virtual machine on your server. Follow the same instructions on setting up a virtual machine for OSCAR, but instead of installing OSCAR, install your own copy of Windows Pro. This lab download method has been tested to work with Windows XP Pro. We recommend Pro versions because it allows you to remote desktop to the Windows and manage the downloader files easily. It has not been tested to work with newer Windows 7 / 10. For this reason, we would recommend dedicating a virtual machine to do the lab download, and not allow anything else or anyone else to use this machine to limit security vulnerabilities.
- Set up a Windows workstation or virtual machine to be the dedicated lab downloader.
- If you don’t know how to set up the downloaders to run as a Service, you could set up Windows to automatically log-in to an account on startup, and then immediately Lock Screen (to switch out but stay logged in)
Setup the Lab Downloaders
- In your Windows or Mac dedicated lab download workstation or virtual machine, use the link in the LifeLabs email to download the LRDS software.
- Install the LRDS software on the machine.
- The installer will create the following folders
- The program to run to download labs is “C:\lifelabs\HL7Main.exe”
- Create a “Scheduled Task” in Windows or Mac to regularly run this program “C:\lifelabs\HL7Main.exe” every hour (or however often you want to check for new labs). Remember to allow this Task to run as administrator.
- The HL7 files will be dropped in to the folder “C:\DATA\LIFELABS”
- The LifeLabs downloader will check for new labs, download the HL7 files, and then generate their “CURHST” file which they use to validate whether a lab was successfully downloaded before deleting it off their server.
- Create a directory “C:\DATA\GDML” to store your log files and incoming HL7 files from Dynacare
- Create a directory “C:\home\GammaDynacareScript” to hold the custom lab download scripts
- Use the custom Windows batch file script (created by KC Lai) and the Python script (created by Ian Pun) to automatically download from Dynacare. Join OSCAR Ontario Facebook Group to contact Kevin or Ian for the custom scripts.
- Edit the batch file and the Python script to match your instance (Dynacare SFTP username and password)
- Create a “Scheduled Task” in Windows or Mac to regularly run KC Lai’s batch file (for logging the execution of Ian Pun’s Python script) on an hourly schedule.
- Create a directory “C:\DATA\ALPHA” for log files and incoming HL7 lab files from AlphaLabs
- Create a directory “C:\home\AlphaLabScript” for the custom lab download script
- Install WinSCP so you can run a custom download script from SFTP servers
- Open WinSCP and make an FTP connection to AlphaLabs SFTP server with your username and password (given by AlphaLabs)
- Copy down the SSH host key fingerprint to the authorized AlphaLab’s SFTP server. You will need this to put in to the custom script to automatically connect without manually confirming an authorized server each time the script runs.
- Create a WinSCP script in “C:\home\AlphaLabScript\AlphaLabDownloadScript.txt” with the following text (be sure to edit the username, password, domain name of the SFTP server, and the hostkey fingerprint details to suit your instance). This text file must be kept secure (as it contains plaintext passwords) and should not be accessible by anyone except yourself or a trusted IT person.
# Connect to AlphaLabs and Download labs # by Kevin Lai (2-July-2019) # Accept any hostkey: # open -hostkey=* sftp://username:firstname.lastname@example.org # Accept verified hostkey: open -hostkey="ssh-rsa 1024 e3:4f:5h:63:g3:5h:6j:6g:2g:k8" sftp://username:email@example.com # Change directory cd /outbox # Download files and delete from source when downloaded get -delete "*.*" "c:\DATA\Alpha\Alpha%TIMESTAMP#yyyymmddhhnnss%-*.*" # Download files but leave on server #get "*.*" "c:\DATA\Alpha\Alpha%TIMESTAMP#yyyymmddhhnnss%-*.*" # Logoff close # Quit WinSCP exit
- Create a Windows batch file and save it to “C:\home\AlphaLabScript\AlphaLabDownloadBatch.bat” with the following text:
REM Batch file for downloading AlphaLabs HL7 labs REM by Kevin Lai (2019) REM Logs output with timestamp echo off REM Determining the datestamp for backup folder: for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a" set "YY=%dt:~2,2%" & set "YYYY=%dt:~0,4%" & set "MM=%dt:~4,2%" & set "DD=%dt:~6,2%" set "HH=%dt:~8,2%" & set "Min=%dt:~10,2%" & set "Sec=%dt:~12,2%" set "datestamp=%YYYY%%MM%%DD%" & set "timestamp=%HH%:%Min%:%Sec%" set "fullstamp=%YYYY%-%MM%-%DD%_%HH%%Min%%Sec%" echo on echo datestamp: "%datestamp%" echo timestamp: "%timestamp%" echo fullstamp: "%fullstamp%" echo Running AlphaLabDownloadBatch.bat on... "%fullstamp%" >> c:\DATA\Alpha\Alpha-log.txt "c:\Program Files (x86)\WinSCP\winscp.com" /ini=nul /script=AlphaLabDownloadScript.txt /log=winscp-log.txt /loglevel=0 >> c:\DATA\Alpha\Alpha-log.txt
- Create a “Scheduled Task” in Windows or Mac to regularly run KC Lai’s script “C:\home\AlphaLabScript\AlphaLabDownloadBatch.bat” on an hourly schedule.
Installing Mule Uploader to EMR
Although you can now take the HL7 lab files download from the above scripts found in “C:\DATA\…” manually upload them in to your OSCAR EMR via “InBox -> HL7 Lab Upload” and select the correct Lab type depending on which lab vendor the file is from:
- ALPHA is AlphaLabs
- GDML is Dynacare
- MDS/LifeLabs is LifeLabs
We recommend you install Mule which automatically takes the HL7 files in a specific directory and uploads it to your EMR. The following section is ported from Peter HC’s original article on oscargalaxy.org. You may need to adapt the instructions for your situation (ie. version of Java or Maven, 32bit or 64bit installer etc)
Labwork can be pushed into OSCAR with the HL7 transport utility from a Windows workstation. Use similar instructions to install on a Linux machine.
Installing the Infrastructure Packages
The version below has been tested, you can use a newer one, just ensure that it is the same type. The wrapper that starts this Mule needs a 32 bit Java.
The following example uses the file pattern for 32 bit Oracle Sun Java 8 update 66.
The typical windows installation process will place the executable files in c:\Program Files (x86)\Java\jdk1.8.0_66\jre
Maven is tool that you will use to compile a java JAR file configured for your system. Download the latest maven from apache. It will be in a bin tar which you need to extract using either a command line tool such as TarTool or a GUI extractor such as 7-zip. Extract to an appropriate location such as Program Files.
TarTool apache-maven-3.3.9-bin.tar.gz "C:\Program Files\maven-3.3.9"
The maven package is thus extracted into a subdirectory.
Git is a software configuration management (SCM) tool that you can use to retrieve the source files for this program from the repository.
You can download either the 32bit or 64bit installer from https://git-scm.com/download/win
Then use Git to clone the source files into an appropriate location.
mkdir C:\home cd C:\home git clone git://git.code.sf.net/p/oscarmcmaster/hl7_file_management oscarmcmaster-hl7_file_management
Configuration of Packages
You should set some system variables so that Windows knows where to find the various components
- Open the ‘System’ control in the ‘Control Panel’
- Under the ‘Advanced’ tab click the ‘Environment Variables’ button
- In the ‘System variables’ window click the ‘New…’ button
- Variable Name: MULE_HOME
- Variable Value: C:\home\oscarmcmaster-hl7_file_management\mule-1.3.3
- Click ‘OK’
- In the ‘System variables’ window click the ‘New…’ button
- Variable Name: MAVEN_HOME
- Variable Value: C:\Program Files\maven-3.3.9
- Click ‘OK’
- In the ‘System variables’ window highlight the ‘Path’ variable and click the ‘Edit…’ button
- add ‘;%MULE_HOME%\bin;%MAVEN_HOME%\bin’ to the end of the ‘Variable Value’
- Click ‘OK’
- Click ‘OK’
Log into the OSCAR server and click the admin tab at the top right. Scroll to the bottom of the Administrative Page and click on the
Key Pair Generator link.
Input ‘oscar’ without the quotes as the service name and press the ‘Create Key Pair’ button. If you received the error: “Failed: The oscar key pair has already been created”. It is ok to ignore it.
Input your service as the service name and select the message type from the drop down menu and click the ‘Create Key Pair’ button. A window will pop up asking you to save ‘keyPair.key’. Remember the location where you saved it. (ie. “C:\home\oscar-keypairs\keypair_lifelabs.key” or “C:\home\oscar-keypairs\keypair_gdml.key”)
If you wish to use multiple message formats such as MDS / Lifelabs, Dynacare (GDML) and AlphaLabs, you will need a different keypair for each message format.
Note: The keypair file saved contains line feeds code (hidden CR/LF) which must be removed for Mule to read the keys correctly. In other words, you need to delete the hidden return/line feed that separates each line of the key so that the key is on one single line with no word wrap. Use an editor such as Notepad++ to edit the keyfile and can save in Linux format and also show you the hidden CR/LF line feeds. The default Windows Notepad does not save well in Linux format and so the keyfile may fail.
Configure Properties File
Edit the file ‘LabProperties.properties’ in the ‘src/main/resources’ directory to use your own directories. If the directories do not exist they will be created at runtime. Also make sure to include the address of the OSCAR server and the location of ‘keyPair.key’ that you saved in the previous step. Instructions are included at the top of the file, example settings are listed below. Note the UNIX style forward slash usage “c:/” for directory paths.
Example settings for a Windows system downloading from two labs (LifeLabs and Dynacare)
incomingHL7dir1 = c:/DATA/LIFELABS incomingHL7dir2 = c:/DATA/GDML incomingHL7dir3 = errorHL7dir = c:/home/messages/errorHL7dir completedHL7dir = c:/home/messages/completedHL7dir keyLocation1 = c:/home/oscar-keypairs/keypair_lifelabs.key keyLocation2 = c:/home/oscar-keypairs/keypair_gdml.key keyLocation3 = oscarURL = https://your.oscaremr.com:8443/oscar
# smtpServer for notifying errors (optional) smtpServer = smtp.server.com:25 senderEmailAddress = firstname.lastname@example.org recipientEmailAddress = email@example.com
Configure Config File
Depending on how many different message formats you wish to use you will have to use a different config file. If you are downloading just one lab then you can leave mule-config.xml as is. Otherwise use the template for the number of message formats (eg. mule-config_two-formats.xml for when you are downloading from Lifelabs and Dynacare), and save to ‘src/main/resources/mule-config.xml’. You will not usually need to edit the source template.
Build the Package
Open the command line by selecting ‘Run…’ from the start menu and typing in ‘cmd’ and clicking ‘OK’
Navigate to the ‘oscarmcmaster-hl7_file_management’ directory within the command line.
Set JAVA_HOME to point to the JDK using the following command ( be sure the directory is correct ):
set JAVA_HOME=c:\Program Files (x86)\Java\jdk1.8.0_66\jre
Use maven to build the source:
copy the resultant jar to the mule library (the existing jar in source needs to be overwritten):
copy target/IncomingHL7Management-1.0-SNAPSHOT.jar mule-1.3.3/lib/user/IncomingHL7Management-1.0-SNAPSHOT.jar
Leave the command line open for the next step.
Type the following into the command line:
Mule is now installed as a Windows service, it will start when windows boots. It can be started or stopped using the following commands:
mule start mule stop
Mule can be removed from being a service with the following command:
Changing the Mule configuration
If you want to add a third lab or change the config file, remember to stop Mule, remove Mule as a service; and then go back to editing the properties file, generating the Java package, and reinstalling the updated Mule.
mule stopmule remove
- Dr. Peter Hutten-Czapski
- Marc Dumontier
- Dr. Kevin Lai
- Dr. Ian Pun