TOOLS

Web API

This Java-based API to the Indexing Initiative Scheduler facility was created to provide users with the ability to programmatically submit jobs to the Scheduler Batch and Interactive facilities instead of using the web-based interface.

Three programs are accessible via the Web API: MetaMap, the NLM Medical Text Indexer (MTI), and SemRep. The functionality in these programs includes: automatic indexing of MEDLINE citations, concept-based query expansion, analysis of complex Metathesaurus strings, accurate identification of the terminology and relationships in anatomical documents, and the extraction of chemical binding relations from biomedical text.

Terms of Service   Web API Download Button

The Unified Medical Language System® (UMLS®) is in the process of moving to a new authentication method (Fall 2020 Changes to UMLS Terminology Services (UTS)). The new login method required an update to our SKR Web API to allow for the UTS API Key instead of password. Please read through the section entitled "API Authentication" from the above link to identify your own UTS API Key which will need to be included in calls to our SKR Web API.

This necessitates you update to our latest version of the SKR Web API (Version 2.4) which is a complete replacement to previous releases. Please follow the directions for downloading and installing the SKR Web API.

Web API Download Button
Version: 2.4
Size: 1.7MB
Description: Libraries, Examples, Documentation, and all Source Code

Installation:

To install the Web API, do the following:

  1. Make a top level directory to store the Web API program.

  2. Download the latest version of the Web API into the directory created in #1.

  3. Extract the jar files from the downloaded jar file - SKR_Web_API_V2_4.jar.
    • On Windows: Run winzip, pkzip, or zip on the downloaded file.

    • On Unix: Run the java jar program on the jar file: java sun.tools.jar.Main xf SKR_Web_API_V2_4.jar
  4. This will create a directory called "SKR_Web_API_V2_4" in the directory you created in #1 and fill it with all of the program files needed.

  5. Now, change to this new directory: cd SKR_Web_API_V2_4

  6. You are now ready to run. Look at the example programs in the "examples" subdirectory for ideas on what to do.

For Web API access to our tools, you must have activated a UMLS Terminology Services (UTS) account. For more information please visit our Help about UTS accounts Web page. This account is free with only the minor requirement of filing a brief annual report on your use of the UMLS.

Users are also responsible for compliance with the UMLS Metathesaurus License Agreement which requires you to respect the copyrights of the constituent vocabularies.

The Web API package is protected under the MetaMap Terms and Conditions. Please review prior to downloading the Web API package.

  • A running version of Java.   We have tested and confirmed that the Web API program compiles and runs under the following versions of Java:
    • 1.8.0_275
  • We have tested and confirmed that the Web API program runs on the following operating systems:
    • Linux
  • Apache's Jakarta Project Ant   program is only required if you wish to modify and/or compile the Web API source code using the existing build file:
    • 1.6 or better
  • Your data in a recognized format. Please see the Supported File Formats section of the Help Information web page for detailed information and examples of valid input formats.

Web API Download Button
Version: 2.4
Size: 1.7MB
Description: Libraries, Examples, Documentation, and all Source Code

Installation:

To install the Web API, do the following:

  1. Make a top level directory to store the Web API program.

  2. Download the latest version of the Web API into the directory created in #1.

  3. Extract the jar files from the downloaded jar file - SKR_Web_API_V2_4.jar.
    • On Windows: Run winzip, pkzip, or zip on the downloaded file.

    • On Unix: Run the java jar program on the jar file: java sun.tools.jar.Main xf SKR_Web_API_V2_4.jar
  4. This will create a directory called "SKR_Web_API_V2_4" in the directory you created in #1 and fill it with all of the program files needed.

  5. Now, change to this new directory: cd SKR_Web_API_V2_4

  6. You are now ready to run. Look at the example programs in the "examples" subdirectory for ideas on what to do.

Change History
Version 1.3 Initial Public Release September 6, 2007
Version 1.4 Minor Change Release (Updated defaults) July 2, 2008
Version 1.5 Minor Change Release (Updated defaults) July 30, 2009
Version 2.0 Updated for UTS Authentication and Simplified June 15, 2011
Version 2.1 Updated to allow looping calls without re-authentication June 1, 2012
Version 2.3 Moved from http to https and skr.nlm.nih.gov to ii.nlm.nih.gov October 27, 2016
Version 2.4 Moved to UTS API authentication and new UTS Authentication Process December 31, 2020

There are six example programs included in the "examples" directory to illustrate how to incorporate the Web API into your application. Two of the examples (GenericBatch.java and GenericBatchNew.java) show how to access the Scheduler Batch facility. MMInteractive and SRInteractive.java examples show how to access Interactive MetaMap and Interactive SemRep respectively. The last two examples GenericBatchUser.java and MMInteractiveUser.java illustrate how to specify the username and password within the program to eliminate the need for prompting when running.

For a list of the available options, please select the appropriate program name: MetaMap and MTI.

GenericBatch.java Example program for submitting a new Generic Batch with Validation job ("GenericObject(true)" turns on validation) request to the Scheduler to run. You will be prompted for your username and password and if they are alright, the job is submitted to the Scheduler and the results are returned in the String "results" below.

This example shows how to setup a basic Generic Batch with Validation job with a small file (sample.txt) with ASCII MEDLINE formatted citations as input data. You must set the Email_Address variable and use the UpLoad_File to specify the data to be processed. This example also shows the user setting the SilentEmail option which tells the Scheduler to NOT send email upon completing the job.

This example is set to run the MTI (Medical Text Indexer) program using the -opt1L_DCMS and -E options. You can also setup any environment variables that will be needed by the program by setting the Batch_Env field.

NOTE: The "-E" option/argument is very important and should be included with whatever program (MetaMap, SemRep, MTI) you decided to run! The reason is that this version of the Generic Batch does validation and the -E option tells the various programs to include a marker denoting when a successful result has been found.
 
GenericBatchNew.java Same as GenericBatch.java above, except you can enter any of the options and the inputfile on the command line. The usage of the program and available options are listed below:
usage: GenericBatchNew [options] inputFilename
  allowed options: 
    --email <address> : set email address. (required option)
    --command <name> : batch command: metamap, semrep, etc. (default: MTI -opt1_DCMS -E)
    --note <notes> : batch notes 
    --silent : don't send email after job completes.
    --silent-errors : Silent on Errors
    --singleLineInput : Single Line Delimited Input
    --singleLinePMID : Single Line Delimited Input w/ID
    --priority : request a Run Priority Level: 0, 1, or 2
NOTE: The "-E" option/argument is very important and should be included with whatever program you decided to run! The reason is that this version of the Generic Batch does validation and the -E option tells the various programs to include a marker denoting when a successful result has been found.
 
MMInteractive.java This example shows how to setup a basic Interactive MetaMap request. This runs the latest version of MetaMap with 1516 (2015AB) version of the UMLS Metathesaurus ("KSOURCE", "1516"), with "ignore_word_order" (-i) and "all_derivational_variants" (D) set as arguments to MetaMap. The default "Human Readable" output will be produced.

NOTE: The "-E" option/argument is NOT required for Interactive use.
 
SRInteractive.java This example shows how to setup a basic Interactive SemRep request. This runs the latest version of SemRep with Full Fielded Output (-D).

NOTE: The "-E" option/argument is NOT required for Interactive use.
 
GenericBatchUser.java This example is the same as "GenericBatchNew.java" above, except that the username and password are set in the program so that the user is not prompted when the program runs. Please Note: Although we understand the operational need for being able to specify the username and password within a program, you must also be aware of the security risks when using this setup. Please take care to secure the software you create using this option.
 
MMInteractiveUser.java This example is the same as "MMInteractive.java" above, except that the username and password are set in the program so that the user is not prompted when the program runs. Please Note: Although we understand the operational need for being able to specify the username and password within a program, you must also be aware of the security risks when using this setup. Please take care to secure the software you create using this option.

Sample Batch Commands:

  • Default MetaMap call (latest version of MetaMap, strict data model, current UMLS Knowledge Source, human readable output, and best mappings only):
    metamap -E
  • MetaMap formatted XML output call (latest version of MetaMap, strict data model, current UMLS Knowledge Source, Formatted XML output, and best mappings only) showing both short and long form of the XML option:
    metamap -% format -E
      or
    metamap --XML format -E
  • Specify Semantic Types to Restrict MetaMap too (latest version of MetaMap, strict data model, current UMLS Knowledge Source, human readable output, best mappings only, only allow "Disease or Syndrome", "Finding", and "Injury or Poisoning" Semantic Types):
    metamap -J "dsyn,fndg,inpo" -E
  • Default MTI call (latest version of MTI, use PubMed Related Articles first and TexTool as backup for PRC, and default output):
    MTI -opt1L_DCMS -E
  • Default SemRep call (latest version of SemRep with Full Fielded Output):
    semrep -D -E

  • To compile the source jar file and/or any of the examples using Ant - change to the top-level directory and run:
    ant jar
  • To compile any of the examples without Ant, change to the examples directory first:
    cd examples

    Unix:
    javac -cp ../classes:../lib/skrAPI.jar:../lib/commons-logging-1.1.1.jar:../lib/httpclient-cache-4.1.1.jar:../lib/httpcore-nio-4.1.jar:../lib/httpclient-4.1.1.jar:../lib/httpcore-4.1.jar:../lib/httpmime-4.1.1.jar -d ../classes GenericBatch.java

    or, you can use the "compile.sh" script provided

         ../compile.sh GenericBatch.java

    Windows:
    javac -cp ..\classes;..\lib\skrAPI.jar;..\lib\commons-logging-1.1.1.jar;..\lib\httpclient-cache-4.1.1.jar;..\lib\httpcore-nio-4.1.jar;..\lib\httpclient-4.1.1.jar;..\lib\httpcore-4.1.jar;..\lib\httpmime-4.1.1.jar -d ..\classes GenericBatch.java

  • To run any of the examples, change to the examples directory first:
    cd examples

    Unix:
    java -cp ../classes:../lib/skrAPI.jar:../lib/commons-logging-1.1.1.jar:../lib/httpclient-cache-4.1.1.jar:../lib/httpcore-nio-4.1.jar:../lib/httpclient-4.1.1.jar:../lib/httpcore-4.1.jar:../lib/httpmime-4.1.1.jar GenericBatch

    java -cp ../classes:../lib/skrAPI.jar:../lib/commons-logging-1.1.1.jar:../lib/httpclient-cache-4.1.1.jar:../lib/httpcore-nio-4.1.jar:../lib/httpclient-4.1.1.jar:../lib/httpcore-4.1.jar:../lib/httpmime-4.1.1.jar GenericBatchNew --email your@email inputFilename

    or, you can use the "run.sh" script provided

         ../run.sh GenericBatch

         ../run.sh GenericBatchNew --email your@email inputFilename

    Windows:
    java -cp ..\classes;..\lib\skrAPI.jar;..\lib\commons-logging-1.1.1.jar;..\lib\httpclient-cache-4.1.1.jar;..\lib\httpcore-nio-4.1.jar;..\lib\httpclient-4.1.1.jar;..\lib\httpcore-4.1.jar;..\lib\httpmime-4.1.1.jar GenericBatch

    java -cp ..\classes;..\lib\skrAPI.jar;..\lib\commons-logging-1.1.1.jar;..\lib\httpclient-cache-4.1.1.jar;..\lib\httpcore-nio-4.1.jar;..\lib\httpclient-4.1.1.jar;..\lib\httpcore-4.1.jar;..\lib\httpmime-4.1.1.jar GenericBatchNew --email your@email filename

Third Party and Associated Licenses Used:

For specifics on the licensing and copyright information on each of these packages, please visit their respective link. Notices/Attributions will also be found in the source code when and where required.

Overview of What Happens:

The Web API has been designed to allow you to interact with our web-based Scheduler using our Batch and Interactive facilities. Interactive access is currently only available for the MetaMap and SemRep programs.

  • Batch Jobs have three main phases:
    1. Request is made for a new batch job and the Web API submits the job.

    2. Program waits for job to finish. This is done by monitoring the Scheduler queue to see when the job exits the queue. The program sleeps for 10 seconds after each check of the queue so that it doesn't become a nuisance process.

    3. Once the job has left the Scheduler queue, the program then checks to see if the "text.out.done" file exists in the job result directory. If it does, then the results file "text.out" is returned as a String to the calling method. If the "text.out.done" file doesn't exist, an error has occurred and the program exits.
  • Interactive Jobs only have two main phases:
    1. Request is made for a new interactive job and the Web API submits the job.

    2. Program waits for job to finish and returns the result as a String to the calling method. Interactive jobs don't really enter the Scheduler - they are run from the command line. The API simply waits for the command to return a result without having to worry about the Scheduler queue, etc., providing the fastest mechanism for receiving results.

    3. Note: Interactive jobs will only accept a single item up to 10,000 characters in length to process - anything larger will generate an error.

Programming Notes:

Whenever you see   in the text below, it means that additional information is available by selecting the symbol.

Required Fields
General Description Name Type
Email Address Email_Address String
File to Upload UpLoad_File String



Optional Fields
General Description Name Type
Silent on Errors   ESilent Boolean
Single Line Delimited Input   SingLine Boolean
Single Line Delimited Input w/ID   SingLinePMID Boolean
Requested Run Priority Level   RPriority String
No Completion Email   SilentEmail Boolean
  • You "should" catch "RuntimeException" exceptions in your program since this is what the Web API throws internally when it encounters an error. See example below for try/catch sample.

  • You must specify an Email Address. This is used for logging as well as notification when Batch jobs finish, or if there are problems with a job.

  • You must specify the UpLoad_File field.

  • Valid Batch job priorities are "0", "1", or "2". Where "0" represents Normal priority, "1" represents Medium, and "2" represents High priority. Medium and High priorities are reserved and you must have the proper permissions to use them.

  • Explanation of setting up a basic Web API program. The following description details the steps to setting up a simple Web API job to send a small file to the Scheduler and receive a result back. See the GenericBatch.java program file in the examples directory for the full program.

    1. The first thing you need to do is create a job object for the type of program you want to run.

      GenericObject myGenericObj = new GenericObject();

      This creates a Generic Batch job object with Validation.

    2. The next thing to do is setup any of the required or batch specific fields.

      myGenericObj.setField("Email_Address", "yourEmailAddress");
      myGenericObj.setField("SilentEmail", true);
      myGenericObj.setField("Batch_Command", "MTI -opt1L_DCMS -E");
      myGenericObj.setField("UpLoad_File", "./sample.txt");

      This tells the job object what your email address is (required), tells the Scheduler that we don't want to be notified after the job completes (SilentEmail), we want to run the MTI command with "-opt1L_DCMS -E" as options, and that we want to use the local file "sample.txt" as the input data to be processed (UpLoad_File).

    3. Now we are finally ready to actually submit the Batch job and receive our results back. We surround this submission code with a try/catch for any "RuntimeException" exceptions that might be thrown back to us from the Web API program.

      try
      {
         String results = myGenericObj.handleSubmission();
         System.out.print(results);
      } catch (RuntimeException ex) {
         System.err.println("");
         System.err.print("An ERROR has occurred while processing your");
         System.err.println(" request, please review any");
         System.err.print("lines beginning with \"Error:\" above and the");
         System.err.println(" trace below for indications of");
         System.err.println("what may have gone wrong.");
         System.err.println("");
         System.err.println("Trace:");
         ex.printStackTrace();
      } // catch

      This calls the job object routine that takes care of the job from submission, monitoring, cleaning up, and returning a result.

Presentation from NLM APIs Webinar on Web API:

The following presentation describing the MetaMap/MTI Web API (Web API) was delivered as part of the April 10, 2012 NLM APIs webinar. For more information on other NLM APIs and/or to see the full recorded webinar, please visit the NLM API page.

 (1.2 mb) Presentation from NLM APIs Webinar on Web API, April 2012
 (12.8 mb) Presentation from NLM APIs Webinar on Web API, April 2012 (PDF version, no slide animations)