Because of a lapse in government funding, the information on this website may not be up to date, transactions submitted via the website may not be processed, and the agency may not be able to respond to inquiries until appropriations are enacted. The NIH Clinical Center (the research hospital of NIH) is open. For more details about its operating status, please visit cc.nih.gov. Updates regarding government operating status and resumption of normal operations can be found at OPM.gov.

The SPECIALIST Lexicon

Annual Lexicon Release

This page describes annual release procedures for the SPECIALIST LEXICON. This Lexicon is included in the Metathesaurus DVD release.

Location

  • Development: ${LEXICON_DIR}/data/${YEAR}
  • Backup: ${BACKUP_DIR}/Releases/UMLS/${YEAR}_pre_release/LEX
  • Web: ${WWW_DIR}/Apache/apache/docs/LexSysGroup/Projects/lexicon/${YEAR}/release
  • Internal Deploy: /net/lexdev/export/home/lu/Backup/Releases/UMLS/${YEAR}_pre_release

  • Internal release: ${BACKUP_DIR}/Releases/UMLS/${YEAR}_pre_release (simplfied after 2023+)
  • Official release: ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release

Preparation steps

  1. Freeze LEXICON and inflVars from LexBuild

  2. Prepare release: ${YEAR}_AA_release/LEX in the following steps
    • LEX:
      • Annual Lexicon: Lexicon LR-Tables, MISC/*, NUMBERS/*
      • Verb Complmentatin Types: VERB_COMPLEMENTS/*
      • Database: LEX_DB/AM_DB,DM_DB,SM_DB, MISC/negCueWords
      • Lvg: DOCS/lvg, LEX_PGMS/lvg${YEAR}.tgz
    • LEX_Extra:
      • Annual Lexicon: ASCII
      • Annual Lexicon: XML
  3. LEXICON Repository (LEX):

    File NameSourceOwner
    DOCS
    DOCS/lvg/index.htmlprevious yearChris
    DOCS/lvg/docs${LVG_DIR}/lvg${YEAR}/docsChris
    DOCS/techrpt.pdfprevious year (if no change)Amanda
    LEX_DB
    LEX_DB/AM.DB${ANTONYM_DIR}/data/0.Antonym/${YEAR}/output/antonyms.data.${YEAR}.release (after 2022+)Chris
    LEX_DB/DM.DB${DERIVATION_DIR}/5.allD/data/${YEAR}/data/derivation.data.${YEAR}.release (after 2012+)Chris
    LEX_DB/NC.DBprevious year (if no change)Chris
    LEX_DB/SM.DB${SYNONYM_DIR}/data/${YEAR}/outData/Results/synonym.data.${YEAR}.release (after 2016+)Chris
    LEX_PGMS
    LEX_PGMS/lvg${YEAR}.tgz$WWW_DIR/Apache/apache/docs/LexSysGroup/Projects/lvg/${YEAR}/release/lvg${YEAR}.tgzChris
    ./
    LEXICON${LEXICON_DIR}/data/${YEAR}/data/LEXICON.frozenChris
    LRABR${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRABRChris
    LRAGR${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRAGRChris
    LRCMP${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRCMPChris
    LRFIL${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRFILChris
    LRFIL.crlf${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRFIL.crlfChris
    LRFLDPrevious year (if no change)Chris
    LRMOD${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRMODChris
    LRNOM${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRNOMChris
    LRPRN${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRPRNChris
    LRPRP${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRPRPChris
    LRSPL${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRSPLChris
    LRTRM${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRTRMChris
    LRTYP${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRTYPChris
    LRWD${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRWDChris
    MISC
    MISC/inflection.table${LEXICON_DIR}/data/${YEAR}/tables.frozen/inflection.tableChris
    MISC/inflVars.data${LEXICON_DIR}/data/${YEAR}/tables.frozen/inflVars.dataChris
    MISC/LRFIL${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRFILChris
    MISC/LRFIL.crlf${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRFIL.crlfChris
    MISC/negCueWords${ANTONYM_DIR}/data/0.Antonym/${YEAR}/output/antonyms.data.${YEAR}.releaseChris
    MISC/prevariants${LEXICON_DIR}/data/${YEAR}/tables.frozen/prevariantsChris
    NUMBERS
    NUMBERS/NRNUMprevious year (if no change)Chris
    NUMBERS/NRVARprevious year (if no change)Chris
    NUMBERS/number.entriesprevious year (if no change)Chris
    NUMBERS/number.grammar.txtprevious year (if no change)Chris
    VERB_COMPLEMENTS
    VERB_COMPLEMENTS/lightVerbs.data${MULTIWORD_DIR}/data/${YEAR}/outData/14.VerbComplements/lightVerbs.data.${YEAR}.releaseChris
    VERB_COMPLEMENTS/lightVerbs.inflVars${MULTIWORD_DIR}/data/${YEAR}/outData/14.VerbComplements/lightVerbs.inflVars.${YEAR}.releaseChris
    VERB_COMPLEMENTS/verbParticles.data${MULTIWORD_DIR}/data/${YEAR}/outData/14.VerbComplements/verbParticles.data.${YEAR}.releaseChris
    VERB_COMPLEMENTS/verbParticles.inflVars${MULTIWORD_DIR}/data/${YEAR}/outData/14.VerbComplements/verbParticles.inflVars.${YEAR}.releaseChris

  4. Prepare ${YEAR}_AA_release/LEX_Extra
    • ASCII
    • XML

  5. Prepare ${YEAR}_AA_release/LEX.tgz
    • Items that need to wait for the Lexical Tools
      • ./DOCS/lvg/docs
      • ./LEX_PGMS/lvg${YEAR}.tgz

    • shell> cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_pre_release
    • shell> tar -czvf LEX.tgz LEX
    • send to OCCS for the internal release (after 2008)

  6. Programs for preparing ${YEAR}_AA_release/LEX and LEX_Extra
    • cd ${LEXICON_DIR}/bin 
    • 6.ReleaseLexicon ${YEAR}
    • 11.ReleaseLexiconExtra ${YEAR}

    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release
    • create LEX.tgz (tar -czvf LEX.tgz LEX)

  7. Copy to ${WWW_LEXICON}
    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release/LEX 
    • cp -rp * ${WWW_LEXICON}/htdocs/release/LEX_DOC

    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release/LEX_Extra 
    • cp -rp * ${WWW_LEXICON}/htdocs/release/LEX_DOC

    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release 
    • cp -rp LEX.tgz ${WWW_LEXICON}/htdocs/release/.

  8. zip LEX_DOC
    => no need to do this, it is done automatically in the uploadFiles script.
    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release 
    • tar -czvf LEX_DOC.tgz LEX_DOC

  9. Upload to LHC-NEXUS: lhc-lexicon-raw/www/lexicon/${YEAR}
    • cd ${WWW_LEXICON}
    • make uploadRelease
      • upload ./release/LEX.tgz
      • upload ./release/LEX_DOC.tgz
    • make uploadAll (not used after 2024+)
      • upload ./release/LEX.tgz
      • upload ./release/LEX_DOC.tgz
      • upload ./docs/baseOrder.tgz
      • upload ./docs/java8_data.tgz
      • upload ./docs/mwConsumerData.tgz
      • upload ./docs/mwData.tgz
    • Test and validate uploaded files 
      • downloadFiles ${YEAR} TRUE
        => downloaded above 6 tgz files to ${HOME}/ci-cd-data/lexicon 
      • unzipFiles ${YEAR} TRUE
        => upzip above 5 tgz files (exclude LEX.tgz) to ${HOME}/ci-cd-data/lexicon
        => manually check contents

    • Upload to LHC-download:

      Send an email to Ci-Cd team (NLM LHC_AWS_DevOps) for following upload

      • From: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw > www > lvg > ${YEAR} > lvg${YEAR}.tgz
        To: https://data.lhncbc.nlm.nih.gov/public/lsg/lvg/${YEAR}/release/lvg${YEAR}.tgz
      • From: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw > www > lvg > ${YEAR} > lvg${YEAR}.tgz
        To: https://data.lhncbc.nlm.nih.gov/public/lsg/lvg/${YEAR}/release/lvg${YEAR}.tgz

      • From: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw > www > lexicon > ${YEAR} > release > LEX.tgz
        To: https://data.lhncbc.nlm.nih.gov/public/lsg/lexicon/${YEAR}/release/LEX.tgz
      • From: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw > www > lexicon > ${YEAR} > release > LEX_DOC.tgz
        To: https://data.lhncbc.nlm.nih.gov/public/lsg/lexicon/${YEAR}/release/LEX_DOC.tgz

        and then unzip to https://data.lhncbc.nlm.nih.gov/public/lsg/lexicon/${YEAR}/release/LEX_DOC