The SPECIALIST Lexicon

Annual Lexicon Release

This page describes annual release procedures for the SPECIALIST LEXICON. This Lexicon is included in the Metathesaurus DVD release.

Location

  • Development: ${LEXICON_DIR}/data/${YEAR}
  • Backup: ${BACKUP_DIR}/Releases/UMLS/${YEAR}_pre_release/LEX
  • Web: ${WWW_DIR}/Apache/apache/docs/LexSysGroup/Projects/lexicon/${YEAR}/release
  • Internal Deploy: /net/lexdev/export/home/lu/Backup/Releases/UMLS/${YEAR}_pre_release

  • Internal release: ${BACKUP_DIR}/Releases/UMLS/${YEAR}_pre_release
  • Official release: ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release

Preparation steps

  1. Freeze LEXICON and inflVars from LexBuild

  2. Prepare Internal release: ${YEAR}_pre_release/LEX

    File NameSourceOwner
    LEX/DOCS
    DOCS/lvg/index.htmlprevious yearChris
    DOCS/lvg/docs${LVG_DIR}/lvg${YEAR}/docsChris
    DOCS/techrpt.pdfprevious year (if no change)Amanda
    LEX/LEX_DB
    LEX_DB/AM.DB${ANTONYM_DIR}/data/0.Antonym/${YEAR}/output/antonyms.data.${YEAR}.release (after 2022+)Chris
    LEX_DB/DM.DB${DERIVATION_DIR}/5.allD/data/${YEAR}/data/derivation.data.${YEAR}.release (after 2012+)Chris
    LEX_DB/NC.DBprevious year (if no change)Chris
    LEX_DB/SM.DB${SYNONYM_DIR}/data/${YEAR}/outData/Results/synonym.data.${RELEASE}.release (after 2016+)Chris
    LEX/LEX_PGMS
    LEX_PGMS/lvg${YEAR}.tgz$WWW_DIR/Apache/apache/docs/LexSysGroup/Projects/lvg/${YEAR}/release/lvg${YEAR}.tgzChris
    LEX
    LEXICON${LEXICON_DIR}/data/${YEAR}/data/LEXICON.frozenChris
    LRABR${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRABRChris
    LRAGR${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRAGRChris
    LRCMP${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRCMPChris
    LRFIL${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRFILChris
    LRFIL.crlf${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRFIL.crlfChris
    LRFLDPrevious year (if no change)Chris
    LRMOD${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRMODChris
    LRNOM${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRNOMChris
    LRPRN${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRPRNChris
    LRPRP${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRPRPChris
    LRSPL${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRSPLChris
    LRTRM${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRTRMChris
    LRTYP${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRTYPChris
    LRWD${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRWDChris
    LEX/MISC
    MISC/inflection.table${LEXICON_DIR}/data/${YEAR}/tables.frozen/inflection.tableChris
    MISC/inflVars.data${LEXICON_DIR}/data/${YEAR}/tables.frozen/inflVars.dataChris
    MISC/LRFIL${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRFILChris
    MISC/LRFIL.crlf${LEXICON_DIR}/data/${YEAR}/tables.frozen/LRFIL.crlfChris
    MISC/negCueWords${ANTONYM_DIR}/data/0.Antonym/${YEAR}/output/antonyms.data.${YEAR}.releaseChris
    MISC/prevariants${LEXICON_DIR}/data/${YEAR}/tables.frozen/prevariantsChris
    LEX/NUMBERS
    NUMBERS/NRNUMprevious year (if no change)Chris
    NUMBERS/NRVARprevious year (if no change)Chris
    NUMBERS/number.entriesprevious year (if no change)Chris
    NUMBERS/number.grammar.txtprevious year (if no change)Chris

  3. Prepare ${YEAR}_pre_release/LEX_Extra
    • ASCII
    • XML

  4. Prepare ${YEAR}_pre_release/LEX.tgz
    • Items that need to wait for the Lexical Tools
      • ./DOCS/lvg/docs
      • ./LEX_PGMS/lvg${YEAR}.tgz

    • shell> cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_pre_release
    • shell> gtar -czvf LEX.tgz LEX
    • send to OCCS for the internal release (after 2008)

  5. Prepare ${YEAR}_AA_release/LEX and LEX_Extra
    • cd ${LEXICON_DIR}/bin 
    • 6.ReleaseLexicon ${YEAR}
    • 11.ReleaseLexiconExtra ${YEAR}

    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release
    • create LEX.tgz (gtar -czvf LEX.tgz LEX)

  6. Copy to ${WWW_LEXICON}
    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release/LEX 
    • cp -rp * ${WWW_LEXICON}/htdocs/release/LEX_DOC

    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release/LEX_Extra 
    • cp -rp * ${WWW_LEXICON}/htdocs/release/LEX_DOC

    • cd ${BACKUP_DIR}/Releases/UMLS/${YEAR}_AA_release 
    • cp -rp LEX.tgz ${WWW_LEXICON}/htdocs/release/.

  7. Upload to LHC-NEXUS: lhc-lexicon-raw/www/lexicon/${YEAR}
    • cd ${WWW_LEXICON}
    • Make uploadAll
      • Upload ./release/LEX.tgz
      • Upload ./release/LEX_DOC.tgz
      • Upload ./docs/baseOrder.tgz
      • Upload ./docs/java8_data.tgz
      • Upload ./docs/mwConsumerData.tgz
      • Upload ./docs/mwData.tgz
    • Test and validate uploaded files 
      • downloadFiles ${YEAR} TRUE
        => downloaded above 6 tgz files to ${HOME}/ci-cd-data/lexicon 
      • unzipFiles ${YEAR} TRUE
        => upzip above 5 tgz files (exclude LEX.tgz) to ${HOME}/ci-cd-data/lexicon
        => manually check contents

    • Upload to LHC-download:

      Sned an email to Ci-Cd person (Anton) for following upload

      • From: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw > www > lvg > ${YEAR} > lvg${YEAR}.tgz
        To: https://data.lhncbc.nlm.nih.gov/public/lsg/lvg/${YEAR}/release/lvg${YEAR}.tgz
      • From: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw > www > lvg > ${YEAR} > lvg${YEAR}.tgz
        To: https://data.lhncbc.nlm.nih.gov/public/lsg/lvg/${YEAR}/release/lvg${YEAR}.tgz

      • From: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw > www > lexicon > ${YEAR} > release > LEX.tgz
        To: https://data.lhncbc.nlm.nih.gov/public/lsg/lexicon/${YEAR}/release/LEX.tgz
      • From: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw > www > lexicon > ${YEAR} > release > LEX_DOC.tgz
        To: https://data.lhncbc.nlm.nih.gov/public/lsg/lexicon/${YEAR}/release/LEX_DOC.tgz

        Need to unzip to https://data.lhncbc.nlm.nih.gov/public/lsg/lexicon/${YEAR}/release/LEX_DOC