Lexical Tools

Lexical Tool Web Site Annual Update Procedure

This page describes annual update procedures for the web site of SPECIALIST Lexical Tools..

I. Baseline

  • Check out from LHC-Git: wwwlvg
  • Dir: ${LHC_Git}/wwwlvg-p
  • only use master branch

II. Contents Update

  • Update direcories from the updated software
    • ${WWWLVG_DIR}/htdocs/docs/apiDoc
    • ${WWWLVG_DIR}/htdocs/docs/sources

  • Updates ${YEAR} from ${PREV_YEAR} in all html files
    • shell> cd ${WWWLVG_DIR}/htdocs/
    • shell> fgrep ${PREV_YEAR} *.html
    • shell> fgrep ${PREV_YEAR} */*.html,
      exceptions are:
      • ./web/faq.html
      • ./web/index.html
    • shell> fgrep ${PREV_YEAR} */*/*.html
      exceptions are:
      • ./docs/apiDoc/*.html, content="date"
      • ./web/release/${PREV_YEAR}.html
    • shell> fgrep ${PREV_YEAR} */*/*/*.html
      exceptions are:
      • ./docs/apiDoc/index-files/index-*.html, content="date"
    • shell> fgrep ${PREV_YEAR} */*/*/*/*.html
      exceptions are:
      • ./docs/designDoc/LifeCycle/test/fE.html
      • ./docs/designDoc/LifeCycle/test/performanceTest.html
      • ./docs/designDoc/UDF/derivations/prefixList.html
    • shell> fgrep ${PREV_YEAR} */*/*/*/*/*.html
      exceptions are:
      • ./docs/designDoc/LifeCycle/deploy/release/index.html
      • ./docs/designDoc/LifeCycle/deploy/release/procedures.html
      • ./docs/designDoc/UDF/unicode/MapTables/GeneralPunctuation.html
      • ./docs/designDoc/UDF/unicode/MapTables/U+2000-U+206F.html
    • shell> fgrep ${PREV_YEAR} */*/*/*/*/*/*.html
      exceptions are:
      • ./docs/designDoc/UDF/derivations/SD-Rules-Opti/Ex-2016/sdRules.stats.out.12.3.html
      • ./docs/designDoc/UDF/derivations/SD-Rules-Opti/Ex-2016/sources.html
      • ./docs/designDoc/UDF/derivations/SD-Rules-Opti/Ex-2020/sdRules.stats.out.09.2.html
    • shell> fgrep ${PREV_YEAR} */*/*/*/*/*/*/*.html
    • shell> fgrep ${PREV_YEAR} */*/*/*/*/*/*/*/*.html
      exceptions are:
      • ./docs/apiDoc/gov/nih/nlm/nls/lvg/../*.html, content="date"
    • shell> fgrep ${PREV_YEAR} */*/*/*/*/*/*/*/*/*.html
      exceptions are:
      • ./docs/apiDoc/gov/nih/nlm/nls/lvg/../*.html, content="date"

  • Automatic Generated HTML files (for Unicode)
    • > cd ${LHC_GIT}/lvg-unicodetables/bin
    • I. Lvg Default Mapping files
      • > 0.VerifyFiles <year>
      • > 1.GenerateDefaultHtmlFiles <year>
    • II. Unicode Core Norm Results
      • > 2.GetUnicodeCoreNormResult <year>
    • III. Norm Results
      • > 3.NormResults <year>

    • IV. Deploy results to Web Site
      • > 4.DeployHtmlFiles <year>
        4

  • Automatic Generated HTML files (for Unit Test)
    • > cd ${LHJC_GIT}/lvg-testexamples/bin
    • I. Generate unit test example files
      • > 1.GenerateExampleHtmlFiles <year>

    • II. Deploy files to Web Site
      • > 2.DeployExampleHtmlFiles <year>

III. Ci-Cd Pre-Processes: upload data

Upload data to LHC-Nexus and the ask Ci-Cd person (Anton) to upload to LHC-Download

  • Dir: ${LHC_GIT}/lvg
  • File: uploadFiles
  • Function: upload relese files from development to LHC_NEXUS
    • uploadFiles ${YEAR} TRUE
      must complete build and pack lvg in both Ant and maven first
      upload release files: lvg${YEAR}.tgz and lvg${YEAR}lite.tgz
    • Source Dir: ${LHC_GIT}/lvg
    • Target Dir: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw -> www -> lvg -> ${YEAR}
    • Files:
      • lvg${YEAR}.tgz
      • lvg${YEAR}lite.tgz
  • Upload files from Nexus to LHC download site:
    • Source: https://lhc-nexus.nlm.nih.gov/#browse/browse:lhc-lexicon-raw -> www -> lvg -> ${YEAR}/release
    • Target: https://data.lhncbc.nlm.nih.gov/public/lsg/lvg/${YEAR}/release
    • Processes: EMail Anton to manually upload.
  • Download and verify files from web site URL:
    • Dir: ${LHC_GIT}/wwwlvg-p
    • downloadFiles ${YEAR} TRUE
    • target dir: ${HOME}/ci-cd-data/lvg.${YEAR}
    • Check md5sum for all files

IV. Ci-Cd Processes: Development - Test - Stage - Deploy

  • Develop: develop in local machine lexdev (${GIT}/wwwlvg-p) 
            shell> git add -A
        shell> git commit -m "LEX_..."
        shell> git push
  • Test: use git tag to triggle Ci-Cd pipleline and deploy to Test site 
            shell> git tag v.${YAER}.stage.##
		shell> git push origin tag v.${YAER}.stage.##
		or
        shell> git push --tags (push all tags to origin)
  • Stage: Use the Git Ci-Cd pipleline and manual sent to Stage site by press the Deploy stage botton
  • Deploy: Manually contact with LHC-Portal staffs by eMail or JIRA ticket