Generate Lexicon

The Lexicon is released at the beginning of every year. This page provides detail information of generating Lexicon tables. This process requires iteratively fix Lexicon between LexBuilders and rerun validation program. It is normally takes 2-3 weeks for Steps 1 to 11; and another 2-3 weeks for tagging derivations (steps 12 to 13), 2 weeks for taggind synonyms (step 14) and 2 weeks for tagging antonyms (step 15).

• Overview

• Complete procedures of generating Lexicon release
  1. Establish baseline for new release
  2. PreProcess: the process of Abb/Acr LMW candidate tagging (to include new EUI from CR-check of Abb/Acr) needs to be completed before freezing the Lexicon
     * After 2020+, This step should be done before freeze the Lexicon
  3. Generate and backup Lexicon
  4. Freeze Lexicon

  5. Validate and fix Lexicon
  6. Generate pure ASCII Lexicon (have to be done before generating the Unicode tables)

  7. Generate Lexicon tables
  8. Generate Lexicon terms and statistics

  9. Generate Lexicon in XML format

  10. Generate verb complements (LVCs and VPCs) (1 week, July-Aug.)
  11. Generate annual Lexicon release
  12. Generate annual Lexicon Extra release (ASCII & XML) (1 month, July)

  13. Ask linguists for new rules of preifxD and suffixD for evaluation
  14. Generate DM.DB (derivations) (1 month, August)
  15. Generate SM.DB (synonyms) (3 weeks, Sept.)
  16. Generate AM.DB (antonyms) (2 weeks, Sept-Oct.)

• Tools to generate Lexicon

• Annual release of Lexicon

• Lexicon tables