Generate Lexicon

The Lexicon is released at the beginning of every year. This page provides detail information of generating Lexicon tables. This process requires iteratively fix Lexicon between LexBuilders and rerun validation program. It is normally takes 2-3 weeks for Steps 1 to 11; and another 2-3 weeks for tagging derivations (steps 12 to 13), 2 weeks for taggind synonyms (step 14) and 2 weeks for tagging antonyms (step 15).

• Overview

• Complete procedures of generating Lexicon release
  1. Establish baseline for new release
  2. ** After 2020+, complete the Abb/Acr LMW candidate tagging (to include new EUI fromCR-check of Abb/Acr). This step should be done before freeze the Lexicon
  3. Generate and backup Lexicon
  4. Freeze Lexicon

  5. Validate and fix Lexicon
  6. Generate pure ASCII Lexicon (have to be done before generating the Unicode tables)

  7. Generate Lexicon tables
  8. Generate annual Lexicon release
  9. Generate Lexicon terms and statistics
  10. Generate Lexicon in XML format
  11. Generate annual Lexicon Extra release (ASCII & XML) (1 month, July)

  12. Ask linguists for new rules of preifxD and suffixD for evaluation
  13. Generate DM.DB (derivations) (1 month, August)
  14. Generate SM.DB (synonyms) (3 weeks, Sept.)
  15. Generate AM.DB (antonyms) (2 weeks, Sept-Oct.)

• Tools to generate Lexicon

• Annual release of Lexicon

• Lexicon tables