You are here

Lexical Systems & Tools (SPECIALIST)

Project information
Research Area: 
Researchers: 

LHNCBC's Lexical Systems Group develops and maintains the SPECIALIST lexicon and the tools that support and exploit it. The SPECIALIST Lexicon and NLP Tools are at the center of NLM's natural language research, providing a foundation for all our natural language processing efforts. In general, we investigate the contributions that natural language processing techniques can make to the task of mediating between the language of users and the language of online biomedical information resources. The SPECIALIST NLP Tools facilitate natural language processing by helping application developers with lexical variation and text analysis tasks in the biomedical domain. A set of JAVA programs designed to help users manage lexical variation, indexing, and normalization, etc. in biomedical text. A nested set of JAVA objects designed to help users analyze free text documents into words, terms, phrases, sentences and sections. Java version of JDI tool to categorizes biomedical text for applications include Word Sense Disambiguation and retrieval according to discipline. Includes two programs GSpell a spelling suggestion tool and BagOwordsPlus a phrase retrieval tool. The dTagger us a Part of Speech (POS) tagger. Visual Tagging Tool (VTT) is a simple, lightweight, portable, Java Swing based annotation tool. It is designed to easily markup text. Sub-Term Mapping Tools (STMT) is a generic tool set that provides comprehensive sub-term related features for NLP applications. It is used to find the longest prefix, prefixes, sub-terms, and synonymous sub-term substitutions in query expansion. CSpell (spell checker for consumer language), a distributable spelling tool to detect and correct various types of spelling errors. CSpell handles non-word errors, real-words errors, word boundary infraction, punctuation errors, informal expression, and combinations of the above and result in high F1 score and real-time performance.

Recently, the Lexical Systems Group began a project to enhance the derivational-variants function of the lexical tools. The derivational-variants function uses a set of derivational facts and rules to generate or identify derivational variants of input terms. Derivational variants are words related by a word-formation process like suffixation, prefixation or conversion (change of category). The current derivational variant system has only suffix rules and facts. These rules and facts are hand entered and curated. In order to add suffixation and conversion functionality to the system, the PDM team has developed a method to automatically extract candidate pairs of words that may be derivationally related, which helps automate the creation of rules and facts for suffixation and conversion. In addition, LSG started to include consumer data in the Lexicon. This task includes: 1) adding terms from a consumer corpus to the Lexicon. This corpus is established by collecting health related articles from 16 public consumer-facing NIH websites. 2) collecting and tagging patient-centered interface terminology/vocabulary through a new feature of classification type in the Lexicon to cover the colloquial terms.

The SPECIALIST Lexicon and Lexical tools are open source and freely downloadable. The 2019 release of the SPECIALIST Lexicon contains over 515,000 records, representing over 972,000 forms, an increase of over 9,800 records from the 2018 release. These new terms are derived from a systematic approach using the NLM MEDLINE n-gram set and expert system models for effective Lexicon building.

Publications/Tools: 
Bashyam V, Divita G, Bennett DB, Browne AC, Taira RK. A normalized lexical lookup approach to identifying UMLS concepts in free text. Stud Health Technol Inform. 2007;129(Pt 1):545-9.
Tolentino HD, Matters MD, Walop W, Law B, Tong W, Liu F, Fontelo P, Kohl K, Payne DC. A UMLS-based Spell Checker for Natural Language Processing in Vaccine Safety. BMC Med Inform Decis Mak. 2007 Feb 12;7:3. DOI: 10.1186/1472-6947-7-3.
Keselman A, Massengale L, Ngo L, Browne A, Zeng Q. The Effect of User Factors on Consumer Familiarity with Health Terms: Using Gender as a Proxy for Background Knowledge about Gender-Specific Illnesses ISBMDA 2006: 472-481
Divita G, Browne AC, Loanne R. dTagger: A POS Tagger AMIA Annu Symp Proc. 2006:200-3
Lin J, Demner-Fushman D. The Role of Knowledge in Conceptual Retrieval: A Study in the Domain of Clinical Medicine Proc SIGIR 2006, pages 99-106, August 2006, Seattle, Washington
Demner-Fushman D, Lin J. Situated Question Answering in the Clinical Domain: Selecting the Best Drug Treatment for Diseases Proc COLING/ACL 2006 Workshop on Task-Focused Summarization and Question Answering, July 2006, Sydney, Australia
Humphrey SM, Lu C, Rogers W, Browne AC. Journal descriptor indexing tool for categorizing text according to discipline or semantic type. AMIA Annu Symp Proc. 2006:960.
Zhang S, Mork P, Bodenreider O. Lessons Learned from Aligning two Representations of Anatomy Proc of the First International Workshop on Formal Biomedical Knowledge Representation (KR-MED 2004); 2004. p. 102-108.
Browne AC, Divita Guy, Aronson AR, McCray AT. Language and Vocabulary Tools AMIA Annu Symp Proc. 2003:798.
McCray AT, Browne AC, Bodenreider O. The Lexical Properties of the Gene Ontology (GO) Proc AMIA Symp. 2002;:504-8.

Pages