gov.nih.nlm.nls.lvg.Flows

Class ToUnicodeCoreNorm

java.lang.Object

gov.nih.nlm.nls.lvg.Flows.Transformation

gov.nih.nlm.nls.lvg.Flows.ToUnicodeCoreNorm

All Implemented Interfaces:

java.lang.Cloneable

public class ToUnicodeCoreNorm
extends Transformation
implements java.lang.Cloneable

This class normalize Unicode characters by recursively performing

• Map Unicode symbols and punctuation to ASCII
• Map Unicode characters to ASCII
• Split ligatures
• Strip diacritics

until the character is ASCII or no further normalized result are obtained. This is the core normalization of covnert Unicode to ASCII. Please note that the results might inlcude non-ASCII characters. StripMapUnicode or getUnicodeName flows are followed to complete the ASCII Normalization.

Four mapping tables are used in this flow:

• $LVG/data/Unicode/symbolMap.data.
• $LVG/data/Unicode/unicodeMap.data.
• $LVG/data/Unicode/ligaturemMap.data.
• $LVG/data/Unicode/diacriticMap.data.

History:

Version:

V-2026

Author:

NLM NLS Development Team

See Also:

Design Document , Unicode Core Norm

Field Summary

Fields inherited from class gov.nih.nlm.nls.lvg.Flows.Transformation

NO_MUTATE_INFO, UPDATE

Constructor Summary

Constructors

Constructor and Description
ToUnicodeCoreNorm()

Method Summary

Modifier and Type	Method and Description
static java.lang.String	GetCoreNormStr(java.lang.String inStr, java.util.Hashtable<java.lang.Character,java.lang.String> symbolMap, java.util.Hashtable<java.lang.Character,java.lang.String> unicodeMap, java.util.Hashtable<java.lang.Character,java.lang.String> ligatureMap, java.util.Hashtable<java.lang.Character,java.lang.Character> diacriticMap) Get unicode synonym
static void	main(java.lang.String[] args) A unit test driver for this flow component.
static java.util.Vector<LexItem>	Mutate(LexItem in, java.util.Hashtable<java.lang.Character,java.lang.String> symbolMap, java.util.Hashtable<java.lang.Character,java.lang.String> unicodeMap, java.util.Hashtable<java.lang.Character,java.lang.String> ligatureMap, java.util.Hashtable<java.lang.Character,java.lang.Character> diacriticMap, boolean detailsFlag, boolean mutateFlag) Performs the mutation of this flow component.

Methods inherited from class gov.nih.nlm.nls.lvg.Flows.Transformation

GetTestStr, PrintResult, PrintResults, UpdateLexItem, UpdateLexItem, UpdateLexItem

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ToUnicodeCoreNorm

public ToUnicodeCoreNorm()

Method Detail

Mutate

public static java.util.Vector<LexItem> Mutate(LexItem in,
                                               java.util.Hashtable<java.lang.Character,java.lang.String> symbolMap,
                                               java.util.Hashtable<java.lang.Character,java.lang.String> unicodeMap,
                                               java.util.Hashtable<java.lang.Character,java.lang.String> ligatureMap,
                                               java.util.Hashtable<java.lang.Character,java.lang.Character> diacriticMap,
                                               boolean detailsFlag,
                                               boolean mutateFlag)

Performs the mutation of this flow component.

Parameters:

in - a LexItem as the input for this flow component

symbolMap - a hash table contain the unicode symbols mapping

unicodeMap - a hash table contain the unicode mapping

ligatureMap - a hash table contain the ligatures mapping

diacriticMap - a hash table contain the diacritics mapping

detailsFlag - a boolean flag for processing details information

mutateFlag - a boolean flag for processing mutate information

Returns:

the results from this flow component - a collection (Vector) of LexItems

GetCoreNormStr

public static java.lang.String GetCoreNormStr(java.lang.String inStr,
                                              java.util.Hashtable<java.lang.Character,java.lang.String> symbolMap,
                                              java.util.Hashtable<java.lang.Character,java.lang.String> unicodeMap,
                                              java.util.Hashtable<java.lang.Character,java.lang.String> ligatureMap,
                                              java.util.Hashtable<java.lang.Character,java.lang.Character> diacriticMap)

Get unicode synonym

Parameters:

inStr - an input string

symbolMap - a hash table contain the unicode symbols mapping

unicodeMap - a hash table contain the unicode mapping

ligatureMap - a hash table contain the ligatures mapping

diacriticMap - a hash table contain the diacritics mapping

Returns:

the base of unicode synonym

main

public static void main(java.lang.String[] args)

A unit test driver for this flow component.

Parameters:

args - arguments