gov.nih.nlm.nls.lvg.Util

Class StripToken

java.lang.Object

gov.nih.nlm.nls.lvg.Util.StripToken

public class StripToken
extends java.lang.Object

This class tokenizes a string into a collection of StrTokenObject.

History:

Version:

V-2026

Author:

NLM NLS Development Team

See Also:

StrTokenObject, Design document

Constructor Summary

Constructors

Constructor and Description
StripToken()

Method Summary

Modifier and Type	Method and Description
static java.util.Vector<StrTokenObject>	CleanUpEnd(java.util.Vector<StrTokenObject> list) Remove the end elements of a tokenized collection of StrTokenObject if it is a character in the bad end list string defined in StrTokenObject.
static java.util.Vector<StrTokenObject>	CleanUpEnd(java.util.Vector<StrTokenObject> list, java.lang.String badEndList) Remove the end elements of a tokenized collection of StrTokenObject if it is a character in the bad end list string.
static java.util.Vector<StrTokenObject>	CleanUpToken(java.util.Vector<StrTokenObject> list) Clean up elements in the tokenized Vector.
static java.util.Vector<StrTokenObject>	CleanUpToken(java.util.Vector<StrTokenObject> list, java.lang.String confList) Clean up elements in the tokenized Vector.
static java.lang.String	ComposeString(java.util.Vector<StrTokenObject> list) Compose a string from a list of tokenized StrTokenObjects.
static void	main(java.lang.String[] args) Test driver for this class
static java.util.Vector<StrTokenObject>	Tokenize(java.lang.String inStr) Tokenize a string and assigned token type to each token.
static java.util.Vector<StrTokenObject>	Tokenize(java.lang.String inStr, java.lang.String restoreD, java.lang.String strippingD, java.lang.String strippableD) Tokenize a string and assigned token type to each token.
static java.util.Vector<StrTokenObject>	Tokenize(java.lang.String inStr, java.lang.String restoreD, java.lang.String strippingD, java.lang.String strippableD, java.lang.String spaceD) Tokenize a string and assigned token type to each token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StripToken

public StripToken()

Method Detail

Tokenize

public static java.util.Vector<StrTokenObject> Tokenize(java.lang.String inStr)

Tokenize a string and assigned token type to each token. This method uses the default delimiters defined in StrTokenObject. They are briefly described as follows:

• restoreD: A String contains delimiters that will be restored after tokenized
• strippingD: A String contains delimiters that will be restored and it's previous token will be stripped out if they are in conflict list or their type are stripped.
• strippableD: A String contains delimiters that will be restored (stripped) if it's previous token is not stripped out (stripped).
• confList: char that can not be used before a delim of strippingD
• spaceD: space deliminator for nor-restore

No overlap is allowed between restoreD, strippingD, and strippableD.

Parameters:

inStr - a String to be tokenized

Returns:

a collection (Vector) of StrTokenObject of tokenized elements

Tokenize

public static java.util.Vector<StrTokenObject> Tokenize(java.lang.String inStr,
                                                        java.lang.String restoreD,
                                                        java.lang.String strippingD,
                                                        java.lang.String strippableD,
                                                        java.lang.String spaceD)

Tokenize a string and assigned token type to each token.

Parameters:

inStr - a String to be tokenized

restoreD - a String contains delimiters that will be restored after tokenized

strippingD - a String contains delimiters that it's previous token will be stripped out if they are in conflict list or their type are stripped.

strippableD - a String contains delimiters that will be restored if it's previous token is not stripped out.

spaceD - a String contains space delimiters.

Returns:

a collection (Vector) of StrTokenObject of tokenized elements

Tokenize

public static java.util.Vector<StrTokenObject> Tokenize(java.lang.String inStr,
                                                        java.lang.String restoreD,
                                                        java.lang.String strippingD,
                                                        java.lang.String strippableD)

Tokenize a string and assigned token type to each token.

Parameters:

inStr - a String to be tokenized

restoreD - a String contains delimiters that will be restored after tokenized

strippingD - a String contains delimiters that it's previous token will be stripped out if they are in conflict list or their type are stripped.

strippableD - a String contains delimiters that will be restored if it's previous token is not stripped out.

Returns:

a collection (Vector) of StrTokenObject - tokenized elements

CleanUpToken

public static java.util.Vector<StrTokenObject> CleanUpToken(java.util.Vector<StrTokenObject> list)

Clean up elements in the tokenized Vector. This method includes lots of smart operations while restoring tokens into a string. This is used when strip with punctuations. In such case, some punctuations need to be stripped automatically. This method uses the defualt conflict string " -,:;".

Parameters:

list - a tokenized vector.

Returns:

a vector with clean-up tokenized elements.

See Also:

StrTokenObject

CleanUpToken

public static java.util.Vector<StrTokenObject> CleanUpToken(java.util.Vector<StrTokenObject> list,
                                                            java.lang.String confList)

Clean up elements in the tokenized Vector. This method includes lots of smart operations while restoring tokens into a string. This is used when strip with punctuations. In such case, some punctuations need to be stripped automatically.

Parameters:

list - a tokenized vector.

confList - a string contians all conflict characters. A conflict character is a character need to be stripped if the following token is stripping type.

Returns:

a vector with clean-up tokenized elements.

CleanUpEnd

public static java.util.Vector<StrTokenObject> CleanUpEnd(java.util.Vector<StrTokenObject> list)

Remove the end elements of a tokenized collection of StrTokenObject if it is a character in the bad end list string defined in StrTokenObject.

Parameters:

list - a vector of tokenized StrTokenObjects that will be removed end element (character) if it is in a badEndList.

Returns:

a collection (Vector) of StrTokenObject - tokenized StrTokenObject without any bad end character at the end.

See Also:

StrTokenObject

CleanUpEnd

public static java.util.Vector<StrTokenObject> CleanUpEnd(java.util.Vector<StrTokenObject> list,
                                                          java.lang.String badEndList)

Remove the end elements of a tokenized collection of StrTokenObject if it is a character in the bad end list string.

Parameters:

list - a vector of tokenized StrTokenObjects that will be removed end element (character) if it is in a badEndList.

badEndList - a String that contains all bad end characters. The default value for badEndList is " -,:;".

Returns:

a vector of tokenized StrTokenObject without any bad end character at the end.

ComposeString

public static java.lang.String ComposeString(java.util.Vector<StrTokenObject> list)

Compose a string from a list of tokenized StrTokenObjects.

Parameters:

list - a collection of StrTokenObject

Returns:

a string that is composed from the input list.

main

public static void main(java.lang.String[] args)

Test driver for this class

Parameters:

args - arguments