triana.types.util

Class StringSplitter

java.lang.Object

java.util.AbstractCollection<E>

java.util.AbstractList<E>

java.util.ArrayList

triana.types.util.StringVector

triana.types.util.StringSplitter

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.lang.Iterable, java.util.Collection, java.util.List, java.util.RandomAccess

public class StringSplitter
extends StringVector

StringSplitter is a utility to divide strings into substrings (called tokens) by recognizing separators (called delimiters). It provides an Iterator called tokenizer that is a generalization of StringTokenizer. Whereas StringTokenizer allows only single-character delimiters, StringSplitter allows any string to be defined as a delimiter, and it adds pre-defined delimiter sets called: whitespace, newline, word, paragraph, character, operator. There can be any of a number of different delimiters. Unlike StringTokenizer, the splitting is done when the object is instantiated, so that the delimiters are fixed once and for all; it is not possible to change delimiter sets for different parts of the string.

Constructors are provided to allow delimiters to be specified as an array of strings, as a StringVector, as a single string (whose characters are assumed to be delimiters, as in StringTokenizer), or as a keyword for some pre-defined delimiter sets: whitespace (the default here and in StringTokenizer), newline, word (uses any punctuation mark or whitespace as a delimiter), paragraph (splits on blank lines or newlines followed by a tab or space), character (no delimiters: each character is a token), and operator (splits on arithmetic operators).

A boolean argument allows the user to choose whether to include the delimiters as tokens. The default is false: no delimiters.

Various extensions for returning the tokens in a convenient form are provided. Although the Iterator is provided for compatibility with StringTokenizer, tokens can be more flexibly accessed by using other methods that work by the index of the token, since StringSplitter is an extension of the ArrayList class. Therefore all the methods of the ArrayList class are available in StringSplitter.

A test is also provided to see if the input string begins with a delimiter or not; this is useful if delimiter strings are returned as tokens, since one may want to know if the odd-numbered tokens or the even ones are delimiters. Another test is provided to see if the input string ends in a delimiter.

See Also:

Serialized Form

Field Summary

Fields inherited from class java.util.AbstractList

modCount

Constructor Summary

Constructors

Constructor and Description
StringSplitter() Constructs an empty StringSpitter with fixed storage capacity and capacityIncrement.
StringSplitter(java.lang.String str) Constructs a StringSplitter by tokenizing the specified input string on white space.
StringSplitter(java.lang.String str, boolean returnTokens) Constructs a StringSplitter by tokenizing the specified input string on white space.
StringSplitter(java.lang.String str, java.lang.String instruction) Constructs a StringSplitter by tokenizing the first String argument according to the instructions given by the second String argument.
StringSplitter(java.lang.String str, java.lang.String[] delims) Constructs a StringSplitter by tokenizing the String first argument on the delimiters given by the String array second argument.
StringSplitter(java.lang.String str, java.lang.String[] delims, boolean returnTokens) Constructs a StringSplitter by tokenizing the String argument on any of the delimiter strings in the String Array argument.
StringSplitter(java.lang.String str, java.lang.String instruction, boolean returnTokens) Constructs a StringSplitter by tokenizing the first String argument in a manner given by the second String argument.
StringSplitter(java.lang.String str, StringVector delimVector) Constructs a StringSplitter by tokenizing the String first argument on the delimiters given by the StringVector second argument.
StringSplitter(java.lang.String str, StringVector delimVector, boolean returnTokens) Constructs a StringSplitter by tokenizing the String first argument on the delimiters given by the StringVector second argument.

Method Summary

Methods

Modifier and Type	Method and Description
boolean	beginsWithDelimiter()
boolean	containsTokens()
int	countTokens()
boolean	endsWithDelimiter()
java.lang.String	firstNonDelimiterElement()
java.lang.String	firstNonDelimiterToken()
java.lang.String	lastNonDelimiterElement()
java.lang.String	lastNonDelimiterToken()
java.util.Iterator	tokenizer()
java.lang.String[]	toStringArray()

Methods inherited from class triana.types.util.StringVector

addElement, append, at, copy, copyInto, elementAt, elements, first, indexOf, last, removeAllElements, removeElementAt, setElementAt, sort, toAString, toDoubles, toInts, toNewLineString

Methods inherited from class java.util.ArrayList

add, add, addAll, addAll, clear, clone, contains, ensureCapacity, get, indexOf, isEmpty, iterator, lastIndexOf, listIterator, listIterator, remove, remove, removeAll, removeRange, retainAll, set, size, subList, toArray, toArray, trimToSize

Methods inherited from class java.util.AbstractList

equals, hashCode

Methods inherited from class java.util.AbstractCollection

containsAll, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.List

containsAll, equals, hashCode

Constructor Detail

StringSplitter

public StringSplitter()

Constructs an empty StringSpitter with fixed storage capacity and capacityIncrement.

StringSplitter

public StringSplitter(java.lang.String str,
              java.lang.String[] delims,
              boolean returnTokens)

Constructs a StringSplitter by tokenizing the String argument on any of the delimiter strings in the String Array argument.

If the boolean argument is true, delimiter sequences will also be returned as tokens.

Parameters:

str - the string to be split into tokens on whitespace

delims - the array of strings that act as delimiters between tokens

returnTokens - a boolean to decide if delimiters are included as tokens

StringSplitter

public StringSplitter(java.lang.String str,
              java.lang.String instruction,
              boolean returnTokens)

Constructs a StringSplitter by tokenizing the first String argument in a manner given by the second String argument. If the second argument is one of a number of special strings, then specific delimiters are assumed: "whitespace" means split on any whitespace character; "paragraph" means look for "\n\n" (paragraphs separated by a blank line), "\n\t" (paragraphs denoted by a tab indentation) or "\n " (paragraphs denoted by a newline and at least one space before text begins); "word" means use whitespace and punctuation marks as delimiters (the punctuation marks used are the single characters .,;:?!"`()[]{}<> and the two-character combinations "--" and "' ", so hyphenated or apostrophe'd words are not split); "character" means each character is a separate token; and "operator" means split on arithmetic symbols "+", "-", "*", and "/". If the second argument is not recognized, then it is assumed to be a string of delimiter characters, and splitting is performed on any of its characters.

The boolean argument determines whether sequences of delimiters are returned as tokens too.

Parameters:

str - the string to be split into tokens on whitespace

instruction - the String that gives the rule for splitting

returnTokens - a boolean to decide if delimiters are included as tokens

StringSplitter

public StringSplitter(java.lang.String str,
              boolean returnTokens)

Constructs a StringSplitter by tokenizing the specified input string on white space. This is the default if the second string-type argument is omitted. Include the option to return delimiter strings as tokens.

Parameters:

str - the string to be split into tokens on whitespace

returnTokens - a boolean to decide if delimiters are included as tokens

StringSplitter

public StringSplitter(java.lang.String str)

Constructs a StringSplitter by tokenizing the specified input string on white space. This is the default if the second string-type argument is omitted. Since the boolean argument is also absent, assume no delimiters are returned.

Parameters:

str - the string to be split into tokens on whitespace

StringSplitter

public StringSplitter(java.lang.String str,
              java.lang.String[] delims)

Constructs a StringSplitter by tokenizing the String first argument on the delimiters given by the String array second argument. Since the boolean argument is omitted, assume no delimiters are returned.

Parameters:

str - the string to be split into tokens on whitespace

delims - the array of strings that act as delimiters between tokens

StringSplitter

public StringSplitter(java.lang.String str,
              java.lang.String instruction)

Constructs a StringSplitter by tokenizing the first String argument according to the instructions given by the second String argument. Since the boolean argument is omitted, assume no delimiters are returned.

Parameters:

str - the String to be split into tokens on whitespace

instruction - the String that gives the rule for splitting

StringSplitter

public StringSplitter(java.lang.String str,
              StringVector delimVector,
              boolean returnTokens)

Constructs a StringSplitter by tokenizing the String first argument on the delimiters given by the StringVector second argument.

The boolean argument determines whether sequences of delimiters are returned as tokens too.

Parameters:

str - the string to be split into tokens on whitespace

delimVector - the array of strings that act as delimiters between tokens

returnTokens - a boolean to decide if delimiters are included as tokens

StringSplitter

public StringSplitter(java.lang.String str,
              StringVector delimVector)

Constructs a StringSplitter by tokenizing the String first argument on the delimiters given by the StringVector second argument. Since the boolean argument is absent, the default of not returning delimiters is assumed.

Parameters:

str - the string to be split into tokens on whitespace

delimVector - the array of strings that act as delimiters between tokens

Method Detail

tokenizer

public final java.util.Iterator tokenizer()

Returns:

an Iterator that acts like StringTokenizer.

countTokens

public final int countTokens()

Returns:

the number of tokens generated by splitting the string.

containsTokens

public final boolean containsTokens()

Returns:

true if delimiters were returned as tokens.

beginsWithDelimiter

public final boolean beginsWithDelimiter()

Returns:

true if the input string begins with a delimiter. (Returns answer independently of whether delimiters will be returned as tokens.)

endsWithDelimiter

public final boolean endsWithDelimiter()

Returns:

true if the input string ends with a delimiter. (Returns answer independently of whether delimiters will be returned as tokens.)

firstNonDelimiterElement

public final java.lang.String firstNonDelimiterElement()

Returns:

the first non-delimiter token (first token if delimiters are not made into tokens).

lastNonDelimiterElement

public final java.lang.String lastNonDelimiterElement()

Returns:

the last non-delimiter token (last token if delimiters are not made into tokens).

firstNonDelimiterToken

public final java.lang.String firstNonDelimiterToken()

Returns:

the first non-delimiter token (first token if delimiters are not made into tokens). Identical to firstNonDelimiterElement().

lastNonDelimiterToken

public final java.lang.String lastNonDelimiterToken()

Returns:

the last non-delimiter token (last token if delimiters are not made into tokens). Identical to lastNonDelimiterElement().

toStringArray

public final java.lang.String[] toStringArray()

Returns:

a String array representation of the tokens. /todo this causes a classcast exception