saxon:tokenize()

tokenize(string-1, string-2?)

The first argument is converted to a string and is treated as a list of separated tokens. If the second argument is present, any character in string-2 is taken as a delimiter character, and any sequence of delimiter characters is taken as a token separator. If the second argument is omitted, any sequence of whitespace is taken as a token separator: or to put it another way, the default for string-2 is '	

 '.
A new sequence is constructed containing one string for each token; if the string is empty or contains a separator only then the result will be empty. For example tokenize("a cup of tea") generates a sequence of four strings: ( "a", "cup", "of", "tea").

This function is deprecated, since the same effect can be achieved using the tokenize() function in the standard library.

Expand

Up  Next