au.id.jericho.lib.html
Class Segment

java.lang.Object
  extended by Segment
All Implemented Interfaces:
java.lang.CharSequence, java.lang.Comparable
Direct Known Subclasses:
Attribute, CharacterReference, Element, FormControl, au.id.jericho.lib.html.nodoc.SequentialListSegment, Source, Tag

public class Segment
extends java.lang.Object
implements java.lang.Comparable, java.lang.CharSequence

Represents a segment of a Source document.

Many of the tag search methods are defined in this class.

The span of a segment is defined by the combination of its begin and end character positions.


Constructor Summary
Segment(Source source, int begin, int end)
          Constructs a new Segment within the specified source document with the specified begin and end character positions.
 
Method Summary
 char charAt(int index)
          Returns the character at the specified index.
 int compareTo(java.lang.Object o)
          Compares this Segment object to another object.
 boolean encloses(int pos)
          Indicates whether this segment encloses the specified character position in the source document.
 boolean encloses(Segment segment)
          Indicates whether this Segment encloses the specified Segment.
 boolean equals(java.lang.Object object)
          Compares the specified object with this Segment for equality.
 java.lang.String extractText()
          Deprecated. Use getTextExtractor().toString() instead.
 java.lang.String extractText(boolean includeAttributes)
          Deprecated. Use getTextExtractor().setIncludeAttributes(includeAttributes).toString() instead.
 java.util.List findAllCharacterReferences()
          Returns a list of all CharacterReference objects that are enclosed by this segment.
 java.util.List findAllElements()
          Returns a list of all Element objects that are enclosed by this segment.
 java.util.List findAllElements(StartTagType startTagType)
          Returns a list of all Element objects with start tags of the specified type that are enclosed by this segment.
 java.util.List findAllElements(java.lang.String name)
          Returns a list of all Element objects with the specified name that are enclosed by this segment.
 java.util.List findAllElements(java.lang.String attributeName, java.lang.String value, boolean valueCaseSensitive)
          Returns a list of all Element objects with the specified attribute name/value pair that are enclosed by this segment.
 java.util.List findAllStartTags()
          Returns a list of all StartTag objects that are enclosed by this segment.
 java.util.List findAllStartTags(java.lang.String name)
          Returns a list of all normal StartTag objects with the specified name that are enclosed by this segment.
 java.util.List findAllStartTags(java.lang.String attributeName, java.lang.String value, boolean valueCaseSensitive)
          Returns a list of all StartTag objects with the specified attribute name/value pair that are enclosed by this segment.
 java.util.List findAllTags()
          Returns a list of all Tag objects that are enclosed by this segment.
 java.util.List findAllTags(TagType tagType)
          Returns a list of all Tag objects of the specified type that are enclosed by this segment.
 java.util.List findFormControls()
          Returns a list of the FormControl objects that are enclosed by this segment.
 FormFields findFormFields()
          Returns the FormFields object representing all form fields that are enclosed by this segment.
 int getBegin()
          Returns the character position in the Source document at which this segment begins, inclusive.
 java.util.List getChildElements()
          Returns a list of the immediate children of this segment in the document element hierarchy.
 java.lang.String getDebugInfo()
          Returns a string representation of this object useful for debugging purposes.
 int getEnd()
          Returns the character position in the Source document immediately after the end of this segment.
 java.util.Iterator getNodeIterator()
          Returns an iterator over every tag and text segment contained within this segment.
 Renderer getRenderer()
          Performs a simple rendering of the HTML markup in this segment into text.
 Source getSource()
          Returns the Source document containing this segment.
 TextExtractor getTextExtractor()
          Extracts the textual content from the HTML markup of this segment.
 int hashCode()
          Returns a hash code value for the segment.
 void ignoreWhenParsing()
          Causes the this segment to be ignored when parsing.
 boolean isWhiteSpace()
          Indicates whether this segment consists entirely of white space.
static boolean isWhiteSpace(char ch)
          Indicates whether the specified character is white space.
 int length()
          Returns the length of the segment.
 Attributes parseAttributes()
          Parses any Attributes within this segment.
 java.lang.CharSequence subSequence(int beginIndex, int endIndex)
          Returns a new character sequence that is a subsequence of this sequence.
 java.lang.String toString()
          Returns the source text of this segment as a String.
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

Segment

public Segment(Source source,
               int begin,
               int end)
Constructs a new Segment within the specified source document with the specified begin and end character positions.

Parameters:
source - the Source document, must not be null.
begin - the character position in the source where this segment begins, inclusive.
end - the character position in the source where this segment ends, exclusive.
Method Detail

getSource

public final Source getSource()
Returns the Source document containing this segment.

Returns:
the Source document containing this segment.

getBegin

public final int getBegin()
Returns the character position in the Source document at which this segment begins, inclusive.

Returns:
the character position in the Source document at which this segment begins, inclusive.

getEnd

public final int getEnd()
Returns the character position in the Source document immediately after the end of this segment.

The character at the position specified by this property is not included in the segment.

Returns:
the character position in the Source document immediately after the end of this segment.

equals

public final boolean equals(java.lang.Object object)
Compares the specified object with this Segment for equality.

Returns true if and only if the specified object is also a Segment, and both segments have the same Source, and the same begin and end positions.

Overrides:
equals in class java.lang.Object
Parameters:
object - the object to be compared for equality with this Segment.
Returns:
true if the specified object is equal to this Segment, otherwise false.

hashCode

public int hashCode()
Returns a hash code value for the segment.

The current implementation returns the sum of the begin and end positions, although this is not guaranteed in future versions.

Overrides:
hashCode in class java.lang.Object
Returns:
a hash code value for the segment.

length

public final int length()
Returns the length of the segment. This is defined as the number of characters between the begin and end positions.

Specified by:
length in interface java.lang.CharSequence
Returns:
the length of the segment.

encloses

public final boolean encloses(Segment segment)
Indicates whether this Segment encloses the specified Segment.

This is the case if getBegin()<=segment.getBegin() && getEnd()>=segment.getEnd().

Parameters:
segment - the segment to be tested for being enclosed by this segment.
Returns:
true if this Segment encloses the specified Segment, otherwise false.

encloses

public final boolean encloses(int pos)
Indicates whether this segment encloses the specified character position in the source document.

This is the case if getBegin() <= pos < getEnd().

Parameters:
pos - the position in the Source document.
Returns:
true if this segment encloses the specified character position in the source document, otherwise false.

toString

public java.lang.String toString()
Returns the source text of this segment as a String.

The returned String is newly created with every call to this method, unless this segment is itself an instance of Source.

Note that before version 2.0 this returned a representation of this object useful for debugging purposes, which can now be obtained via the getDebugInfo() method.

Specified by:
toString in interface java.lang.CharSequence
Overrides:
toString in class java.lang.Object
Returns:
the source text of this segment as a String.

getRenderer

public Renderer getRenderer()
Performs a simple rendering of the HTML markup in this segment into text.

The output can be configured by setting any number of properties on the returned Renderer instance before obtaining its output.

Returns:
an instance of Renderer based on this segment.
See Also:
getTextExtractor()

getTextExtractor

public TextExtractor getTextExtractor()
Extracts the textual content from the HTML markup of this segment.

The output can be configured by setting properties on the returned TextExtractor instance before obtaining its output.

Returns:
an instance of TextExtractor based on this segment.
See Also:
getRenderer()

getNodeIterator

public java.util.Iterator getNodeIterator()
Returns an iterator over every tag and text segment contained within this segment.

Every tag found in the findAllTags() list is included in this iterator, including all server tags.

Segments of the document between the tags are also included, resulting in a sequential walk-through of every "node" in this segment, where a node is either a tag or a segment of text. The end position of each segment should correspond with the begin position of the subsequent segment, unless any of the tags are enclosed by other tags, which is common when server tags are present.

The CharacterReference.decodeCollapseWhiteSpace(CharSequence) method can be used to retrieve the text from each text segment.

Example:

The following code demonstrates the typical usage of this method.

 for (Iterator nodeIterator=segment.getNoteIterator(); nodeIterator.hasNext();) {
   Segment nodeSegment=(Segment)nodeIterator.next();
   if (nodeSegment instanceof Tag) {
     Tag tag=(Tag)nodeSegment;
     if (tag.getTagType().isServerTag()) continue; // ignore server tags
     // Process the tag (just output it in this example):
     System.out.println(tag.tidy());
   } else {
     // Segment is a text segment.
     // Process the text segment (just output its text in this example):
     String text=CharacterReference.decodeCollapseWhiteSpace(nodeSegment);
     System.out.println(text);
   }
 }

Returns:
an iterator over every tag and text segment contained within this segment.

findAllTags

public java.util.List findAllTags()
Returns a list of all Tag objects that are enclosed by this segment.

The Source.fullSequentialParse() method should be called after construction of the Source object if this method is to be used on a large proportion of the source. It is called automatically if this method is called on the Source object itself.

See the Tag class documentation for more details about the behaviour of this method.

Returns:
a list of all Tag objects that are enclosed by this segment.

findAllTags

public java.util.List findAllTags(TagType tagType)
Returns a list of all Tag objects of the specified type that are enclosed by this segment.

See the Tag class documentation for more details about the behaviour of this method.

Specifying a null argument to the tagType parameter is equivalent to findAllTags().

Parameters:
tagType - the type of tags to find.
Returns:
a list of all Tag objects of the specified type that are enclosed by this segment.

findAllStartTags

public java.util.List findAllStartTags()
Returns a list of all StartTag objects that are enclosed by this segment.

The Source.fullSequentialParse() method should be called after construction of the Source object if this method is to be used on a large proportion of the source. It is called automatically if this method is called on the Source object itself.

See the Tag class documentation for more details about the behaviour of this method.

Returns:
a list of all StartTag objects that are enclosed by this segment.

findAllStartTags

public java.util.List findAllStartTags(java.lang.String name)
Returns a list of all normal StartTag objects with the specified name that are enclosed by this segment.

See the Tag class documentation for more details about the behaviour of this method.

Specifying a null argument to the name parameter is equivalent to findAllStartTags().

This method also returns unregistered tags if the specified name is not a valid XML tag name.

Parameters:
name - the name of the start tags to find.
Returns:
a list of all normal StartTag objects with the specified name that are enclosed by this segment.

findAllStartTags

public java.util.List findAllStartTags(java.lang.String attributeName,
                                       java.lang.String value,
                                       boolean valueCaseSensitive)
Returns a list of all StartTag objects with the specified attribute name/value pair that are enclosed by this segment.

See the Tag class documentation for more details about the behaviour of this method.

Parameters:
attributeName - the attribute name (case insensitive) to search for, must not be null.
value - the value of the specified attribute to search for, must not be null.
valueCaseSensitive - specifies whether the attribute value matching is case sensitive.
Returns:
a list of all StartTag objects with the specified attribute name/value pair that are enclosed by this segment.

getChildElements

public java.util.List getChildElements()
Returns a list of the immediate children of this segment in the document element hierarchy.

The returned list may include an element that extends beyond the end of this segment, as long as it begins within this segment.

An element found at the start of this segment is included in the list. Note however that if this segment is an Element, the overriding Element.getChildElements() method is called instead, which only returns the children of the element.

Calling getChildElements() on an Element is usually more efficient than calling it on a Segment.

The objects in the list are all of type Element.

The Source.fullSequentialParse() method should be called after construction of the Source object if this method is to be used on a large proportion of the source. It is called automatically if this method is called on the Source object itself.

See the Source.getChildElements() method for more details.

Returns:
the a list of the immediate children of this segment in the document element hierarchy, guaranteed not null.
See Also:
Element.getParentElement()

findAllElements

public java.util.List findAllElements()
Returns a list of all Element objects that are enclosed by this segment.

The Source.fullSequentialParse() method should be called after construction of the Source object if this method is to be used on a large proportion of the source. It is called automatically if this method is called on the Source object itself.

The elements returned correspond exactly with the start tags returned in the findAllStartTags() method.

Returns:
a list of all Element objects that are enclosed by this segment.

findAllElements

public java.util.List findAllElements(java.lang.String name)
Returns a list of all Element objects with the specified name that are enclosed by this segment.

The elements returned correspond exactly with the start tags returned in the findAllStartTags(String name) method.

Specifying a null argument to the name parameter is equivalent to findAllElements().

This method also returns elements consisting of unregistered tags if the specified name is not a valid XML tag name.

Parameters:
name - the name of the elements to find.
Returns:
a list of all Element objects with the specified name that are enclosed by this segment.

findAllElements

public java.util.List findAllElements(StartTagType startTagType)
Returns a list of all Element objects with start tags of the specified type that are enclosed by this segment.

The elements returned correspond exactly with the start tags returned in the findAllTags(TagType) method.

Parameters:
startTagType - the type of start tags to find, must not be null.
Returns:
a list of all Element objects with start tags of the specified type that are enclosed by this segment.

findAllElements

public java.util.List findAllElements(java.lang.String attributeName,
                                      java.lang.String value,
                                      boolean valueCaseSensitive)
Returns a list of all Element objects with the specified attribute name/value pair that are enclosed by this segment.

The elements returned correspond exactly with the start tags returned in the findAllStartTags(String attributeName, String value, boolean valueCaseSensitive) method.

Parameters:
attributeName - the attribute name (case insensitive) to search for, must not be null.
value - the value of the specified attribute to search for, must not be null.
valueCaseSensitive - specifies whether the attribute value matching is case sensitive.
Returns:
a list of all Element objects with the specified attribute name/value pair that are enclosed by this segment.

findAllCharacterReferences

public java.util.List findAllCharacterReferences()
Returns a list of all CharacterReference objects that are enclosed by this segment.

Returns:
a list of all CharacterReference objects that are enclosed by this segment.

findFormControls

public java.util.List findFormControls()
Returns a list of the FormControl objects that are enclosed by this segment.

Returns:
a list of the FormControl objects that are enclosed by this segment.

findFormFields

public FormFields findFormFields()
Returns the FormFields object representing all form fields that are enclosed by this segment.

This is equivalent to new FormFields(findFormControls()).

Returns:
the FormFields object representing all form fields that are enclosed by this segment.
See Also:
findFormControls()

parseAttributes

public Attributes parseAttributes()
Parses any Attributes within this segment. This method is only used in the unusual situation where attributes exist outside of a start tag. The StartTag.getAttributes() method should be used in normal situations.

This is equivalent to source.parseAttributes(getBegin(),getEnd()).

Returns:
the Attributes within this segment, or null if too many errors occur while parsing.

ignoreWhenParsing

public void ignoreWhenParsing()
Causes the this segment to be ignored when parsing.

Ignored segments are treated as blank spaces by the parsing mechanism, but are included as normal text in all other functions.

This method was originally the only means of preventing server tags located inside normal tags from interfering with the parsing of the tags (such as where an attribute of a normal tag uses a server tag to dynamically set its value), as well as preventing non-server tags from being recognised inside server tags.

As of version 2.4 it is no longer necessary to use this method to ignore server tags located inside normal tags, as the attributes parser now automatically ignores any server tags.

As of version 2.5 it is no longer necessary to use this method to ignore the contents of SCRIPT elements, as the parser now does this automatically when performing a full sequential parse.

As of version 2.6 is is no longer necessary to use this method to ignore non-server tags inside server tags, as the parser now does this automatically when performing a full sequential parse.

This leaves only very few scenarios where calling this method still provides a significant benefit.

One such case is where XML-style server tags are used inside normal tags. Here is an example using an XML-style JSP tag:

<a href="<i18n:resource path="/Portal"/>?BACK=TRUE">back</a>
The first double-quote of "/Portal" will be interpreted as the end quote for the href attribute, as there is no way for the parser to recognise the il8n:resource element as a server tag. Such use of XML-style server tags inside normal tags is generally seen as bad practice, but it is nevertheless valid JSP. The only way to ensure that this library is able to parse the normal tag surrounding it is to find these server tags first and call the ignoreWhenParsing method to ignore them before parsing the rest of the document.

It is important to understand the difference between ignoring the segment when parsing and removing the segment completely. Any text inside a segment that is ignored when parsing is treated by most functions as content, and as such is included in the output of tools such as TextExtractor and Renderer.

To remove segments completely, create an OutputDocument and call its remove(Segment) or replaceWithSpaces(int begin, int end) method for each segment. Then create a new source document using new Source(outputDocument.toString()) and perform the desired operations on this new source object.

Calling this method after the Source.fullSequentialParse() method has been called is not permitted and throws an IllegalStateException.

Any tags appearing in this segment that are found before this method is called will remain in the tag cache, and so will continue to be found by the tag search methods. If this is undesirable, the Source.clearCache() method can be called to remove them from the cache. Calling the Source.fullSequentialParse() method after this method clears the cache automatically.

For best performance, this method should be called on all segments that need to be ignored without calling any of the tag search methods in between.

See Also:
Source.ignoreWhenParsing(Collection segments)

compareTo

public int compareTo(java.lang.Object o)
Compares this Segment object to another object.

If the argument is not a Segment, a ClassCastException is thrown.

A segment is considered to be before another segment if its begin position is earlier, or in the case that both segments begin at the same position, its end position is earlier.

Segments that begin and end at the same position are considered equal for the purposes of this comparison, even if they relate to different source documents.

Note: this class has a natural ordering that is inconsistent with equals. This means that this method may return zero in some cases where calling the equals(Object) method with the same argument returns false.

Specified by:
compareTo in interface java.lang.Comparable
Parameters:
o - the segment to be compared
Returns:
a negative integer, zero, or a positive integer as this segment is before, equal to, or after the specified segment.
Throws:
java.lang.ClassCastException - if the argument is not a Segment

isWhiteSpace

public final boolean isWhiteSpace()
Indicates whether this segment consists entirely of white space.

Returns:
true if this segment consists entirely of white space, otherwise false.

isWhiteSpace

public static final boolean isWhiteSpace(char ch)
Indicates whether the specified character is white space.

The HTML 4.01 specification section 9.1 specifies the following white space characters:

Despite the explicit inclusion of the zero-width space in the HTML specification, Microsoft IE6 does not recognise them as white space and renders them as an unprintable character (empty square). Even zero-width spaces included using the numeric character reference &#x200B; are rendered this way.

Parameters:
ch - the character to test.
Returns:
true if the specified character is white space, otherwise false.

getDebugInfo

public java.lang.String getDebugInfo()
Returns a string representation of this object useful for debugging purposes.

Returns:
a string representation of this object useful for debugging purposes.

charAt

public final char charAt(int index)
Returns the character at the specified index.

This is logically equivalent to toString().charAt(index) for valid argument values 0 <= index < length().

However because this implementation works directly on the underlying document source string, it should not be assumed that an IndexOutOfBoundsException is thrown for an invalid argument value.

Specified by:
charAt in interface java.lang.CharSequence
Parameters:
index - the index of the character.
Returns:
the character at the specified index.

subSequence

public final java.lang.CharSequence subSequence(int beginIndex,
                                                int endIndex)
Returns a new character sequence that is a subsequence of this sequence.

This is logically equivalent to toString().subSequence(beginIndex,endIndex) for valid values of beginIndex and endIndex.

However because this implementation works directly on the underlying document source string, it should not be assumed that an IndexOutOfBoundsException is thrown for invalid argument values as described in the String.subSequence(int,int) method.

Specified by:
subSequence in interface java.lang.CharSequence
Parameters:
beginIndex - the begin index, inclusive.
endIndex - the end index, exclusive.
Returns:
a new character sequence that is a subsequence of this sequence.

extractText

public java.lang.String extractText()
Deprecated. Use getTextExtractor().toString() instead.

Extracts the textual content from the HTML markup of this segment.

This method has been deprecated as of version 2.4 and replaced with the getTextExtractor() method.

Returns:
the textual content from the HTML markup of this segment.

extractText

public java.lang.String extractText(boolean includeAttributes)
Deprecated. Use getTextExtractor().setIncludeAttributes(includeAttributes).toString() instead.

Extracts the textual content from the HTML markup of this segment.

This method has been deprecated as of version 2.4 and replaced with the getTextExtractor() method.

Parameters:
includeAttributes - specifies whether the values of title, alt, label, and summary attributes are included in the output.
Returns:
the textual content from the HTML markup of this segment.