org.apache.commons.io.input
Class XmlStreamReader

java.lang.Object
  extended by java.io.Reader
      extended by org.apache.commons.io.input.XmlStreamReader
All Implemented Interfaces:
java.io.Closeable, java.lang.Readable

public class XmlStreamReader
extends java.io.Reader

Character stream that handles all the necessary Voodo to figure out the charset encoding of the XML document within the stream.

IMPORTANT: This class is not related in any way to the org.xml.sax.XMLReader. This one IS a character stream.

All this has to be done without consuming characters from the stream, if not the XML parser will not recognized the document as a valid XML. This is not 100% true, but it's close enough (UTF-8 BOM is not handled by all parsers right now, XmlStreamReader handles it and things work in all parsers).

The XmlStreamReader class handles the charset encoding of XML documents in Files, raw streams and HTTP streams by offering a wide set of constructors.

By default the charset encoding detection is lenient, the constructor with the lenient flag can be used for an script (following HTTP MIME and XML specifications). All this is nicely explained by Mark Pilgrim in his blog, Determining the character encoding of a feed.

Originally developed for ROME under Apache License 2.0.

Since:
Commons IO 2.0
Version:
$Id: XmlStreamReader.java 1052161 2010-12-23 03:12:09Z niallp $
Author:
Alejandro Abdelnur
See Also:
XmlStreamWriter

Field Summary
private static ByteOrderMark[] BOMS
           
private static int BUFFER_SIZE
           
private static java.util.regex.Pattern CHARSET_PATTERN
           
private  java.lang.String defaultEncoding
           
private static java.lang.String EBCDIC
           
private  java.lang.String encoding
           
static java.util.regex.Pattern ENCODING_PATTERN
           
private static java.lang.String HTTP_EX_1
           
private static java.lang.String HTTP_EX_2
           
private static java.lang.String HTTP_EX_3
           
private static java.lang.String RAW_EX_1
           
private static java.lang.String RAW_EX_2
           
private  java.io.Reader reader
           
private static java.lang.String US_ASCII
           
private static java.lang.String UTF_16
           
private static java.lang.String UTF_16BE
           
private static java.lang.String UTF_16LE
           
private static java.lang.String UTF_8
           
private static ByteOrderMark[] XML_GUESS_BYTES
           
 
Fields inherited from class java.io.Reader
lock
 
Constructor Summary
XmlStreamReader(java.io.File file)
          Creates a Reader for a File.
XmlStreamReader(java.io.InputStream is)
          Creates a Reader for a raw InputStream.
XmlStreamReader(java.io.InputStream is, boolean lenient)
          Creates a Reader for a raw InputStream.
XmlStreamReader(java.io.InputStream is, boolean lenient, java.lang.String defaultEncoding)
          Creates a Reader for a raw InputStream.
XmlStreamReader(java.io.InputStream is, java.lang.String httpContentType)
          Creates a Reader using an InputStream an the associated content-type header.
XmlStreamReader(java.io.InputStream is, java.lang.String httpContentType, boolean lenient)
          Creates a Reader using an InputStream an the associated content-type header.
XmlStreamReader(java.io.InputStream is, java.lang.String httpContentType, boolean lenient, java.lang.String defaultEncoding)
          Creates a Reader using an InputStream an the associated content-type header.
XmlStreamReader(java.net.URL url)
          Creates a Reader using the InputStream of a URL.
XmlStreamReader(java.net.URLConnection conn, java.lang.String defaultEncoding)
          Creates a Reader using the InputStream of a URLConnection.
 
Method Summary
(package private)  java.lang.String calculateHttpEncoding(java.lang.String httpContentType, java.lang.String bomEnc, java.lang.String xmlGuessEnc, java.lang.String xmlEnc, boolean lenient)
          Calculate the HTTP encoding.
(package private)  java.lang.String calculateRawEncoding(java.lang.String bomEnc, java.lang.String xmlGuessEnc, java.lang.String xmlEnc)
          Calculate the raw encoding.
 void close()
          Closes the XmlStreamReader stream.
private  java.lang.String doHttpStream(BOMInputStream bom, BOMInputStream pis, java.lang.String httpContentType, boolean lenient)
          Process a HTTP stream.
private  java.lang.String doLenientDetection(java.lang.String httpContentType, XmlStreamReaderException ex)
          Do lenient detection.
private  java.lang.String doRawStream(BOMInputStream bom, BOMInputStream pis, boolean lenient)
          Process the raw stream.
(package private) static java.lang.String getContentTypeEncoding(java.lang.String httpContentType)
          Returns charset parameter value, NULL if not present, NULL if httpContentType is NULL.
(package private) static java.lang.String getContentTypeMime(java.lang.String httpContentType)
          Returns MIME type or NULL if httpContentType is NULL.
 java.lang.String getDefaultEncoding()
          Returns the default encoding to use if none is set in HTTP content-type, XML prolog and the rules based on content-type are not adequate.
 java.lang.String getEncoding()
          Returns the charset encoding of the XmlStreamReader.
private static java.lang.String getXmlProlog(java.io.InputStream is, java.lang.String guessedEnc)
          Returns the encoding declared in the , NULL if none.
(package private) static boolean isAppXml(java.lang.String mime)
          Indicates if the MIME type belongs to the APPLICATION XML family.
(package private) static boolean isTextXml(java.lang.String mime)
          Indicates if the MIME type belongs to the TEXT XML family.
 int read(char[] buf, int offset, int len)
          Invokes the underlying reader's read(char[], int, int) method.
 
Methods inherited from class java.io.Reader
mark, markSupported, read, read, read, ready, reset, skip
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

BUFFER_SIZE

private static final int BUFFER_SIZE
See Also:
Constant Field Values

UTF_8

private static final java.lang.String UTF_8
See Also:
Constant Field Values

US_ASCII

private static final java.lang.String US_ASCII
See Also:
Constant Field Values

UTF_16BE

private static final java.lang.String UTF_16BE
See Also:
Constant Field Values

UTF_16LE

private static final java.lang.String UTF_16LE
See Also:
Constant Field Values

UTF_16

private static final java.lang.String UTF_16
See Also:
Constant Field Values

EBCDIC

private static final java.lang.String EBCDIC
See Also:
Constant Field Values

BOMS

private static final ByteOrderMark[] BOMS

XML_GUESS_BYTES

private static final ByteOrderMark[] XML_GUESS_BYTES

reader

private final java.io.Reader reader

encoding

private final java.lang.String encoding

defaultEncoding

private final java.lang.String defaultEncoding

CHARSET_PATTERN

private static final java.util.regex.Pattern CHARSET_PATTERN

ENCODING_PATTERN

public static final java.util.regex.Pattern ENCODING_PATTERN

RAW_EX_1

private static final java.lang.String RAW_EX_1
See Also:
Constant Field Values

RAW_EX_2

private static final java.lang.String RAW_EX_2
See Also:
Constant Field Values

HTTP_EX_1

private static final java.lang.String HTTP_EX_1
See Also:
Constant Field Values

HTTP_EX_2

private static final java.lang.String HTTP_EX_2
See Also:
Constant Field Values

HTTP_EX_3

private static final java.lang.String HTTP_EX_3
See Also:
Constant Field Values
Constructor Detail

XmlStreamReader

public XmlStreamReader(java.io.File file)
                throws java.io.IOException
Creates a Reader for a File.

It looks for the UTF-8 BOM first, if none sniffs the XML prolog charset, if this is also missing defaults to UTF-8.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters:
file - File to create a Reader from.
Throws:
java.io.IOException - thrown if there is a problem reading the file.

XmlStreamReader

public XmlStreamReader(java.io.InputStream is)
                throws java.io.IOException
Creates a Reader for a raw InputStream.

It follows the same logic used for files.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters:
is - InputStream to create a Reader from.
Throws:
java.io.IOException - thrown if there is a problem reading the stream.

XmlStreamReader

public XmlStreamReader(java.io.InputStream is,
                       boolean lenient)
                throws java.io.IOException
Creates a Reader for a raw InputStream.

It follows the same logic used for files.

If lenient detection is indicated and the detection above fails as per specifications it then attempts the following:

If the content type was 'text/html' it replaces it with 'text/xml' and tries the detection again.

Else if the XML prolog had a charset encoding that encoding is used.

Else if the content type had a charset encoding that encoding is used.

Else 'UTF-8' is used.

If lenient detection is indicated an XmlStreamReaderException is never thrown.

Parameters:
is - InputStream to create a Reader from.
lenient - indicates if the charset encoding detection should be relaxed.
Throws:
java.io.IOException - thrown if there is a problem reading the stream.
XmlStreamReaderException - thrown if the charset encoding could not be determined according to the specs.

XmlStreamReader

public XmlStreamReader(java.io.InputStream is,
                       boolean lenient,
                       java.lang.String defaultEncoding)
                throws java.io.IOException
Creates a Reader for a raw InputStream.

It follows the same logic used for files.

If lenient detection is indicated and the detection above fails as per specifications it then attempts the following:

If the content type was 'text/html' it replaces it with 'text/xml' and tries the detection again.

Else if the XML prolog had a charset encoding that encoding is used.

Else if the content type had a charset encoding that encoding is used.

Else 'UTF-8' is used.

If lenient detection is indicated an XmlStreamReaderException is never thrown.

Parameters:
is - InputStream to create a Reader from.
lenient - indicates if the charset encoding detection should be relaxed.
defaultEncoding - The default encoding
Throws:
java.io.IOException - thrown if there is a problem reading the stream.
XmlStreamReaderException - thrown if the charset encoding could not be determined according to the specs.

XmlStreamReader

public XmlStreamReader(java.net.URL url)
                throws java.io.IOException
Creates a Reader using the InputStream of a URL.

If the URL is not of type HTTP and there is not 'content-type' header in the fetched data it uses the same logic used for Files.

If the URL is a HTTP Url or there is a 'content-type' header in the fetched data it uses the same logic used for an InputStream with content-type.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters:
url - URL to create a Reader from.
Throws:
java.io.IOException - thrown if there is a problem reading the stream of the URL.

XmlStreamReader

public XmlStreamReader(java.net.URLConnection conn,
                       java.lang.String defaultEncoding)
                throws java.io.IOException
Creates a Reader using the InputStream of a URLConnection.

If the URLConnection is not of type HttpURLConnection and there is not 'content-type' header in the fetched data it uses the same logic used for files.

If the URLConnection is a HTTP Url or there is a 'content-type' header in the fetched data it uses the same logic used for an InputStream with content-type.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters:
conn - URLConnection to create a Reader from.
defaultEncoding - The default encoding
Throws:
java.io.IOException - thrown if there is a problem reading the stream of the URLConnection.

XmlStreamReader

public XmlStreamReader(java.io.InputStream is,
                       java.lang.String httpContentType)
                throws java.io.IOException
Creates a Reader using an InputStream an the associated content-type header.

First it checks if the stream has BOM. If there is not BOM checks the content-type encoding. If there is not content-type encoding checks the XML prolog encoding. If there is not XML prolog encoding uses the default encoding mandated by the content-type MIME type.

It does a lenient charset encoding detection, check the constructor with the lenient parameter for details.

Parameters:
is - InputStream to create the reader from.
httpContentType - content-type header to use for the resolution of the charset encoding.
Throws:
java.io.IOException - thrown if there is a problem reading the file.

XmlStreamReader

public XmlStreamReader(java.io.InputStream is,
                       java.lang.String httpContentType,
                       boolean lenient,
                       java.lang.String defaultEncoding)
                throws java.io.IOException
Creates a Reader using an InputStream an the associated content-type header. This constructor is lenient regarding the encoding detection.

First it checks if the stream has BOM. If there is not BOM checks the content-type encoding. If there is not content-type encoding checks the XML prolog encoding. If there is not XML prolog encoding uses the default encoding mandated by the content-type MIME type.

If lenient detection is indicated and the detection above fails as per specifications it then attempts the following:

If the content type was 'text/html' it replaces it with 'text/xml' and tries the detection again.

Else if the XML prolog had a charset encoding that encoding is used.

Else if the content type had a charset encoding that encoding is used.

Else 'UTF-8' is used.

If lenient detection is indicated an XmlStreamReaderException is never thrown.

Parameters:
is - InputStream to create the reader from.
httpContentType - content-type header to use for the resolution of the charset encoding.
lenient - indicates if the charset encoding detection should be relaxed.
defaultEncoding - The default encoding
Throws:
java.io.IOException - thrown if there is a problem reading the file.
XmlStreamReaderException - thrown if the charset encoding could not be determined according to the specs.

XmlStreamReader

public XmlStreamReader(java.io.InputStream is,
                       java.lang.String httpContentType,
                       boolean lenient)
                throws java.io.IOException
Creates a Reader using an InputStream an the associated content-type header. This constructor is lenient regarding the encoding detection.

First it checks if the stream has BOM. If there is not BOM checks the content-type encoding. If there is not content-type encoding checks the XML prolog encoding. If there is not XML prolog encoding uses the default encoding mandated by the content-type MIME type.

If lenient detection is indicated and the detection above fails as per specifications it then attempts the following:

If the content type was 'text/html' it replaces it with 'text/xml' and tries the detection again.

Else if the XML prolog had a charset encoding that encoding is used.

Else if the content type had a charset encoding that encoding is used.

Else 'UTF-8' is used.

If lenient detection is indicated an XmlStreamReaderException is never thrown.

Parameters:
is - InputStream to create the reader from.
httpContentType - content-type header to use for the resolution of the charset encoding.
lenient - indicates if the charset encoding detection should be relaxed.
Throws:
java.io.IOException - thrown if there is a problem reading the file.
XmlStreamReaderException - thrown if the charset encoding could not be determined according to the specs.
Method Detail

getDefaultEncoding

public java.lang.String getDefaultEncoding()
Returns the default encoding to use if none is set in HTTP content-type, XML prolog and the rules based on content-type are not adequate.

If it is NULL the content-type based rules are used.

Returns:
the default encoding to use.

getEncoding

public java.lang.String getEncoding()
Returns the charset encoding of the XmlStreamReader.

Returns:
charset encoding.

read

public int read(char[] buf,
                int offset,
                int len)
         throws java.io.IOException
Invokes the underlying reader's read(char[], int, int) method.

Specified by:
read in class java.io.Reader
Parameters:
buf - the buffer to read the characters into
offset - The start offset
len - The number of bytes to read
Returns:
the number of characters read or -1 if the end of stream
Throws:
java.io.IOException - if an I/O error occurs

close

public void close()
           throws java.io.IOException
Closes the XmlStreamReader stream.

Specified by:
close in interface java.io.Closeable
Specified by:
close in class java.io.Reader
Throws:
java.io.IOException - thrown if there was a problem closing the stream.

doRawStream

private java.lang.String doRawStream(BOMInputStream bom,
                                     BOMInputStream pis,
                                     boolean lenient)
                              throws java.io.IOException
Process the raw stream.

Parameters:
bom - BOMInputStream to detect byte order marks
pis - BOMInputStream to guess XML encoding
lenient - indicates if the charset encoding detection should be relaxed.
Returns:
the encoding to be used
Throws:
java.io.IOException - thrown if there is a problem reading the stream.

doHttpStream

private java.lang.String doHttpStream(BOMInputStream bom,
                                      BOMInputStream pis,
                                      java.lang.String httpContentType,
                                      boolean lenient)
                               throws java.io.IOException
Process a HTTP stream.

Parameters:
bom - BOMInputStream to detect byte order marks
pis - BOMInputStream to guess XML encoding
httpContentType - The HTTP content type
lenient - indicates if the charset encoding detection should be relaxed.
Returns:
the encoding to be used
Throws:
java.io.IOException - thrown if there is a problem reading the stream.

doLenientDetection

private java.lang.String doLenientDetection(java.lang.String httpContentType,
                                            XmlStreamReaderException ex)
                                     throws java.io.IOException
Do lenient detection.

Parameters:
httpContentType - content-type header to use for the resolution of the charset encoding.
ex - The thrown exception
Returns:
the encoding
Throws:
java.io.IOException - thrown if there is a problem reading the stream.

calculateRawEncoding

java.lang.String calculateRawEncoding(java.lang.String bomEnc,
                                      java.lang.String xmlGuessEnc,
                                      java.lang.String xmlEnc)
                                throws java.io.IOException
Calculate the raw encoding.

Parameters:
bomEnc - BOM encoding
xmlGuessEnc - XML Guess encoding
xmlEnc - XML encoding
Returns:
the raw encoding
Throws:
java.io.IOException - thrown if there is a problem reading the stream.

calculateHttpEncoding

java.lang.String calculateHttpEncoding(java.lang.String httpContentType,
                                       java.lang.String bomEnc,
                                       java.lang.String xmlGuessEnc,
                                       java.lang.String xmlEnc,
                                       boolean lenient)
                                 throws java.io.IOException
Calculate the HTTP encoding.

Parameters:
httpContentType - The HTTP content type
bomEnc - BOM encoding
xmlGuessEnc - XML Guess encoding
xmlEnc - XML encoding
lenient - indicates if the charset encoding detection should be relaxed.
Returns:
the HTTP encoding
Throws:
java.io.IOException - thrown if there is a problem reading the stream.

getContentTypeMime

static java.lang.String getContentTypeMime(java.lang.String httpContentType)
Returns MIME type or NULL if httpContentType is NULL.

Parameters:
httpContentType - the HTTP content type
Returns:
The mime content type

getContentTypeEncoding

static java.lang.String getContentTypeEncoding(java.lang.String httpContentType)
Returns charset parameter value, NULL if not present, NULL if httpContentType is NULL.

Parameters:
httpContentType - the HTTP content type
Returns:
The content type encoding

getXmlProlog

private static java.lang.String getXmlProlog(java.io.InputStream is,
                                             java.lang.String guessedEnc)
                                      throws java.io.IOException
Returns the encoding declared in the , NULL if none.

Parameters:
is - InputStream to create the reader from.
guessedEnc - guessed encoding
Returns:
the encoding declared in the
Throws:
java.io.IOException - thrown if there is a problem reading the stream.

isAppXml

static boolean isAppXml(java.lang.String mime)
Indicates if the MIME type belongs to the APPLICATION XML family.

Parameters:
mime - The mime type
Returns:
true if the mime type belongs to the APPLICATION XML family, otherwise false

isTextXml

static boolean isTextXml(java.lang.String mime)
Indicates if the MIME type belongs to the TEXT XML family.

Parameters:
mime - The mime type
Returns:
true if the mime type belongs to the TEXT XML family, otherwise false


Copyright (c) 2002-2012 Apache Software Foundation