org.antlr.v4.runtime

Class CodePointCharStream

java.lang.Object

org.antlr.v4.runtime.CodePointCharStream

All Implemented Interfaces:

CharStream, IntStream

Direct Known Subclasses:

CodePointCharStream.CodePoint16BitCharStream, CodePointCharStream.CodePoint32BitCharStream, CodePointCharStream.CodePoint8BitCharStream

public abstract class CodePointCharStream
extends java.lang.Object
implements CharStream

Alternative to ANTLRInputStream which treats the input as a series of Unicode code points, instead of a series of UTF-16 code units. Use this if you need to parse input which potentially contains Unicode values > U+FFFF.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	CodePointCharStream.CodePoint16BitCharStream
private static class	CodePointCharStream.CodePoint32BitCharStream
private static class	CodePointCharStream.CodePoint8BitCharStream

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.String	name
protected int	position
protected int	size

Fields inherited from interface org.antlr.v4.runtime.IntStream

EOF, UNKNOWN_SOURCE_NAME

Constructor Summary

Constructors

Modifier	Constructor and Description
private	CodePointCharStream(int position, int remaining, java.lang.String name)

Method Summary

Modifier and Type	Method and Description
void	consume() Consumes the current symbol in the stream.
static CodePointCharStream	fromBuffer(CodePointBuffer codePointBuffer) Constructs a CodePointCharStream which provides access to the Unicode code points stored in codePointBuffer.
static CodePointCharStream	fromBuffer(CodePointBuffer codePointBuffer, java.lang.String name) Constructs a named CodePointCharStream which provides access to the Unicode code points stored in codePointBuffer.
(package private) abstract java.lang.Object	getInternalStorage()
java.lang.String	getSourceName() Gets the name of the underlying symbol source.
int	index() Return the index into the stream of the input symbol referred to by LA(1).
int	mark() mark/release do nothing; we have entire buffer
void	release(int marker) This method releases a marked range created by a call to mark().
void	seek(int index) Set the input cursor to the position indicated by index.
int	size() Returns the total number of symbols in the stream, including a single EOF symbol.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.antlr.v4.runtime.CharStream

getText

Methods inherited from interface org.antlr.v4.runtime.IntStream

LA

Field Detail

size

protected final int size

name

protected final java.lang.String name

position

protected int position

Constructor Detail

CodePointCharStream

private CodePointCharStream(int position,
                            int remaining,
                            java.lang.String name)

Method Detail

getInternalStorage

abstract java.lang.Object getInternalStorage()

fromBuffer

public static CodePointCharStream fromBuffer(CodePointBuffer codePointBuffer)

Constructs a CodePointCharStream which provides access to the Unicode code points stored in codePointBuffer.

fromBuffer

public static CodePointCharStream fromBuffer(CodePointBuffer codePointBuffer,
                                             java.lang.String name)

Constructs a named CodePointCharStream which provides access to the Unicode code points stored in codePointBuffer.

consume

public final void consume()

Description copied from interface: IntStream

Consumes the current symbol in the stream. This method has the following effects:

• Forward movement: The value of index() before calling this method is less than the value of index() after calling this method.
• Ordered lookahead: The value of LA(1) before calling this method becomes the value of LA(-1) after calling this method.

Note that calling this method does not guarantee that index() is incremented by exactly 1, as that would preclude the ability to implement filtering streams (e.g. CommonTokenStream which distinguishes between "on-channel" and "off-channel" tokens).

Specified by:

consume in interface IntStream

index

public final int index()

Description copied from interface: IntStream

Return the index into the stream of the input symbol referred to by LA(1).

The behavior of this method is unspecified if no call to an initializing method has occurred after this stream was constructed.

Specified by:

index in interface IntStream

size

public final int size()

Description copied from interface: IntStream

Returns the total number of symbols in the stream, including a single EOF symbol.

Specified by:

size in interface IntStream

mark

public final int mark()

mark/release do nothing; we have entire buffer

Specified by:

mark in interface IntStream

Returns:

An opaque marker which should be passed to release() when the marked range is no longer required.

release

public final void release(int marker)

Description copied from interface: IntStream

This method releases a marked range created by a call to mark(). Calls to release() must appear in the reverse order of the corresponding calls to mark(). If a mark is released twice, or if marks are not released in reverse order of the corresponding calls to mark(), the behavior is unspecified.

For more information and an example, see IntStream.mark().

Specified by:

release in interface IntStream

Parameters:

marker - A marker returned by a call to mark().

See Also:

IntStream.mark()

seek

public final void seek(int index)

Description copied from interface: IntStream

Set the input cursor to the position indicated by index. If the specified index lies past the end of the stream, the operation behaves as though index was the index of the EOF symbol. After this method returns without throwing an exception, then at least one of the following will be true.

• index() will return the index of the first symbol appearing at or after the specified index. Specifically, implementations which filter their sources should automatically adjust index forward the minimum amount required for the operation to target a non-ignored symbol.
• LA(1) returns IntStream.EOF

This operation is guaranteed to not throw an exception if index lies within a marked region. For more information on marked regions, see IntStream.mark(). The behavior of this method is unspecified if no call to an initializing method has occurred after this stream was constructed.

Specified by:

seek in interface IntStream

Parameters:

index - The absolute index to seek to.

getSourceName

public final java.lang.String getSourceName()

Description copied from interface: IntStream

Gets the name of the underlying symbol source. This method returns a non-null, non-empty string. If such a name is not known, this method returns IntStream.UNKNOWN_SOURCE_NAME.

Specified by:

getSourceName in interface IntStream

toString

public final java.lang.String toString()

Overrides:

toString in class java.lang.Object