Class Syntax::Token
In: lib/syntax/common.rb
Parent: String

A single token extracted by a tokenizer. It is simply the lexeme itself, decorated with a ‘group’ attribute to identify the type of the lexeme.

Methods

new  

Attributes

group  [R]  the type of the lexeme that was extracted.
instruction  [R]  the instruction associated with this token (:none, :region_open, or :region_close)

Public Class methods

Create a new Token representing the given text, and belonging to the given group.

[Validate]