Class | Syntax::Token |
In: |
lib/syntax/common.rb
|
Parent: | String |
A single token extracted by a tokenizer. It is simply the lexeme itself, decorated with a ‘group’ attribute to identify the type of the lexeme.
group | [R] | the type of the lexeme that was extracted. |
instruction | [R] | the instruction associated with this token (:none, :region_open, or :region_close) |
Create a new Token representing the given text, and belonging to the given group.