aud.example.graph.Tokenizer Class Reference

Breaks input string into pieces ("tokens"). More...

Inheritance diagram for aud.example.graph.Tokenizer:

Collaboration diagram for aud.example.graph.Tokenizer:

Public Member Functions
	Tokenizer (String input)
	create new tokenizer for input More...
Public Member Functions inherited from aud.util.LexicalScanner
	LexicalScanner (Rule[] rules, String input)
	create new scanner processing input @endiliteral More...
void	setInput (String input)
	set input (resets scanner state) More...
String	matchedText ()
	get text of last match or call to next More...
int	matchedTokenId ()
	get result of last call to next() More...
String	remainder ()
	get remaining text More...
boolean	endOfInput ()
	reached end of input? More...
int	next ()
	match remainder to rules provided to constructor More...

Static Public Attributes
static final int	LEFT_BRACE ='['
static final int	RIGHT_BRACE =']'
static final int	LEFT_PAREN ='('
static final int	RIGHT_PAREN =')'
static final int	AT ='@'
static final int	COMMA =','
static final int	IDENTIFIER =0x100
static final int	NUMBER =0x101
static final int	EDGE =0x102
static final int	DEDGE =0x103
Static Public Attributes inherited from aud.util.LexicalScanner
static final int	END_OF_INPUT = -1
	no more input More...
static final int	NO_MATCH = -2
	no match (usually implies a syntax error) More...
static final Pattern	P_WHITESPACE = Pattern.compile("\\s+")
	white space More...
static final Pattern	P_IDENTIFIER
	identifiers More...
static final Pattern	P_FLOAT
	floating point number More...

Static Protected Attributes
static final Pattern	P_LEFTBRACE = Pattern.compile("\\[")
static final Pattern	P_RIGHTBRACE = Pattern.compile("\\]")
static final Pattern	P_LEFTPAREN = Pattern.compile("\\‍(")
static final Pattern	P_RIGHTPAREN = Pattern.compile("\\‍)")
static final Pattern	P_EDGE = Pattern.compile("--")
static final Pattern	P_DEDGE = Pattern.compile("->")
static final Pattern	P_AT = Pattern.compile("@")
static final Pattern	P_COMMA = Pattern.compile(",")
static final LexicalScanner.Rule[]	RULES

Additional Inherited Members
Static Public Member Functions inherited from aud.util.LexicalScanner
static void	main (String[] args)
	testing and example for usage More...
Protected Member Functions inherited from aud.util.LexicalScanner
void	eatWhiteSpace ()
	ignore white space (called by match More...
boolean	match (Pattern p)
	Match remainder against pattern p. More...
int	next (Rule[] rules)
	match remainder to table of rules @endiliteral More...

Detailed Description

Breaks input string into pieces ("tokens").

See also

GraphParser

Definition at line 10 of file graph/Tokenizer.java.

Constructor & Destructor Documentation

Tokenizer()

aud.example.graph.Tokenizer.Tokenizer

(

String

input

)

create new tokenizer for input

Definition at line 49 of file graph/Tokenizer.java.

                                 {
    super(RULES,input);
  }

References aud.example.graph.Tokenizer.RULES.

Member Data Documentation

AT

final int aud.example.graph.Tokenizer.AT ='@'

static

Definition at line 25 of file graph/Tokenizer.java.

Referenced by aud.example.graph.GraphParser.pos().

COMMA

final int aud.example.graph.Tokenizer.COMMA =','

static

Definition at line 26 of file graph/Tokenizer.java.

DEDGE

final int aud.example.graph.Tokenizer.DEDGE =0x103

static

Definition at line 31 of file graph/Tokenizer.java.

Referenced by aud.example.graph.GraphParser.edge().

EDGE

final int aud.example.graph.Tokenizer.EDGE =0x102

static

Definition at line 30 of file graph/Tokenizer.java.

Referenced by aud.example.graph.GraphParser.edge().

IDENTIFIER

final int aud.example.graph.Tokenizer.IDENTIFIER =0x100

static

Definition at line 28 of file graph/Tokenizer.java.

Referenced by aud.example.graph.GraphParser.node().

LEFT_BRACE

final int aud.example.graph.Tokenizer.LEFT_BRACE ='['

static

Definition at line 21 of file graph/Tokenizer.java.

Referenced by aud.example.graph.GraphParser.weight().

LEFT_PAREN

final int aud.example.graph.Tokenizer.LEFT_PAREN ='('

static

Definition at line 23 of file graph/Tokenizer.java.

Referenced by aud.example.graph.GraphParser.pos().

NUMBER

final int aud.example.graph.Tokenizer.NUMBER =0x101

static

Definition at line 29 of file graph/Tokenizer.java.

Referenced by aud.example.graph.GraphParser.node(), aud.example.graph.GraphParser.pos(), and aud.example.graph.GraphParser.weight().

P_AT

final Pattern aud.example.graph.Tokenizer.P_AT = Pattern.compile("@")

staticprotected

Definition at line 18 of file graph/Tokenizer.java.

P_COMMA

final Pattern aud.example.graph.Tokenizer.P_COMMA = Pattern.compile(",")

staticprotected

Definition at line 19 of file graph/Tokenizer.java.

P_DEDGE

final Pattern aud.example.graph.Tokenizer.P_DEDGE = Pattern.compile("->")

staticprotected

Definition at line 17 of file graph/Tokenizer.java.

P_EDGE

final Pattern aud.example.graph.Tokenizer.P_EDGE = Pattern.compile("--")

staticprotected

Definition at line 16 of file graph/Tokenizer.java.

P_LEFTBRACE

final Pattern aud.example.graph.Tokenizer.P_LEFTBRACE = Pattern.compile("\\[")

staticprotected

Definition at line 12 of file graph/Tokenizer.java.

P_LEFTPAREN

final Pattern aud.example.graph.Tokenizer.P_LEFTPAREN = Pattern.compile("\\‍(")

staticprotected

Definition at line 14 of file graph/Tokenizer.java.

P_RIGHTBRACE

final Pattern aud.example.graph.Tokenizer.P_RIGHTBRACE = Pattern.compile("\\]")

staticprotected

Definition at line 13 of file graph/Tokenizer.java.

P_RIGHTPAREN

final Pattern aud.example.graph.Tokenizer.P_RIGHTPAREN = Pattern.compile("\\‍)")

staticprotected

Definition at line 15 of file graph/Tokenizer.java.

RIGHT_BRACE

final int aud.example.graph.Tokenizer.RIGHT_BRACE =']'

static

Definition at line 22 of file graph/Tokenizer.java.

RIGHT_PAREN

final int aud.example.graph.Tokenizer.RIGHT_PAREN =')'

static

Definition at line 24 of file graph/Tokenizer.java.

RULES

final LexicalScanner.Rule [] aud.example.graph.Tokenizer.RULES

staticprotected

Initial value:

=
  {
    new LexicalScanner.Rule(EDGE,P_EDGE),
    new LexicalScanner.Rule(DEDGE,P_DEDGE),
    new LexicalScanner.Rule(AT,P_AT),
    new LexicalScanner.Rule(COMMA,P_COMMA),
    new LexicalScanner.Rule(LEFT_BRACE,P_LEFTBRACE),
    new LexicalScanner.Rule(RIGHT_BRACE,P_RIGHTBRACE),
    new LexicalScanner.Rule(LEFT_PAREN,P_LEFTPAREN),
    new LexicalScanner.Rule(RIGHT_PAREN,P_RIGHTPAREN),
    new LexicalScanner.Rule(NUMBER,LexicalScanner.P_FLOAT),
    new LexicalScanner.Rule(IDENTIFIER,LexicalScanner.P_IDENTIFIER)
  }

Definition at line 34 of file graph/Tokenizer.java.

Referenced by aud.example.graph.Tokenizer.Tokenizer().

---

The documentation for this class was generated from the following file:

• graph/Tokenizer.java