twisted.xish.xpathparser.Scanner

Package twisted :: Package xish :: Module xpathparser :: Class Scanner

Class Scanner

Known Subclasses:: XPathParserScanner

Yapps scanner.

The Yapps scanner can work in context sensitive or context insensitive modes. The token(i) method is used to retrieve the i-th token. It takes a restrict set that limits the set of tokens it is allowed to return. In context sensitive mode, this restrict set guides the scanner. In context insensitive mode, there is no restriction (the set is always the full set of tokens).

Method Summary
	`__init__(self, patterns, ignore, input)` Initialize the scanner.
	`__repr__(self)` Print the last 10 tokens that have been scanned in
	`get_char_pos(self)` Get the current char position in the input text.
	`get_column_number(self)` Get the column number of the current position in the input text.
	`get_input_scanned(self)` Get the portion of the input that has been tokenized.
	`get_input_unscanned(self)` Get the portion of the input that has not yet been tokenized.
	`get_line_number(self)` Get the line number of the current position in the input text.
	`get_prev_char_pos(self, i)` Get the previous position (one token back) in the input text.
	`get_token_pos(self)` Get the current token position in the input text.
	`scan(self, restrict)` Should scan another token and add it to the list, self.tokens, and add the restriction to self.restrictions
	`token(self, i, restrict)` Get the i'th token in the input.

Method Details

init(self, patterns, ignore, input)
(Constructor)

Initialize the scanner.

Parameters:
  patterns : [(terminal, uncompiled regex), ...] or None
  ignore : [terminal,...]
  input : string

If patterns is None, we assume that the subclass has
defined self.patterns : [(terminal, compiled regex), ...].
Note that the patterns parameter expects uncompiled regexes,
whereas the self.patterns field expects compiled regexes.

repr(self)
(Representation operator)

Print the last 10 tokens that have been scanned in

get_char_pos(self)

Get the current char position in the input text.

get_column_number(self)

Get the column number of the current position in the input text.

get_input_scanned(self)

Get the portion of the input that has been tokenized.

get_input_unscanned(self)

Get the portion of the input that has not yet been tokenized.

get_line_number(self)

Get the line number of the current position in the input text.

get_prev_char_pos(self, i=None)

Get the previous position (one token back) in the input text.

get_token_pos(self)

Get the current token position in the input text.

scan(self, restrict)

Should scan another token and add it to the list, self.tokens, and add the restriction to self.restrictions

token(self, i, restrict=None)

Get the i'th token in the input.

If i is one past the end, then scan for another token.

Args:

restrict : [token, ...] or None; if restrict is None, then any token is allowed. You may call token(i) more than once. However, the restrict set may never be larger than what was passed in on the first call to token(i).

Generated by Epydoc 2.0 on Sat May 15 20:07:36 2004

http://epydoc.sf.net