Module syntax::ext::tt::macro_parser [−] [src]

Unstable (rustc_private)

This is an Earley-like parser, without support for in-grammar nonterminals, only by calling out to the main rust parser for named nonterminals (which it commits to fully when it hits one in a grammar). This means that there are no completer or predictor rules, and therefore no need to store one column per token: instead, there's a set of current Earley items and a set of next ones. Instead of NTs, we have a special case for Kleene star. The big-O, in pathological cases, is worse than traditional Earley parsing, but it's an easier fit for Macro-by-Example-style rules, and I think the overhead is lower. (In order to prevent the pathological case, we'd need to lazily construct the resulting NamedMatches at the very end. It'd be a pain, and require more memory to keep around old items, but it would also save overhead)

Quick intro to how the parser works:

A 'position' is a dot in the middle of a matcher, usually represented as a dot. For example · a $( a )* a b is a position, as is a $( · a )* a b.

The parser walks through the input a character at a time, maintaining a list of items consistent with the current position in the input string: cur_eis.

As it processes them, it fills up eof_eis with items that would be valid if the macro invocation is now over, bb_eis with items that are waiting on a Rust nonterminal like $e:expr, and next_eis with items that are waiting on a particular token. Most of the logic concerns moving the · through the repetitions indicated by Kleene stars. It only advances or calls out to the real Rust parser when no cur_eis items remain

Example: Start parsing a a a a b against [· a $( a )* a b].

Remaining input: a a a a b next_eis: [· a $( a )* a b]

Remaining input: a a a b cur: [a · $( a )* a b] Descend/Skip (first item). next: [a $( · a )* a b] [a $( a )* · a b].

Remaining input: a a b cur: [a $( a · )* a b] next: [a $( a )* a · b] Finish/Repeat (first item) next: [a $( a )* · a b] [a $( · a )* a b] [a $( a )* a · b]

Remaining input: a b cur: [a $( a · )* a b] next: [a $( a )* a · b] Finish/Repeat (first item) next: [a $( a )* · a b] [a $( · a )* a b] [a $( a )* a · b]

Remaining input: b cur: [a $( a · )* a b] next: [a $( a )* a · b] Finish/Repeat (first item) next: [a $( a )* · a b] [a $( · a )* a b]

Remaining input: `` eof: [a $( a )* a b ·]

Reexports

pub use self::NamedMatch::*;
pub use self::ParseResult::*;

Structs

MatcherPos [Unstable]

Enums

NamedMatch [Unstable]

NamedMatch is a pattern-match result for a single token::MATCH_NONTERMINAL: so it is associated with a single ident in a parse, and all MatchedNonterminals in the NamedMatch have the same nonterminal type (expr, item, etc). Each leaf in a single NamedMatch corresponds to a single token::MATCH_NONTERMINAL in the TokenTree that produced it.
ParseResult [Unstable]

Functions

count_names [Unstable]
initial_matcher_pos [Unstable]
nameize [Unstable]
parse [Unstable]
parse_nt [Unstable]
token_name_eq [Unstable]

Perform a token equality check, ignoring syntax context (that is, an unhygienic comparison)

Type Definitions

NamedParseResult [Unstable]
PositionalParseResult [Unstable]