clang
9.0.0
|
Simple tokenizer for the parser. More...
Public Member Functions | |
CodeTokenizer (StringRef MatcherCode, Diagnostics *Error) | |
CodeTokenizer (StringRef MatcherCode, Diagnostics *Error, unsigned CodeCompletionOffset) | |
const TokenInfo & | peekNextToken () const |
Returns but doesn't consume the next token. More... | |
TokenInfo | consumeNextToken () |
Consumes and returns the next token. More... | |
TokenInfo::TokenKind | nextTokenKind () const |
Simple tokenizer for the parser.
Definition at line 66 of file Parser.cpp.
|
inlineexplicit |
Definition at line 68 of file Parser.cpp.
|
inline |
Definition at line 73 of file Parser.cpp.
|
inline |
Consumes and returns the next token.
Definition at line 84 of file Parser.cpp.
|
inline |
Definition at line 90 of file Parser.cpp.
References c, clang::ast_matchers::dynamic::SourceLocation::Column, clang::ast_matchers::dynamic::SourceRange::End, clang::isAlphanumeric(), clang::isHexDigit(), clang::isWhitespace(), clang::ast_matchers::dynamic::Parser::TokenInfo::Kind, clang::ast_matchers::dynamic::SourceLocation::Line, Line, clang::ast_matchers::dynamic::Parser::TokenInfo::Range, clang::ast_matchers::dynamic::SourceRange::Start, clang::ast_matchers::dynamic::Parser::TokenInfo::Text, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_CloseParen, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_CodeCompletion, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Comma, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Eof, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Error, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Ident, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_InvalidChar, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Literal, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_OpenParen, clang::ast_matchers::dynamic::Parser::TokenInfo::TK_Period, clang::toLowercase(), clang::ast_matchers::dynamic::Parser::TokenInfo::Value, and clang::ast_matchers::dynamic::Parser::Sema::~Sema().
|
inline |
Returns but doesn't consume the next token.
Definition at line 81 of file Parser.cpp.
Referenced by clang::ast_matchers::dynamic::Parser::parseExpression().