Crate compiler_project_tc3002_b

Lexical Analyzer

Diego Eduardo Hernandez Cadena - A00834015

Implements a lexical analyzer using the plex crate. It reads the specified input file, tries to parse it, and prints a list of the resulting tokens.

Usage

To use the lexical analyzer, set the INPUT_FILE_PATH constant to the desired location and run cargo run to execute the program.

Adding more keywords and operators

To add a keyword or operator:

• Add it to the Keyword or Operator enum.
• Add an entry to the corresponding phf_map! static map.
• If necessary, update the regular expressions in the lexer! macro.

Adding more tokens

To add a new token:

• Add a new variant to the Token enum.
• Add a matching rule in the lexer! macro that maps input to the new token.

Enums

Keyword

Represents supported keywords that the lexer can recognize

Operator

Represents supported operators in the language

Token

Represents all possible tokens that can be produced by the lexer

Functions

extract_file_contents

Reads the contents of the file at the specified path

extract_tokens

Extracts all tokens from the input string using the lexer

parse_keyword

Tries to match a string slice to a known Keyword

parse_operator

Tries to match a string slice to a known Operator

run

Main function: reads input, tokenizes it, and prints each token (excluding whitespace)