In the process of adding floating point numbers, I ran into something a little bit unexpected. The issue turned out to be pretty simple, but I thought it was worth mentioning.
Subscribe to just rust posts via RSS.
I’ve gone through the lexer, parser, and evaluator and added subtraction, multiplication, and division in addition to, uh… addition. And they kind of work, but there’s one glaring issue that I mentioned back in part 2. It’s that the parser has no understanding of operator precedence. That is to say, it doesn’t know which operators have a higher priority in the order of operations when implicit grouping is taking place.
Now that the lexer is actually lexing, we can start parsing. This is where the Tree in Abstract Syntax Tree really comes in. What the parser is going to do is take a flat sequence of tokens and transform it into a shape that represents the actual structure of the code.
The first part of the language I’ve built is the lexer. It takes the program text as input and produces a vector of tokens. Tokens are the individual units that the parser will work with, rather than it having to work directly with characters. A token could be a bunch of different things. It could be a literal value (like a number or string), or it could be an identifier, or a specific symbol (like a plus sign).