refactor: rewrote tokenize and modified ohter funcs

Well i wanted to wildly change a lot of things about the lexer thinking
i could do something better but really all i found was automatic lexers
that at least for me don't really fit the project so a manual one it is,
i guess technically is a automata. Whatever, is good enough.
This commit is contained in:
2026-04-30 21:34:27 -06:00
parent f2c906c6aa
commit c41847e120
2 changed files with 42 additions and 34 deletions

View File

@@ -66,12 +66,12 @@ typedef struct {
LexerErr err;
int64_t number;
};
} I64Result;
} LexerI64Result;
// Lexer funtions as well as few functionality
TokenizeResult tokenize(const char* input);
ASTNodeResult tokenize_number(const char* input, size_t *offset);
I64Result string_to_integer(const char buf[]);
LexerI64Result string_to_integer(const char buf[]);
bool isoperator(int c);
Operator char_to_operator(int c);
char operator_to_char(Operator op);