X-Git-Url: http://demsky.eecs.uci.edu/git/?a=blobdiff_plain;f=docs%2Ftutorial%2FOCamlLangImpl1.html;h=c7b0954021e3cd3e74556c33cbc16d66cb06f050;hb=6fa1c051dc515b6fd1f9a26ac12fed985469bff5;hp=4b252a411ead1a7049f7c0c0059155273aa6ccf4;hpb=9ba8a5736a9140232629fdbbad7b7c5c2bd5ffcb;p=oota-llvm.git diff --git a/docs/tutorial/OCamlLangImpl1.html b/docs/tutorial/OCamlLangImpl1.html index 4b252a411ea..c7b0954021e 100644 --- a/docs/tutorial/OCamlLangImpl1.html +++ b/docs/tutorial/OCamlLangImpl1.html @@ -219,15 +219,15 @@ type token =

Each token returned by our lexer will be one of the token variant values. -An unknown character like '+' will be returned as Kwd '+'. If the -curr token is an identifier, the value will be Ident s. If the -current token is a numeric literal (like 1.0), the value will be -Number 1.0. +An unknown character like '+' will be returned as Token.Kwd '+'. If +the curr token is an identifier, the value will be Token.Ident s. If +the current token is a numeric literal (like 1.0), the value will be +Token.Number 1.0.

The actual implementation of the lexer is a collection of functions driven -by a function named lex. The lex function is called to -return the next token from standard input. We will use +by a function named Lexer.lex. The Lexer.lex function is +called to return the next token from standard input. We will use Camlp4 to simplify the tokenization of the standard input. Its definition starts as:

@@ -245,13 +245,13 @@ let rec lex = parser

-lex works by recursing over a char Stream.t to read +Lexer.lex works by recursing over a char Stream.t to read characters one at a time from the standard input. It eats them as it recognizes -them and stores them in in a token variant. The first thing that it -has to do is ignore whitespace between tokens. This is accomplished with the +them and stores them in in a Token.token variant. The first thing that +it has to do is ignore whitespace between tokens. This is accomplished with the recursive call above.

-

The next thing lex needs to do is recognize identifiers and +

The next thing Lexer.lex needs to do is recognize identifiers and specific keywords like "def". Kaleidoscope does this with this a pattern match and a helper function.

@@ -300,8 +300,8 @@ and lex_number buffer = parser

This is all pretty straight-forward code for processing input. When reading a numeric value from input, we use the ocaml float_of_string function -to convert it to a numeric value that we store in NumVal. Note that -this isn't doing sufficient error checking: it will raise Failure +to convert it to a numeric value that we store in Token.Number. Note +that this isn't doing sufficient error checking: it will raise Failure if the string "1.23.45.67". Feel free to extend it :). Next we handle comments: