From d9ff5aa48cbbc7bc4388cd743aa354f60ab125a5 Mon Sep 17 00:00:00 2001 From: aarne Date: Mon, 10 Nov 2008 15:53:38 +0000 Subject: lexer documentation in help and tutorial updated --- doc/gf-tutorial.txt | 24 ++++++------------------ 1 file changed, 6 insertions(+), 18 deletions(-) (limited to 'doc') diff --git a/doc/gf-tutorial.txt b/doc/gf-tutorial.txt index cf9518cff..79d924a16 100644 --- a/doc/gf-tutorial.txt +++ b/doc/gf-tutorial.txt @@ -4277,24 +4277,12 @@ In linearization, we use a corresponding **unlexer**: ===Most common lexers and unlexers=== - || lexer | description || - | ``words`` | (default) tokens are separated by spaces or newlines - | ``literals`` | like words, but integer and string literals recognized - | ``chars`` | each character is a token - | ``code`` | program code conventions (uses Haskell's lex) - | ``text`` | with conventions on punctuation and capital letters - | ``codelit`` | like code, but recognize literals (unknown words as strings) - | ``textlit`` | like text, but recognize literals (unknown words as strings) - - || unlexer | description || - | ``unwords`` | (default) space-separated token list - | ``text`` | format as text: punctuation, capitals, paragraph

- | ``code`` | format as code (spacing, indentation) - | ``textlit`` | like text, but remove string literal quotes - | ``codelit`` | like code, but remove string literal quotes - | ``concat`` | remove all spaces - -%TODO: update the names + || lexer | unlexer | description || + | ``chars`` | ``unchars`` | each character is a token + | ``lexcode`` | ``unlexcode`` | program code conventions (uses Haskell's lex) + | ``lexmixed`` | ``unlexmixed`` | like text, but between $ signs like code + | ``lextext`` | ``unlextext`` | with conventions on punctuation and capitals + | ``words`` | ``unwords`` | (default) tokens separated by space characters %TODO: also on alphabet encodings - although somewhere else -- cgit v1.2.3