summaryrefslogtreecommitdiff
path: root/src/GF/GFCC/doc/old-gfcc.txt
diff options
context:
space:
mode:
authoraarne <aarne@cs.chalmers.se>2008-06-25 16:43:48 +0000
committeraarne <aarne@cs.chalmers.se>2008-06-25 16:43:48 +0000
commitb96b36f43de3e2f8b58d5f539daa6f6d47f25870 (patch)
tree0992334be13cec6538a1dea22fbbf26ad6bdf224 /src/GF/GFCC/doc/old-gfcc.txt
parentfe367412e0aeb4ad5c02de68e6eca382e0f96984 (diff)
removed src for 2.9
Diffstat (limited to 'src/GF/GFCC/doc/old-gfcc.txt')
-rw-r--r--src/GF/GFCC/doc/old-gfcc.txt656
1 files changed, 0 insertions, 656 deletions
diff --git a/src/GF/GFCC/doc/old-gfcc.txt b/src/GF/GFCC/doc/old-gfcc.txt
deleted file mode 100644
index 6ffd9bd64..000000000
--- a/src/GF/GFCC/doc/old-gfcc.txt
+++ /dev/null
@@ -1,656 +0,0 @@
-The GFCC Grammar Format
-Aarne Ranta
-October 19, 2006
-
-Author's address:
-[``http://www.cs.chalmers.se/~aarne`` http://www.cs.chalmers.se/~aarne]
-
-% to compile: txt2tags -thtml --toc gfcc.txt
-
-History:
-- 19 Oct: translation of lincats, new figures on C++
-- 3 Oct 2006: first version
-
-
-==What is GFCC==
-
-GFCC is a low-level format for GF grammars. Its aim is to contain the minimum
-that is needed to process GF grammars at runtime. This minimality has three
-advantages:
-- compact grammar files and run-time objects
-- time and space efficient processing
-- simple definition of interpreters
-
-
-The idea is that all embedded GF applications are compiled to GFCC.
-The GF system would be primarily used as a compiler and as a grammar
-development tool.
-
-Since GFCC is implemented in BNFC, a parser of the format is readily
-available for C, C++, Haskell, Java, and OCaml. Also an XML
-representation is generated in BNFC. A
-[reference implementation ../]
-of linearization and some other functions has been written in Haskell.
-
-
-==GFCC vs. GFC==
-
-GFCC is aimed to replace GFC as the run-time grammar format. GFC was designed
-to be a run-time format, but also to
-support separate compilation of grammars, i.e.
-to store the results of compiling
-individual GF modules. But this means that GFC has to contain extra information,
-such as type annotations, which is only needed in compilation and not at
-run-time. In particular, the pattern matching syntax and semantics of GFC is
-complex and therefore difficult to implement in new platforms.
-
-The main differences of GFCC compared with GFC can be summarized as follows:
-- there are no modules, and therefore no qualified names
-- a GFCC grammar is multilingual, and consists of a common abstract syntax
- together with one concrete syntax per language
-- records and tables are replaced by arrays
-- record labels and parameter values are replaced by integers
-- record projection and table selection are replaced by array indexing
-- there is (so far) no support for dependent types or higher-order abstract
- syntax (which would be easy to add, but make interpreters much more difficult
- to write)
-
-
-Here is an example of a GF grammar, consisting of three modules,
-as translated to GFCC. The representations are aligned, with the exceptions
-due to the alphabetical sorting of GFCC grammars.
-```
- grammar Ex(Eng,Swe);
-
-abstract Ex = { abstract {
- cat
- S ; NP ; VP ;
- fun
- Pred : NP -> VP -> S ; Pred : NP,VP -> S = (Pred);
- She, They : NP ; She : -> NP = (She);
- Sleep : VP ; Sleep : -> VP = (Sleep);
- They : -> NP = (They);
-} } ;
-
-concrete Eng of Ex = { concrete Eng {
- lincat
- S = {s : Str} ;
- NP = {s : Str ; n : Num} ;
- VP = {s : Num => Str} ;
- param
- Num = Sg | Pl ;
- lin
- Pred np vp = { Pred = [(($0!1),(($1!0)!($0!0)))];
- s = np.s ++ vp.s ! np.n} ;
- She = {s = "she" ; n = Sg} ; She = [0, "she"];
- They = {s = "they" ; n = Pl} ;
- Sleep = {s = table { Sleep = [("sleep" + ["s",""])];
- Sg => "sleeps" ;
- Pl => "sleep" They = [1, "they"];
- } } ;
- } ;
-}
-
-concrete Swe of Ex = { concrete Swe {
- lincat
- S = {s : Str} ;
- NP = {s : Str} ;
- VP = {s : Str} ;
- param
- Num = Sg | Pl ;
- lin
- Pred np vp = { Pred = [(($0!0),($1!0))];
- s = np.s ++ vp.s} ;
- She = {s = "hon"} ; She = ["hon"];
- They = {s = "de"} ; They = ["de"];
- Sleep = {s = "sover"} ; Sleep = ["sover"];
-} } ;
-```
-
-==The syntax of GFCC files==
-
-===Top level===
-
-A grammar has a header telling the name of the abstract syntax
-(often specifying an application domain), and the names of
-the concrete languages. The abstract syntax and the concrete
-syntaxes themselves follow.
-```
- Grammar ::= Header ";" Abstract ";" [Concrete] ;
- Header ::= "grammar" CId "(" [CId] ")" ;
- Abstract ::= "abstract" "{" [AbsDef] "}" ;
- Concrete ::= "concrete" CId "{" [CncDef] "}" ;
-```
-Abstract syntax judgements give typings and semantic definitions.
-Concrete syntax judgements give linearizations.
-```
- AbsDef ::= CId ":" Type "=" Exp ;
- CncDef ::= CId "=" Term ;
-```
-Also flags are possible, local to each "module" (i.e. abstract and concretes).
-```
- AbsDef ::= "%" CId "=" String ;
- CncDef ::= "%" CId "=" String ;
-```
-For the run-time system, the reference implementation in Haskell
-uses a structure that gives efficient look-up:
-```
- data GFCC = GFCC {
- absname :: CId ,
- cncnames :: [CId] ,
- abstract :: Abstr ,
- concretes :: Map CId Concr
- }
-
- data Abstr = Abstr {
- funs :: Map CId Type, -- find the type of a fun
- cats :: Map CId [CId] -- find the funs giving a cat
- }
-
- type Concr = Map CId Term
-```
-
-
-===Abstract syntax===
-
-Types are first-order function types built from
-category symbols. Syntax trees (``Exp``) are
-rose trees with the head (``Atom``) either a function
-constant, a metavariable, or a string, integer, or float
-literal.
-```
- Type ::= [CId] "->" CId ;
- Exp ::= "(" Atom [Exp] ")" ;
- Atom ::= CId ; -- function constant
- Atom ::= "?" ; -- metavariable
- Atom ::= String ; -- string literal
- Atom ::= Integer ; -- integer literal
- Atom ::= Double ; -- float literal
-```
-
-
-===Concrete syntax===
-
-Linearization terms (``Term``) are built as follows.
-Constructor names are shown to make the later code
-examples readable.
-```
- R. Term ::= "[" [Term] "]" ; -- array
- P. Term ::= "(" Term "!" Term ")" ; -- access to indexed field
- S. Term ::= "(" [Term] ")" ; -- sequence with ++
- K. Term ::= Tokn ; -- token
- V. Term ::= "$" Integer ; -- argument
- C. Term ::= Integer ; -- array index
- FV. Term ::= "[|" [Term] "|]" ; -- free variation
- TM. Term ::= "?" ; -- linearization of metavariable
-```
-Tokens are strings or (maybe obsolescent) prefix-dependent
-variant lists.
-```
- KS. Tokn ::= String ;
- KP. Tokn ::= "[" "pre" [String] "[" [Variant] "]" "]" ;
- Var. Variant ::= [String] "/" [String] ;
-```
-Three special forms of terms are introduced by the compiler
-as optimizations. They can in principle be eliminated, but
-their presence makes grammars much more compact. Their semantics
-will be explained in a later section.
-```
- F. Term ::= CId ; -- global constant
- W. Term ::= "(" String "+" Term ")" ; -- prefix + suffix table
- RP. Term ::= "(" Term "@" Term ")"; -- record parameter alias
-```
-Identifiers are like ``Ident`` in GF and GFC, except that
-the compiler produces constants prefixed with ``_`` in
-the common subterm elimination optimization.
-```
- token CId (('_' | letter) (letter | digit | '\'' | '_')*) ;
-```
-
-
-==The semantics of concrete syntax terms==
-
-===Linearization and realization===
-
-The linearization algorithm is essentially the same as in
-GFC: a tree is linearized by evaluating its linearization term
-in the environment of the linearizations of the subtrees.
-Literal atoms are linearized in the obvious way.
-The function also needs to know the language (i.e. concrete syntax)
-in which linearization is performed.
-```
- linExp :: GFCC -> CId -> Exp -> Term
- linExp mcfg lang tree@(Tr at trees) = case at of
- AC fun -> comp (Prelude.map lin trees) $ look fun
- AS s -> R [kks (show s)] -- quoted
- AI i -> R [kks (show i)]
- AF d -> R [kks (show d)]
- AM -> TM
- where
- lin = linExp mcfg lang
- comp = compute mcfg lang
- look = lookLin mcfg lang
-```
-The result of linearization is usually a record, which is realized as
-a string using the following algorithm.
-```
- realize :: Term -> String
- realize trm = case trm of
- R (t:_) -> realize t
- S ss -> unwords $ Prelude.map realize ss
- K (KS s) -> s
- K (KP s _) -> unwords s ---- prefix choice TODO
- W s t -> s ++ realize t
- FV (t:_) -> realize t
- TM -> "?"
-```
-Since the order of record fields is not necessarily
-the same as in GF source,
-this realization does not work securely for
-categories whose lincats more than one field.
-
-
-===Term evaluation===
-
-Evaluation follows call-by-value order, with two environments
-needed:
-- the grammar (a concrete syntax) to give the global constants
-- an array of terms to give the subtree linearizations
-
-
-The code is presented in one-level pattern matching, to
-enable reimplementations in languages that do not permit
-deep patterns (such as Java and C++).
-```
-compute :: GFCC -> CId -> [Term] -> Term -> Term
-compute mcfg lang args = comp where
- comp trm = case trm of
- P r p -> proj (comp r) (comp p)
- RP i t -> RP (comp i) (comp t)
- W s t -> W s (comp t)
- R ts -> R $ Prelude.map comp ts
- V i -> idx args (fromInteger i) -- already computed
- F c -> comp $ look c -- not computed (if contains V)
- FV ts -> FV $ Prelude.map comp ts
- S ts -> S $ Prelude.filter (/= S []) $ Prelude.map comp ts
- _ -> trm
-
- look = lookLin mcfg lang
-
- idx xs i = xs !! i
-
- proj r p = case (r,p) of
- (_, FV ts) -> FV $ Prelude.map (proj r) ts
- (W s t, _) -> kks (s ++ getString (proj t p))
- _ -> comp $ getField r (getIndex p)
-
- getString t = case t of
- K (KS s) -> s
- _ -> trace ("ERROR in grammar compiler: string from "++ show t) "ERR"
-
- getIndex t = case t of
- C i -> fromInteger i
- RP p _ -> getIndex p
- TM -> 0 -- default value for parameter
- _ -> trace ("ERROR in grammar compiler: index from " ++ show t) 0
-
- getField t i = case t of
- R rs -> idx rs i
- RP _ r -> getField r i
- TM -> TM
- _ -> trace ("ERROR in grammar compiler: field from " ++ show t) t
-```
-
-===The special term constructors===
-
-The three forms introduced by the compiler may a need special
-explanation.
-
-Global constants
-```
- Term ::= CId ;
-```
-are shorthands for complex terms. They are produced by the
-compiler by (iterated) common subexpression elimination.
-They are often more powerful than hand-devised code sharing in the source
-code. They could be computed off-line by replacing each identifier by
-its definition.
-
-Prefix-suffix tables
-```
- Term ::= "(" String "+" Term ")" ;
-```
-represent tables of word forms divided to the longest common prefix
-and its array of suffixes. In the example grammar above, we have
-```
- Sleep = [("sleep" + ["s",""])]
-```
-which in fact is equal to the array of full forms
-```
- ["sleeps", "sleep"]
-```
-The power of this construction comes from the fact that suffix sets
-tend to be repeated in a language, and can therefore be collected
-by common subexpression elimination. It is this technique that
-explains the used syntax rather than the more accurate
-```
- "(" String "+" [String] ")"
-```
-since we want the suffix part to be a ``Term`` for the optimization to
-take effect.
-
-The most curious construct of GFCC is the parameter array alias,
-```
- Term ::= "(" Term "@" Term ")";
-```
-This form is used as the value of parameter records, such as the type
-```
- {n : Number ; p : Person}
-```
-The problem with parameter records is their double role.
-They can be used like parameter values, as indices in selection,
-```
- VP.s ! {n = Sg ; p = P3}
-```
-but also as records, from which parameters can be projected:
-```
- {n = Sg ; p = P3}.n
-```
-Whichever use is selected as primary, a prohibitively complex
-case expression must be generated at compilation to GFCC to get the
-other use. The adopted
-solution is to generate a pair containing both a parameter value index
-and an array of indices of record fields. For instance, if we have
-```
- param Number = Sg | Pl ; Person = P1 | P2 | P3 ;
-```
-we get the encoding
-```
- {n = Sg ; p = P3} ---> (2 @ [0,2])
-```
-The GFCC computation rules are essentially
-```
- (t ! (i @ _)) = (t ! i)
- ((_ @ r) ! j) =(r ! j)
-```
-
-
-==Compiling to GFCC==
-
-Compilation to GFCC is performed by the GF grammar compiler, and
-GFCC interpreters need not know what it does. For grammar writers,
-however, it might be interesting to know what happens to the grammars
-in the process.
-
-The compilation phases are the following
-+ translate GF source to GFC, as always in GF
-+ undo GFC back-end optimizations
-+ perform the ``values`` optimization to normalize tables
-+ create a symbol table mapping the GFC parameter and record types to
- fixed-size arrays, and parameter values and record labels to integers
-+ traverse the linearization rules replacing parameters and labels by integers
-+ reorganize the created GFC grammar so that it has just one abstract syntax
- and one concrete syntax per language
-+ apply UTF8 encoding to the grammar, if not yet applied (this is told by the
- ``coding`` flag)
-+ translate the GFC syntax tree to a GFCC syntax tree, using a simple
- compositional mapping
-+ perform the word-suffix optimization on GFCC linearization terms
-+ perform subexpression elimination on each concrete syntax module
-+ print out the GFCC code
-
-
-Notice that a major part of the compilation is done within GFC, so that
-GFC-related tasks (such as parser generation) could be performed by
-using the old algorithms.
-
-
-===Problems in GFCC compilation===
-
-Two major problems had to be solved in compiling GFC to GFCC:
-- consistent order of tables and records, to permit the array translation
-- run-time variables in complex parameter values.
-
-
-The current implementation is still experimental and may fail
-to generate correct code. Any errors remaining are likely to be
-related to the two problems just mentioned.
-
-The order problem is solved in different ways for tables and records.
-For tables, the ``values`` optimization of GFC already manages to
-maintain a canonical order. But this order can be destroyed by the
-``share`` optimization. To make sure that GFCC compilation works properly,
-it is safest to recompile the GF grammar by using the ``values``
-optimization flag.
-
-Records can be canonically ordered by sorting them by labels.
-In fact, this was done in connection of the GFCC work as a part
-of the GFC generation, to guarantee consistency. This means that
-e.g. the ``s`` field will in general no longer appear as the first
-field, even if it does so in the GF source code. But relying on the
-order of fields in a labelled record would be misplaced anyway.
-
-The canonical form of records is further complicated by lock fields,
-i.e. dummy fields of form ``lock_C = <>``, which are added to grammar
-libraries to force intensionality of linearization types. The problem
-is that the absence of a lock field only generates a warning, not
-an error. Therefore a GFC grammar can contain objects of the same
-type with and without a lock field. This problem was solved in GFCC
-generation by just removing all lock fields (defined as fields whose
-type is the empty record type). This has the further advantage of
-(slightly) reducing the grammar size. More importantly, it is safe
-to remove lock fields, because they are never used in computation,
-and because intensional types are only needed in grammars reused
-as libraries, not in grammars used at runtime.
-
-While the order problem is rather bureaucratic in nature, run-time
-variables are an interesting problem. They arise in the presence
-of complex parameter values, created by argument-taking constructors
-and parameter records. To give an example, consider the GF parameter
-type system
-```
- Number = Sg | Pl ;
- Person = P1 | P2 | P3 ;
- Agr = Ag Number Person ;
-```
-The values can be translated to integers in the expected way,
-```
- Sg = 0, Pl = 1
- P1 = 0, P2 = 1, P3 = 2
- Ag Sg P1 = 0, Ag Sg P2 = 1, Ag Sg P3 = 2,
- Ag Pl P1 = 3, Ag Pl P2 = 4, Ag Pl P3 = 5
-```
-However, an argument of ``Agr`` can be a run-time variable, as in
-```
- Ag np.n P3
-```
-This expression must first be translated to a case expression,
-```
- case np.n of {
- 0 => 2 ;
- 1 => 5
- }
-```
-which can then be translated to the GFCC term
-```
- ([2,5] ! ($0 ! $1))
-```
-assuming that the variable ``np`` is the first argument and that its
-``Number`` field is the second in the record.
-
-This transformation of course has to be performed recursively, since
-there can be several run-time variables in a parameter value:
-```
- Ag np.n np.p
-```
-A similar transformation would be possible to deal with the double
-role of parameter records discussed above. Thus the type
-```
- RNP = {n : Number ; p : Person}
-```
-could be uniformly translated into the set ``{0,1,2,3,4,5}``
-as ``Agr`` above. Selections would be simple instances of indexing.
-But any projection from the record should be translated into
-a case expression,
-```
- rnp.n ===>
- case rnp of {
- 0 => 0 ;
- 1 => 0 ;
- 2 => 0 ;
- 3 => 1 ;
- 4 => 1 ;
- 5 => 1
- }
-```
-To avoid the code bloat resulting from this, we chose the alias representation
-which is easy enough to deal with in interpreters.
-
-
-===The representation of linearization types===
-
-Linearization types (``lincat``) are not needed when generating with
-GFCC, but they have been added to enable parser generation directly from
-GFCC. The linearization type definitions are shown as a part of the
-concrete syntax, by using terms to represent types. Here is the table
-showing how different linearization types are encoded.
-```
- P* = size(P) -- parameter type
- {_ : I ; __ : R}* = (I* @ R*) -- record of parameters
- {r1 : T1 ; ... ; rn : Tn}* = [T1*,...,Tn*] -- other record
- (P => T)* = [T* ,...,T*] -- size(P) times
- Str* = ()
-```
-The category symbols are prefixed with two underscores (``__``).
-For example, the linearization type ``present/CatEng.NP`` is
-translated as follows:
-```
- NP = {
- a : { -- 6 = 2*3 values
- n : {ParamX.Number} ; -- 2 values
- p : {ParamX.Person} -- 3 values
- } ;
- s : {ResEng.Case} => Str -- 3 values
- }
-
- __NP = [(6@[2,3]),[(),(),()]]
-```
-
-
-
-
-===Running the compiler and the GFCC interpreter===
-
-GFCC generation is a part of the
-[developers' version http://www.cs.chalmers.se/Cs/Research/Language-technology/darcs/GF/doc/darcs.html]
-of GF since September 2006. To invoke the compiler, the flag
-``-printer=gfcc`` to the command
-``pm = print_multi`` is used. It is wise to recompile the grammar from
-source, since previously compiled libraries may not obey the canonical
-order of records. To ``strip`` the grammar before
-GFCC translation removes unnecessary interface references.
-Here is an example, performed in
-[example/bronzeage ../../../../../examples/bronzeage].
-```
- i -src -path=.:prelude:resource-1.0/* -optimize=all_subs BronzeageEng.gf
- i -src -path=.:prelude:resource-1.0/* -optimize=all_subs BronzeageGer.gf
- strip
- pm -printer=gfcc | wf bronze.gfcc
-```
-
-
-
-==The reference interpreter==
-
-The reference interpreter written in Haskell consists of the following files:
-```
- -- source file for BNFC
- GFCC.cf -- labelled BNF grammar of gfcc
-
- -- files generated by BNFC
- AbsGFCC.hs -- abstrac syntax of gfcc
- ErrM.hs -- error monad used internally
- LexGFCC.hs -- lexer of gfcc files
- ParGFCC.hs -- parser of gfcc files and syntax trees
- PrintGFCC.hs -- printer of gfcc files and syntax trees
-
- -- hand-written files
- DataGFCC.hs -- post-parser grammar creation, linearization and evaluation
- GenGFCC.hs -- random and exhaustive generation, generate-and-test parsing
- RunGFCC.hs -- main function - a simple command interpreter
-```
-It is included in the
-[developers' version http://www.cs.chalmers.se/Cs/Research/Language-technology/darcs/GF/doc/darcs.html]
-of GF, in the subdirectory [``GF/src/GF/Canon/GFCC`` ../].
-
-To compile the interpreter, type
-```
- make gfcc
-```
-in ``GF/src``. To run it, type
-```
- ./gfcc <GFCC-file>
-```
-The available commands are
-- ``gr <Cat> <Int>``: generate a number of random trees in category.
- and show their linearizations in all languages
-- ``grt <Cat> <Int>``: generate a number of random trees in category.
- and show the trees and their linearizations in all languages
-- ``gt <Cat> <Int>``: generate a number of trees in category from smallest,
- and show their linearizations in all languages
-- ``gtt <Cat> <Int>``: generate a number of trees in category from smallest,
- and show the trees and their linearizations in all languages
-- ``p <Int> <Cat> <String>``: "parse", i.e. generate trees until match or
- until the given number have been generated
-- ``<Tree>``: linearize tree in all languages, also showing full records
-- ``quit``: terminate the system cleanly
-
-
-==Interpreter in C++==
-
-A base-line interpreter in C++ has been started.
-Its main functionality is random generation of trees and linearization of them.
-
-Here are some results from running the different interpreters, compared
-to running the same grammar in GF, saved in ``.gfcm`` format.
-The grammar contains the English, German, and Norwegian
-versions of Bronzeage. The experiment was carried out on
-Ubuntu Linux laptop with 1.5 GHz Intel centrino processor.
-
-|| | GF | gfcc(hs) | gfcc++ |
-| program size | 7249k | 803k | 113k
-| grammar size | 336k | 119k | 119k
-| read grammar | 1150ms | 510ms | 100ms
-| generate 222 | 9500ms | 450ms | 800ms
-| memory | 21M | 10M | 20M
-
-
-
-To summarize:
-- going from GF to gfcc is a major win in both code size and efficiency
-- going from Haskell to C++ interpreter is not a win yet, because of a space
- leak in the C++ version
-
-
-
-==Some things to do==
-
-Interpreter in Java.
-
-Parsing via MCFG
-- the FCFG format can possibly be simplified
-- parser grammars should be saved in files to make interpreters easier
-
-
-Hand-written parsers for GFCC grammars to reduce code size
-(and efficiency?) of interpreters.
-
-Binary format and/or file compression of GFCC output.
-
-Syntax editor based on GFCC.
-
-Rewriting of resource libraries in order to exploit the
-word-suffix sharing better (depth-one tables, as in FM).
-
-
-