| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
Because the prompt included the name of the abstract syntax, the loading
of the PGF was forced even if -retain was used. Even worse,
if an error occured while loading the PGF, it was repeated and caught
every time the prompt was printed, creating an infite loop. The solution
is to not print the name of the abstract syntax when the grammar is
imported with -retain, which is the way things were before anyway.
|
|
Renamed appIOE to tryIOE (it is analogous to 'try' in the standard libraries).
Removed unused IOE operations & documented the remaining ones.
Removed/simplified superfluous uses of IOE operations.
|
|
|
|
|
|
parsing line by line
|
|
The def rules are now compiled to byte code by the compiler and then to
native code by the JIT compiler in the runtime. Not all constructions
are implemented yet. The partial implementation is now in the repository
but it is not activated by default since this requires changes in the
PGF format. I will enable it only after it is complete.
|
|
All compiler modules now use GF.Text.Pretty instead of Text.PrettyPrint
|
|
GF.Infra.Location for modularity
GF.Text.Pretty provides the class Pretty and overloaded versions of the pretty
printing combinators in Text.PrettyPrint, allowing pretty printable values to
be used directly instead of first having to convert them to Doc with functions
like text, int, char and ppIdent. Some modules have been converted to use
GF.Text.Pretty, but not all. Precedences could be added to simplify the pretty
printers for terms and patterns.
GF.Infra.Location contains the types Location and L, factored out from
GF.Grammar.Grammar, and the class HasSourcePath. This allowed the import
of GF.Grammar.Grammar to be removed from GF.Infra.CheckM, making it more
like a pure library module.
|
|
PGF exports the public, stable API.
PGF.Internal exports additional things needed in the GF compiler & shell,
including the nonstardard version of Data.Binary.
|
|
source code. This makes it quick and lightweight to compile big grammars such as the Berkley grammar
|
|
using GF.Grammar.Parser just like the ordinary GF grammars. Furthermore now GF.Speech.CFG is moved to GF.Grammar.CFG. The new module is used by both the speech conversion utils and by the compiler for CFG grammars. The parser for CFG now consumes a lot less memory and can be used with grammars with more than 4 000 000 productions.
|
|
When running a command like
gf -make L_1.gf ... L_n.gf
gf now avoids recreating the target PGF file if it already exists and is
up-to-date.
gf still reads all required .gfo files, so significant additional speed
improvements are still possible. This could be done by reading .gfo files
more lazily...
|
|
Trees are not generated with increasing depth.
|
|
1. No temporary files are created.
2. The output of a system command is read lazily, making it feasible to
process large or even infinite output, e.g. the following works as
expected:
? "yes" | ? "head -5" | ps -lextext
|
|
The system_pipe (aka "?") command creates a temporary file _tmpi containing
the input of the system command. It *both* appends _tmpi as an extra argument
to the system command line *and* adds an input redirection "< _tmpi". (It
also uses and output redirection "> _tmpo" to captures the output of the
command.)
With this patch, the _tmpi argument is no longer appended to the command line.
This allows system_pipe to work with pure filters, such as the "tr" commands,
but it will no longer work with commands that require an input file name.
(It is possible to use write_file instead...)
TODO: it would also be fairly easy to eliminate the creation of the _tmpi and
_tmpo files altogether.
|
|
Trailing spaces caused the command line parse to be ambiguous, and
ambiguous parses were rejected by function readCommandLine, causing
the cryptic error message "command not parsed".
|
|
The only use of PGF.Tree outside the PGF library was in GF.Command.Commands,
and it was eliminated by using PGF.Expr directly instead.
PGF.Paraphrase still uses PGF.Tree.
|
|
later is now retrained. Once the grammar is compiled with the .probs file now it doesn't need anything more to do robust parsing. The robustness itself is controlled by the flags 'heuristic_search_factor', 'meta_prob' and 'meta_token_prob' in ParseEngAbs.gf
|
|
|
|
'l' requires type checking and trees with metavariable function heads don't type check. This will hopefully be a temporary command.
|
|
+ References to modules under src/compiler have been eliminated from the PGF
library (under src/runtime/haskell). Only two functions had to be moved (from
GF.Data.Utilities to PGF.Utilities) to make this possible, other apparent
dependencies turned out to be vacuous.
+ In gf.cabal, the GF executable no longer directly depends on the PGF library
source directory, but only on the exposed library modules. This means that
there is less duplication in gf.cabal and that the 30 modules in the
PGF library will no longer be compiled twice while building GF.
To make this possible, additional PGF library modules have been exposed, even
though they should probably be considered for internal use only. They could
be collected in a PGF.Internal module, or marked as "unstable", to make
this explicit.
+ Also, by using the -fwarn-unused-imports flag, ~220 redundant imports were
found and removed, reducing the total number of imports by ~15%.
|
|
command in the shell now can linearize discontinuous phrases
|
|
* In the shell, the new command tt (to_trie) merges a list of trees into a
trie and prints it in a readable way, where unique subtrees are marked with
a "*" and alternative subtrees are marked with numbers.
* In the PGF web service, adding the parameter trie=yes to the parse and
translate commands augments the JSON output with a trie.
Example to try in the shell:
Phrasebook> p -lang=Eng "your son waits for you" | tt
|
|
|
|
The fact that identifiers are represented as ByteStrings is now an internal
implentation detail in module GF.Infra.Ident. Conversion between ByteString
and identifiers is only needed in the lexer and the Binary instances.
|
|
|
|
|
|
|
|
Also in Commands.hs: be explicit about things imported from the PGF library
that are not in the public API.
Also a couple of haddock documentation fixes.
|
|
|
|
|
|
-subtrees <bigtree> | l -treebank for debugging the lin of a big tree
|
|
lexicon, as required by Xerox lexc
|
|
|
|
The output from commands is represented as ([Expr],String), where the [Expr] is
used when data is piped between commands and the String is used for the final
output. The String can represent the same list of trees as the [Expr] and/or
contain diagnostic information.
Sometimes the data that is piped between commands is not a list of trees, but
e.g. a string or a list of strings. In those cases, functions like fromStrings
and toStrings are used to encode the data as a [Expr].
This patch introduces a newtype for CommandOutput and collects the functions
dealing with command output in one place to make it clearer what is going on.
It also makes it easier to change to a more direct representation of piped
data, and make pipes more "type safe", if desired.
|
|
+ The restrictions on arbitrary IO when GF is running in restricted mode is now
enforced in the types.
+ This hopefully also solves an intermittent problem when accessing the GF
shell through the web API provided by gf -server. This was visible in the
Simple Translation Tool and probably caused by some low-level bug in the
GHC IO libraries.
|
|
The dependency on PGFEnv has been moved from the list to the exec function of
the commands in the list. This means that the help command no longer needs
to generate a new list of commands and that the state of the shell
(type GF.Command.Interpreter.CommandEnv) no longer needs to contain the list
of commands.
|
|
syntax
|
|
|
|
An apparent bug in ghc-7.2.2 causes the type Value to be exported from PGF.Data.
Workaround: restrict the imports from PGF.Data in GF.Command.Abstract and
GF.Compile.GeneratePMCFG to avoid the clash with locally defined type Value.
(ghc-7.0.4 and ghc-7.4.1 appear to be free from this bug.)
|
|
of the different definitions. There is a --tags option which generates a list of all identifiers with their source locations.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
