1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD>
<META NAME="generator" CONTENT="http://txt2tags.org">
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">
<LINK REL="stylesheet" TYPE="text/css" HREF="../css/style.css">
<TITLE>GF Developers Guide</TITLE>
</HEAD><BODY BGCOLOR="white" TEXT="black">
<CENTER>
<H1><a href="../"><IMG src="../doc/Logos/gf0.png"></a>GF Developers Guide</H1>
<FONT SIZE="4"><I>Authors: Björn Bringert, Krasimir Angelov and Thomas Hallgren</I></FONT><BR>
<FONT SIZE="4">Last update: 2014-06-17, 16:14</FONT>
</CENTER>
<P></P>
<HR NOSHADE SIZE=1>
<P></P>
<UL>
<LI><A HREF="#toc1">Before you start</A>
<LI><A HREF="#toc2">Setting up your system for building GF</A>
<UL>
<LI><A HREF="#toc3">The Haskell Platform</A>
<LI><A HREF="#toc4">Darcs</A>
<LI><A HREF="#toc5">The haskeline library</A>
</UL>
<LI><A HREF="#toc6">Getting the source</A>
<UL>
<LI><A HREF="#toc7">Read-only access</A>
<LI><A HREF="#toc8">Read-write access</A>
<LI><A HREF="#toc9">Further information about Darcs</A>
</UL>
<LI><A HREF="#toc10">Compilation from source with Cabal</A>
<UL>
<LI><A HREF="#toc11">Configure</A>
<LI><A HREF="#toc12">Build</A>
<LI><A HREF="#toc13">Install</A>
<LI><A HREF="#toc14">Clean</A>
<LI><A HREF="#toc15">Known problems with Cabal</A>
</UL>
<LI><A HREF="#toc16">Compilation with make</A>
<LI><A HREF="#toc17">Compiling GF with C run-time system support</A>
<LI><A HREF="#toc18">Creating .deb packages for Ubuntu</A>
<UL>
<LI><A HREF="#toc19">Preparations</A>
<LI><A HREF="#toc20">Creating the package</A>
</UL>
<LI><A HREF="#toc21">Creating .rpm packages for Fedora</A>
<LI><A HREF="#toc22">Running the testsuite</A>
</UL>
<P></P>
<HR NOSHADE SIZE=1>
<P></P>
<A NAME="toc1"></A>
<H2>Before you start</H2>
<P>
This guide is intended for people who want to contribute to
the development of the GF compiler or the Resource Grammar Library. If
you are a GF user who just wants to download and install GF
(e.g to develop your own grammars), the simpler guide on
<A HREF="../download/index.html">the GF download page</A> should be sufficient.
</P>
<A NAME="toc2"></A>
<H2>Setting up your system for building GF</H2>
<P>
To build GF from source you need to install some tools on your
system: the <I>Haskell Platform</I>, <I>Darcs</I> and the <I>Haskeline library</I>.
</P>
<P>
<B>On Linux</B> the best option is to install the tools via the standard
software distribution channels, i.e. by using the <I>Software Center</I>
in Ubuntu or the corresponding tool in other popular Linux distributions.
Or, from a Terminal window, the following command should be enough:
</P>
<UL>
<LI>On Ubuntu: <CODE>sudo apt-get install haskell-platform darcs libghc6-haskeline-dev</CODE>
<LI>On Fedora: <CODE>sudo yum install haskell-platform darcs ghc-haskeline-devel</CODE>
</UL>
<P>
<B>On Mac OS and Windows</B>, the tools can be downloaded from their respective
web sites, as described below.
</P>
<A NAME="toc3"></A>
<H3>The Haskell Platform</H3>
<P>
GF is written in Haskell, so first of all you need
the <I>Haskell Platform</I>, version 2012.4.0.0 or 2013.2.0.0. Downloads
and installation instructions are available from here:
</P>
<P>
<A HREF="http://hackage.haskell.org/platform/">http://hackage.haskell.org/platform/</A>
</P>
<P>
Once you have installed the Haskell Platform, open a terminal
(Command Prompt on Windows) and try to execute the following command:
</P>
<PRE>
$ ghc --version
</PRE>
<P>
This command should show you which version of GHC you have. If the installation
of the Haskell Platform was successful you should see a message like:
</P>
<PRE>
The Glorious Glasgow Haskell Compilation System, version 7.6.3
</PRE>
<P>
Other required tools included in the Haskell Platform are
<A HREF="http://www.haskell.org/cabal/">Cabal</A>,
<A HREF="http://www.haskell.org/alex/">Alex</A>
and
<A HREF="http://www.haskell.org/happy/">Happy</A>.
</P>
<A NAME="toc4"></A>
<H3>Darcs</H3>
<P>
To get the GF source code, you also need <I>Darcs</I>, version 2 or later. Darcs
is a distributed version control system, see <A HREF="http://darcs.net/">http://darcs.net/</A> for
more information. There are precompiled packages for many platforms
available and source code if you want to compile it yourself. Darcs
is also written in Haskell and so you can use GHC to compile it.
</P>
<A NAME="toc5"></A>
<H3>The haskeline library</H3>
<P>
GF uses <I>haskeline</I> to enable command line editing in the GF shell.
This should work automatically on Mac OS and Windows, but on Linux one
extra step is needed to make sure the C libraries required by <I>haskeline</I>
are installed. Here is one way to do this:
</P>
<UL>
<LI>On Ubuntu: <CODE>sudo apt-get install libghc-haskeline-dev</CODE>
<LI>On Fedora: <CODE>sudo yum install ghc-haskeline-devel</CODE>
</UL>
<A NAME="toc6"></A>
<H2>Getting the source</H2>
<P>
Once you have all tools in place you can get the GF source code. If you
just want to compile and use GF then it is enough to have read-only
access. It is also possible to make changes in the source code but if you
want these changes to be applied back to the main source repository you will
have to send the changes to us. If you plan to work continuously on
GF then you should consider getting read-write access.
</P>
<A NAME="toc7"></A>
<H3>Read-only access</H3>
<H4>Getting a fresh copy for read-only access</H4>
<P>
Anyone can get the latest development version of GF by running (all on one line):
</P>
<PRE>
$ darcs get --lazy --set-scripts-executable http://www.grammaticalframework.org/ gf
</PRE>
<P>
This will create a directory called <CODE>gf</CODE> in the current
directory.
</P>
<H4>Updating your copy</H4>
<P>
To get all new patches from the main repo:
</P>
<PRE>
$ darcs pull -a
</PRE>
<P>
This can be done anywhere in your local repository, i.e. in the <CODE>gf</CODE>
directory, or any of its subdirectories.
Without <CODE>-a</CODE>, you can choose which patches you want to get.
</P>
<A NAME="record"></A>
<H4>Recording local changes</H4>
<P>
Since every copy is a repository, you can have local version control
of your changes.
</P>
<P>
If you have added files, you first need to tell your local repository to
keep them under revision control:
</P>
<PRE>
$ darcs add file1 file2 ...
</PRE>
<P>
To record changes, use:
</P>
<PRE>
$ darcs record
</PRE>
<P>
This creates a patch against the previous version and stores it in your
local repository. You can record any number of changes before
pushing them to the main repo. In fact, you don't have to push them at
all if you want to keep the changes only in your local repo.
</P>
<P>
If you think there are too many questions about what to record, you
can use the <CODE>-a</CODE> flag to <CODE>record</CODE>. Or answer <CODE>a</CODE> to the first
question. Both of these record all the changes you have in your local
repository.
</P>
<H4>Submitting patches</H4>
<P>
If you are using read-only access, send your patches by email to
someone with write-access. First record your changes in your local
repository, as described above. You can send any number of recorded
patches as one patch bundle. You create the patch bundle with:
</P>
<PRE>
$ darcs send -o mypatch.patch
$ gzip mypatch.patch
</PRE>
<P>
(where <CODE>mypatch</CODE> is hopefully replaced by a slightly more
descriptive name). Since some e-mail setups change text attachments
(most likely by changing the newline characters) you need to send
the patch in some compressed format, such as GZIP, BZIP2 or ZIP.
</P>
<P>
Send it as an e-mail attachment. If you have
sendmail or something equivalent installed, it is possible to send the
patch directly from darcs. If so, replace <CODE>-o mypatch.patch</CODE> with
<CODE>--to=EMAIL</CODE> where <CODE>EMAIL</CODE> is the address to send it to.
</P>
<A NAME="toc8"></A>
<H3>Read-write access</H3>
<P>
If you have a user account on <CODE>www.grammaticalframework.org</CODE>, you can
get read-write access over SSH to the GF repository.
</P>
<H4>Getting a fresh copy</H4>
<P>
Get your copy with (all on one line),
replacing <CODE>bringert</CODE> with your own username on <CODE>www.grammaticalframework.org</CODE>:
</P>
<PRE>
$ darcs get --lazy --set-scripts-executable bringert@www.grammaticalframework.org:/usr/local/www/GF/ gf
</PRE>
<P>
The option <CODE>--lazy</CODE> means that darcs defers downloading all the
history for the repository. This saves space, bandwidth and CPU time,
and most people don't need the full history of all changes in the
past.
</P>
<H4>Updating your copy</H4>
<P>
Get all new patches from the main repo:
</P>
<PRE>
$ darcs pull -a
</PRE>
<P>
Without <CODE>-a</CODE>, you can choose which patches you want to get.
</P>
<H4>Commit your changes</H4>
<P>
There are two steps to commiting a change to the main repo. First you
have to record the changes that you want to commit
(see <A HREF="#record">Recording local changes</A> above), then you push them
to the main repo. If you are using ssh-access, all you need to do is:
</P>
<PRE>
$ darcs push
</PRE>
<P>
If you use the <CODE>-a</CODE> flag to push, all local patches which are not in
the main repo are pushed.
</P>
<H4>Apply a patch from someone else</H4>
<P>
Use:
</P>
<PRE>
$ darcs apply < mypatch.patch
</PRE>
<P>
This applies the patch to your local repository. To commit it to the
main repo, use <CODE>darcs push</CODE>.
</P>
<A NAME="toc9"></A>
<H3>Further information about Darcs</H3>
<P>
For more info about what you can do with darcs, see <A HREF="http://darcs.net/manual/">http://darcs.net/manual/</A>
</P>
<A NAME="toc10"></A>
<H2>Compilation from source with Cabal</H2>
<P>
The build system of GF is based on <I>Cabal</I>, which is part of the
Haskell Platform, so no extra steps are needed to install it. In the simplest
case, all you need to do to compile and install GF, after downloading the
source code as described above, is
</P>
<PRE>
$ cd gf
$ cabal install
</PRE>
<P>
This will automatically download any additional Haskell libraries needed to
build GF. If this is the first time you use Cabal, you might need to run
<CODE>cabal update</CODE> first, to update the list of available libraries.
</P>
<P>
If you want more control, the process can also be split up into the usual
<I>configure</I>, <I>build</I> and <I>install</I> steps.
</P>
<A NAME="toc11"></A>
<H3>Configure</H3>
<P>
During the configuration phase Cabal will check that you have all
necessary tools and libraries needed for GF. The configuration is
started by the command:
</P>
<PRE>
$ cabal configure
</PRE>
<P>
If you don't see any error message from the above command then you
have everything that is needed for GF. You can also add the option
<CODE>-v</CODE> to see more details about the configuration.
</P>
<P>
You can use <CODE>cabal configure --help</CODE> to get a list of configuration options.
</P>
<A NAME="toc12"></A>
<H3>Build</H3>
<P>
The build phase does two things. First it builds the GF compiler from
the Haskell source code and after that it builds the GF Resource Grammar
Library using the already build compiler. The simplest command is:
</P>
<PRE>
$ cabal build
</PRE>
<P>
Again you can add the option <CODE>-v</CODE> if you want to see more details.
</P>
<P>
Sometimes you just want to work on the GF compiler and don't want to
recompile the resource library after each change. In this case use
this extended command:
</P>
<PRE>
$ cabal build rgl-none
</PRE>
<P>
The resource library could also be compiled in two modes: with present
tense only and with all tenses. By default it is compiled with all
tenses. If you want to use the library with only present tense you can
compile it in this special mode with the command:
</P>
<PRE>
$ cabal build present
</PRE>
<P>
You could also control which languages you want to be recompiled by
adding the option <CODE>langs=list</CODE>. For example the following command
will compile only the English and the Swedish language:
</P>
<PRE>
$ cabal build langs=Eng,Swe
</PRE>
<A NAME="toc13"></A>
<H3>Install</H3>
<P>
After you have compiled GF you need to install the executable and libraries
to make the system usable.
</P>
<PRE>
$ cabal copy
$ cabal register
</PRE>
<P>
This command installs the GF compiler for a single user, in the standard
place used by Cabal.
On Linux and Mac this could be <CODE>$HOME/.cabal/bin</CODE>.
On Mac it could also be <CODE>$HOME/Library/Haskell/bin</CODE>.
On Windows this is <CODE>C:\Program Files\Haskell\bin</CODE>.
</P>
<P>
The compiled GF Resource Grammar Library will be installed
under the same prefix, e.g. in
<CODE>$HOME/.cabal/share/gf-3.3.3/lib</CODE> on Linux and
in <CODE>C:\Program Files\Haskell\gf-3.3.3\lib</CODE> on Windows.
</P>
<P>
If you want to install in some other place then use the <CODE>--prefix</CODE>
option during the configuration phase.
</P>
<A NAME="toc14"></A>
<H3>Clean</H3>
<P>
Sometimes you want to clean up the compilation and start again from clean
sources. Use the clean command for this purpose:
</P>
<PRE>
$ cabal clean
</PRE>
<A NAME="toc15"></A>
<H3>Known problems with Cabal</H3>
<P>
Some versions of Cabal (at least version 1.16) seem to have a bug that can
cause the following error:
</P>
<PRE>
Configuring gf-3.x...
setup: Distribution/Simple/PackageIndex.hs:124:8-13: Assertion failed
</PRE>
<P>
The exact cause of this problem is unclear, but it seems to happen
during the configure phase if the same version of GF is already installed,
so a workaround is to remove the existing installation with
</P>
<PRE>
ghc-pkg unregister gf
</PRE>
<P>
You can check with <CODE>ghc-pkg list gf</CODE> that it is gone.
</P>
<A NAME="toc16"></A>
<H2>Compilation with make</H2>
<P>
If you feel more comfortable with Makefiles then there is a thin Makefile
wrapper arround Cabal for you. If you just type:
</P>
<PRE>
$ make
</PRE>
<P>
the configuration phase will be run automatically if needed and after that
the sources will be compiled. If you don't want to compile the resource library
every time then you can use:
</P>
<PRE>
$ make gf
</PRE>
<P>
For installation use:
</P>
<PRE>
$ make install
</PRE>
<P>
For cleaning:
</P>
<PRE>
$ make clean
</PRE>
<A NAME="toc17"></A>
<H2>Compiling GF with C run-time system support</H2>
<P>
The C run-time system is separate implementation of the PGF run-time services
that makes it possible to work with very large, ambiguous grammars, using
probabilistic models to obtain probable parses.
</P>
<P>
Support for calling the C run-time library is available in the web API
provided by <CODE>gf -server</CODE>, if C run-time support is enabled.
</P>
<P>
To enable the C run-time system, first go to the <CODE>src/runtime/c</CODE> directory
and follow the instructions in the <CODE>INSTALL</CODE> file to install the C run-time
system. Once this is done, you can install GF with C run-time support by
doing
</P>
<PRE>
cabal install -fserver -fc-runtime
</PRE>
<P>
from the top directory.
</P>
<A NAME="toc18"></A>
<H2>Creating .deb packages for Ubuntu</H2>
<P>
This was tested on Ubuntu 14.04 for the release of GF 3.6, and the
resulting <CODE>.deb</CODE> packages appears to work on Ubuntu 12.04, 13.10 and 14.04.
</P>
<P>
Under Ubuntu, Haskell executables are statically linked against other Haskell
libraries, so the .deb packages are fairly self-contained.
</P>
<A NAME="toc19"></A>
<H3>Preparations</H3>
<PRE>
sudo apt-get install dpkg-dev debhelper
</PRE>
<A NAME="toc20"></A>
<H3>Creating the package</H3>
<P>
Make sure the <CODE>debian/changelog</CODE> starts with an entry that describes the
version you are building. Then run
</P>
<PRE>
make deb
</PRE>
<A NAME="toc21"></A>
<H2>Creating .rpm packages for Fedora</H2>
<P>
This is possible, but the procedure has not been automated.
It involves using the cabal-rpm tool,
</P>
<PRE>
sudo yum install cabal-rpm
</PRE>
<P>
and following the Fedora guide
<A HREF="http://fedoraproject.org/wiki/How_to_create_an_RPM_package">How to create an RPM package</A>.
</P>
<P>
Under Fedora, Haskell executables are dynamically linked against other Haskell
libraries, so <CODE>.rpm</CODE> packages for all Haskell libraries that GF depends on
are required. Most of them are already available in the Fedora distribution,
but a few of them might have to be built and distributed along with
the GF <CODE>.rpm</CODE> package.
When building <CODE>.rpm</CODE> packages for GF 3.4, we also had to build <CODE>.rpm</CODE>s for
<CODE>fst</CODE> and <CODE>httpd-shed</CODE>.
</P>
<A NAME="toc22"></A>
<H2>Running the testsuite</H2>
<P>
<B>NOTE:</B> The test suite has not been maintained recently, so expect many
tests to fail.
</P>
<P>
GF has testsuite. It is run with the following command:
</P>
<PRE>
$ cabal test
</PRE>
<P>
The testsuite architecture for GF is very simple but still very flexible.
GF by itself is an interpreter and could execute commands in batch mode.
This is everything that we need to organize a testsuite. The root of the
testsuite is the testsuite/ directory. It contains subdirectories which
themself contain GF batch files (with extension .gfs). The above command
searches the subdirectories of the testsuite/ directory for files with extension
.gfs and when it finds one it is executed with the GF interpreter.
The output of the script is stored in file with extension .out and is compared
with the content of the corresponding file with extension .gold, if there is one.
If the contents are identical the command reports that the test was passed successfully.
Otherwise the test had failed.
</P>
<P>
Every time when you make some changes to GF that have to be tested, instead of
writing the commands by hand in the GF shell, add them to one .gfs file in the testsuite
and run the test. In this way you can use the same test later and we will be sure
that we will not incidentaly break your code later.
</P>
<P>
If you don't want to run the whole testsuite you can write the path to the subdirectory
in which you are interested. For example:
</P>
<PRE>
$ cabal test testsuite/compiler
</PRE>
<P>
will run only the testsuite for the compiler.
</P>
<!-- html code generated by txt2tags 2.6 (http://txt2tags.org) -->
<!-- cmdline: txt2tags -thtml ./doc/gf-developers.t2t -->
</BODY></HTML>
|
