diff options
| author | hallgren <hallgren@chalmers.se> | 2014-08-22 12:13:04 +0000 |
|---|---|---|
| committer | hallgren <hallgren@chalmers.se> | 2014-08-22 12:13:04 +0000 |
| commit | 6bde56bef77c15cec6f5030b158c940d2f99725a (patch) | |
| tree | 7e28a01136c7fb1f0942d10a308d1a00206740ce | |
| parent | 8dfaf2ef65915e7ac91139155d60df85ed66adbb (diff) | |
src/www/robots.txt: robot exclusion file for the GF cloud service.
To reduce potential server load caused by search bots.
| -rw-r--r-- | src/www/robots.txt | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/src/www/robots.txt b/src/www/robots.txt new file mode 100644 index 000000000..5b3ee3354 --- /dev/null +++ b/src/www/robots.txt @@ -0,0 +1,4 @@ +User-agent: * +Disallow: /grammars +Disallow: /robust +Disallow: /*.pgf |