From bc14a56f83402f73acd6540c9ec3cd278ddf986e Mon Sep 17 00:00:00 2001 From: Meng Weng Wong Date: Fri, 25 Feb 2022 14:31:10 +0800 Subject: "now try this" instructions for people flailing with Apple Silicon M1 --- src/runtime/python/README.org | 166 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 166 insertions(+) create mode 100644 src/runtime/python/README.org (limited to 'src/runtime') diff --git a/src/runtime/python/README.org b/src/runtime/python/README.org new file mode 100644 index 000000000..8885694b5 --- /dev/null +++ b/src/runtime/python/README.org @@ -0,0 +1,166 @@ +* INSTALL + +You will need the python-devel package or similar. + +You must have installed the PGF C runtime (see ../c/INSTALL) + +#+begin_src sh + $ python setup.py build + $ sudo python setup.py install +#+end_src + +* Apple Silicon + +The following install instructions were written with the following config in mind: + +| OS | Hardware | GF | +|-----------------------+----------+--------------------------+ +| MacOS Monterey 12.2.1 | Apple M1 | 3.11 from binary package | + +We assume that you may have installed GF as a binary package downloaded from Github. + +From that starting point, try all the solutions below, in sequence, until you achieve success. + +** Validation Goal + +Our goal is to be able to +- run python 3 +- import pgf +- type "pgf." +- hit tab + +and get this: + +#+begin_example + >>> import pgf + >>> pgf. + pgf.BIND( pgf.Concr( pgf.Iter( pgf.PGFError( pgf.Type( pgf.readExpr( pgf.readType( + pgf.Bracket( pgf.Expr( pgf.PGF( pgf.ParseError( pgf.TypeError( pgf.readPGF( +#+end_example + +When that works, we can consider the Python PGF bindings to be installed successfully. + +** The GF binary package won't install + +We assume you've tried [[https://github.com/GrammaticalFramework/gf-core/releases][downloading a binary package]]. + +If MacOS is being secure, go to System Preferences, Security & Privacy, General, and click Open Anyway. + +** gu/mem.h file not found + +Maybe you tried running something like ~pip install pgf~ or ~pip3 install pgf~. + +Did you get this error? + +#+begin_example + python3 setup.py build + running build + running build_ext + creating build/temp.macosx-12-arm64-3.9 + clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c pypgf.c -o build/temp.macosx-12-arm64-3.9/pypgf.o -std=c99 + pypgf.c:5:10: fatal error: 'gu/mem.h' file not found + #include + ^~~~~~~~~~ + 1 error generated. + error: command '/usr/bin/clang' failed with exit code 1 +#+end_example + +Solution: + +#+begin_example + $ EXTRA_INCLUDE_DIRS=/usr/local/include EXTRA_LIB_DIRS=/usr/local/lib pip install pgf +#+end_example + +This should tell the build where to find the include and lib files it needs to compile: + +#+begin_example + $ ls /usr/local/include/gu + assert.h choice.h enum.h file.h hash.h map.h out.h seq.h sysdeps.h utf8.h + bits.h defs.h exn.h fun.h in.h mem.h prime.h string.h ucs.h variant.h + + $ ls /usr/local/lib/libgu* + /usr/local/lib/libgu.0.dylib /usr/local/lib/libgu.a /usr/local/lib/libgu.dylib /usr/local/lib/libgu.la +#+end_example + +If those files don't exist, or you get the following error, you will need to rebuild the C runtime. + +** symbol not found in flat namespace + +Did you get this error? + +#+begin_example + Python 3.9.10 (main, Jan 15 2022, 11:40:53) + [Clang 13.0.0 (clang-1300.0.29.3)] on darwin + Type "help", "copyright", "credits" or "license" for more information. + >>> import pgf + Traceback (most recent call last): + File "", line 1, in + ImportError: dlopen(/opt/homebrew/lib/python3.9/site-packages/pgf.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_gu_alloc_variant' +#+end_example + +This may be a sign that you're trying to get binaries and libraries compiled with different compilers to play nicely. We're trying to get three things to align: + +- the Python interpreter +- the C runtime libraries +- the Python pgf libraries + +Solution: + +Maybe your Python isn't the Apple-provided Python. In the above error message we see ~python3~ is provided by Homebrew. Assuming you prefer to keep things this way, we'll try to rebuild things to match your Python. + +Rebuilding needs a C compiler. The Apple-provided system ~clang~ is preferred. If you have multiple ~clang~ compilers installed, try disabling the others. For example, if your ~clang~ was provided by nix, run ~nix-env --uninstall clang~. Similarly for brew. + +Then try rebuilding the C runtime. + +** How to re-build the C runtime + +Maybe the C runtime is missing from ~/usr/local/lib~, or maybe the version you have installed is causing the "symbol not found" error. + +Build the C runtime by following the instructions in ~gf-core/src/runtime/c/INSTALL~. + +After a successful ~make install~, rebuild the Python bindings. + +** configure: error: cpu aarch64 not supported + +When building the C runtime, did you get this error? + +#+begin_example + configure: error: cpu aarch64 not supported +#+end_example + +Solution: edit ~src/runtime/c/configure.ac~ to add ~aarch*~ to the ~arm64~ line. + +#+begin_example + aarch* | arm*) cpu=arm; AC_DEFINE(LIGHTNING_ARM, 1, +#+end_example + +** How to re-build the Python bindings using pip + +Sometimes a ~pip install pgf~ will decline to recompile, because a cached wheel exists. + +To return to a more pristine state, + +#+begin_example + pip uninstall pgf + pip cache remove pgf +#+end_example + +You may need to ~sudo~ some of the above commands. + +Then you can repeat + +#+begin_example + $ EXTRA_INCLUDE_DIRS=/usr/local/include EXTRA_LIB_DIRS=/usr/local/lib pip install pgf +#+end_example + +** How to re-build the Python bindings manually + +If the ~pip install pgf~ just isn't working, try building it directly in ~gf-core/src/runtime/python~: + +#+begin_example + $ python setup.py build + $ sudo python setup.py install +#+end_example + +You may need to add the ~EXTRA~ environment prefixes as shown in previous commands. + -- cgit v1.2.3 From b53a102c98ec9693a5cbecb9ade08bc1a1d07742 Mon Sep 17 00:00:00 2001 From: Meng Weng Wong Date: Fri, 25 Feb 2022 14:56:50 +0800 Subject: if this PR is accepted we don't need these instructions --- src/runtime/python/README.org | 14 -------------- 1 file changed, 14 deletions(-) (limited to 'src/runtime') diff --git a/src/runtime/python/README.org b/src/runtime/python/README.org index 8885694b5..86584dc01 100644 --- a/src/runtime/python/README.org +++ b/src/runtime/python/README.org @@ -120,20 +120,6 @@ Build the C runtime by following the instructions in ~gf-core/src/runtime/c/INST After a successful ~make install~, rebuild the Python bindings. -** configure: error: cpu aarch64 not supported - -When building the C runtime, did you get this error? - -#+begin_example - configure: error: cpu aarch64 not supported -#+end_example - -Solution: edit ~src/runtime/c/configure.ac~ to add ~aarch*~ to the ~arm64~ line. - -#+begin_example - aarch* | arm*) cpu=arm; AC_DEFINE(LIGHTNING_ARM, 1, -#+end_example - ** How to re-build the Python bindings using pip Sometimes a ~pip install pgf~ will decline to recompile, because a cached wheel exists. -- cgit v1.2.3 From 862aeb5d9bbd04feade6dccef3a98e3187f89999 Mon Sep 17 00:00:00 2001 From: Inari Listenmaa Date: Sat, 5 Mar 2022 13:41:45 +0800 Subject: Update base <4.15 to <4.16 for tests + pgf*.cabal --- gf.cabal | 2 +- src/runtime/haskell-bind/pgf2.cabal | 2 +- src/runtime/haskell/pgf.cabal | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) (limited to 'src/runtime') diff --git a/gf.cabal b/gf.cabal index c8386d3a8..53548cd40 100644 --- a/gf.cabal +++ b/gf.cabal @@ -400,7 +400,7 @@ test-suite gf-tests main-is: run.hs hs-source-dirs: testsuite build-depends: - base >= 4.9.1 && < 4.15, + base >= 4.9.1 && < 4.16, Cabal >= 1.8, directory >= 1.3.0 && < 1.4, filepath >= 1.4.1 && < 1.5, diff --git a/src/runtime/haskell-bind/pgf2.cabal b/src/runtime/haskell-bind/pgf2.cabal index 1d5f61991..83c990bf3 100644 --- a/src/runtime/haskell-bind/pgf2.cabal +++ b/src/runtime/haskell-bind/pgf2.cabal @@ -26,7 +26,7 @@ library PGF2.Expr, PGF2.Type build-depends: - base >= 4.9.1 && < 4.15, + base >= 4.9.1 && < 4.16, containers >= 0.5.7 && < 0.7, pretty >= 1.1.3 && < 1.2 default-language: Haskell2010 diff --git a/src/runtime/haskell/pgf.cabal b/src/runtime/haskell/pgf.cabal index 41e67f6ae..cb5841a1c 100644 --- a/src/runtime/haskell/pgf.cabal +++ b/src/runtime/haskell/pgf.cabal @@ -15,7 +15,7 @@ library default-language: Haskell2010 build-depends: array >= 0.5.1 && < 0.6, - base >= 4.9.1 && < 4.15, + base >= 4.9.1 && < 4.16, bytestring >= 0.10.8 && < 0.11, containers >= 0.5.7 && < 0.7, ghc-prim >= 0.5.0 && < 0.7, -- cgit v1.2.3 From 221597bd79054af47d3d5866d74a03ee9d7cd6e6 Mon Sep 17 00:00:00 2001 From: Andreas Källberg Date: Mon, 11 Oct 2021 10:33:43 +0800 Subject: When profiling, don't add cost centres in Data.Binary.Get This change speeds up profiling by an order of magnitude. Without it, the >>= function for Get dominates runtime completely during profiling. --- src/runtime/haskell/Data/Binary/Get.hs | 2 ++ 1 file changed, 2 insertions(+) (limited to 'src/runtime') diff --git a/src/runtime/haskell/Data/Binary/Get.hs b/src/runtime/haskell/Data/Binary/Get.hs index 54f17ae95..07a3db40e 100644 --- a/src/runtime/haskell/Data/Binary/Get.hs +++ b/src/runtime/haskell/Data/Binary/Get.hs @@ -1,4 +1,6 @@ {-# LANGUAGE CPP, MagicHash #-} +-- This module makes profiling a lot slower +{-# OPTIONS_GHC -fno-prof-auto #-} -- for unboxed shifts ----------------------------------------------------------------------------- -- cgit v1.2.3 From b0b2a06f3b3485e41ebfb66c2d4ebbd5dd8e6119 Mon Sep 17 00:00:00 2001 From: Andreas Källberg Date: Tue, 3 May 2022 13:10:29 +0800 Subject: Improve comment --- src/runtime/haskell/Data/Binary/Get.hs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'src/runtime') diff --git a/src/runtime/haskell/Data/Binary/Get.hs b/src/runtime/haskell/Data/Binary/Get.hs index 07a3db40e..895789061 100644 --- a/src/runtime/haskell/Data/Binary/Get.hs +++ b/src/runtime/haskell/Data/Binary/Get.hs @@ -1,5 +1,5 @@ {-# LANGUAGE CPP, MagicHash #-} --- This module makes profiling a lot slower +-- This module makes profiling a lot slower, so don't add automatic cost centres {-# OPTIONS_GHC -fno-prof-auto #-} -- for unboxed shifts -- cgit v1.2.3