Giter Club home page Giter Club logo

python-cmake-buildsystem's Introduction

CPython CMake Buildsystem

Overview

A replacement buildsystem for CPython.

This CMake buildsystem has the following advantages:

  • No compiled program for the target architecture is used in the build itself. This makes cross-compiling easier, less error prone, and reduces manual steps.
  • Same build information for all platforms - there's no need to maintain the autotools configuration separately from four different MSVC project files.
  • Support for other build systems and IDE's like Ninja, Sublime Text, and many others.
  • Easily build C-extensions against other C/C++ libraries built with CMake.
  • It's much faster to compile: 7 seconds instead of 58 seconds in my unscientific test.

Usage

How to use this buildsystem:

  1. Checkout the buildsystem
cd ~/scratch
git clone git://github.com/python-cmake-buildsystem/python-cmake-buildsystem
  1. Build
# Unix
cd ~/scratch
mkdir -p python-build && mkdir -p python-install
cd python-build
cmake -DCMAKE_INSTALL_PREFIX:PATH=${HOME}/scratch/python-install ../python-cmake-buildsystem
make -j10
make install

# Windows
cd %HOME%/scratch
mkdir python-build
mkdir python-install
cd python-build
cmake -G "Visual Studio 16 2019" -A x64 -DCMAKE_INSTALL_PREFIX:PATH=%HOME%/scratch/python-install ../python-cmake-buildsystem
cmake --build . --config Release -- /m
cmake --build . --config Release --target INSTALL

Note

By default, the build system will download the python 3.9.17 source from http://www.python.org/ftp/python/

CMake Options

You can pass options to CMake to control the way Python is built. You only need to give each option once - they get saved in CMakeCache.txt. Pass options on the commandline with -DOPTION=VALUE, or use the "ccmake" gui.

PYTHON_VERSION=major.minor.patch (defaults to 3.9.17)
  The version of Python to build.

PYTHON_APPLY_PATCHES=ON|OFF (defaults to ON)
  Apply patches required to build CPython based on the system and compiler
  found when configuring the project. Note that when cross-compiling, patches
  coresponding to the target system are applied.
  Patches can be found in "patches" directory.

CMAKE_BUILD_TYPE=Debug|Release
  Build with debugging symbols or with optimisations.

CMAKE_INSTALL_PREFIX=<path>   (defaults to /usr/local)
  Path in which to install Python.

DOWNLOAD_SOURCES=ON|OFF      (defaults to ON)
  Download, check MD5 sum and extract python sources in the parent directory.
  Source archive is downloaded from http://www.python.org/ftp/python

BUILD_LIBPYTHON_SHARED=ON|OFF (defaults to OFF)
  Build libpython as a shared library (.so or .dll) or a static library
  (.a).

  Note that Python extensions are always built as shared libraries.  On
  Windows it is not possible to build shared .dll extensions against a
  static libpython, so you must build any extensions you want into libpython
  itself (see the BUILTIN flags below).

BUILD_EXTENSIONS_AS_BUILTIN=ON|OFF (defaults to OFF)
  If enabled, all extensions are statically compiled into the built python
  libraries (static and/or shared).

  Note that all previously set BUILTIN_<extension> options are ignored and
  reset to their original value.

WITH_STATIC_DEPENDENCIES=ON|OFF    (defaults to OFF, available only on UNIX)
  If this is set to ON then cmake will compile statically libpython and all
  extensions. External dependencies (ncurses, sqlite, ...) will be builtin
  only if they are available as static libraries.

BUILD_WININST=ON|OFF (only for windows, defaults to ON if not crosscompiling)
  If enabled, build the 'Windows Installer' program for distutils if not
  already provided in the source tree.

BUILD_WININST_ALWAYS=ON|OFF (only for windows, defaults to OFF)
  If enabled, always build 'Windows Installer' program for distutils even
  if it is already provided in the source tree.

INSTALL_DEVELOPMENT=ON|OFF (defaults to ON)
  If enabled, install files required to develop C extensions.

INSTALL_MANUAL=ON|OFF (defaults to ON)
  If enabled, install manuals.

INSTALL_TEST=ON|OFF (defaults to ON)
  If enabled, install test files.

ENABLE_<extension>=ON|OFF     (defaults to ON)
BUILTIN_<extension>=ON|OFF    (defaults to OFF except for POSIX, PWD and
                               NT extensions which are builtin by default)
  These two options control how individual python extensions are built.
  <extension> is the name of the extension in upper case, and without any
  leading underscore (_).  Known extensions for 2.7.12 include:

    ARRAY AUDIOOP BINASCII BISECT BSDDB BZ2 CMATH CODECS_CN CODECS_HK
    CODECS_ISO2022 CODECS_JP CODECS_KR CODECS_TW COLLECTIONS CPICKLE CRYPT
    CSTRINGIO CSV CTYPES CTYPES_TEST CURSES CURSES_PANEL DATETIME DBM
    ELEMENTTREE FCNTL FUNCTOOLS FUTURE_BUILTINS GDBM GRP HASHLIB HEAPQ
    HOTSHOT IO ITERTOOLS JSON LINUXAUDIODEV LOCALE LSPROF LZMA MATH MMAP
    MULTIBYTECODEC MULTIPROCESSING NIS NT OPERATOR OSSAUDIODEV PARSER POSIX
    PWD PYEXPAT RANDOM READLINE RESOURCE SELECT SOCKET SPWD SQLITE3 SSL
    STROP STRUCT SYSLOG TERMIOS TESTCAPI TIME TKINTER UNICODEDATA ZLIB

  All extensions are enabled by default, but some might depend on system
  libraries and will get disabled if they're not available (a list of
  extensions that didn't have all their prerequisites available will be
  printed when you run cmake).

  By default extensions are compiled as separate shared libraries (.so or
  .dll files) and installed in lib/python2.7/lib-dynload.  If you set
  BUILTIN_<extension> to ON then the extension is compiled into libpython
  instead.

USE_LIB64=ON|OFF              (defaults to OFF)
  If this is set to ON then cmake will look for dependencies in lib64 as
  well as lib directories.  Compiled python extensions will also be
  installed into lib64/python2.7/lib-dynload instead of
  lib/python2.7/lib-dynload.

Py_USING_UNICODE             (only for python2, defaults to ON)
  Enable unicode support. By default, ucs2 is used. It can be
  forced to ucs4 setting Py_UNICODE_SIZE to 4.

WITH_C_LOCALE_COERCION       (only for python3.7 and above, default to ON)
  Enable C locale coercion to a UTF-8 based locale.

WITH_DECIMAL_CONTEXTVAR      (only for python3.8 and above, default to ON)
  Build _decimal module using a coroutine-local rather than a thread-local context.

WITH_TRACE_REFS              (only for python3.8 and above, default to OFF)
  Enable tracing references for debugging purpose

WITH_SSL_DEFAULT_SUITES      (only for python3.7 and above, default to "python")
  Override default cipher suites string:
  * python: use Python's preferred selection (default)
  * openssl: leave OpenSSL's defaults untouched
  * STRING: use a custom string, PROTOCOL_SSLv2 ignores the setting

EXTRA_PYTHONPATH=dir1:dir2    (defaults to "")
  Colon (:) separated list of extra directories to add to the compiled-in
  PYTHONPATH.

USE_SYSTEM_LIBRARIES=ON|OFF   (defaults to ON)
  If set to OFF, no attempt to detect system libraries will be done.
  Options documented below allow to enable/disable detection of particular
  libraries.

USE_SYSTEM_Curses=ON|OFF      (defaults to ON)
  If set to OFF, no attempt to detect Curses libraries will be done.
  Associated python extensions are: CURSES, CURSES_PANEL, READLINE
  Following CMake variables can manually be set: CURSES_LIBRARIES, PANEL_LIBRARIES

USE_SYSTEM_EXPAT=ON|OFF       (defaults to ON)
  If set to OFF, no attempt to detect Expat libraries will be done.
  Associated python extensions are: ELEMENTTREE, PYEXPAT
  Following CMake variables can manually be set: EXPAT_LIBRARIES, EXPAT_INCLUDE_DIRS

USE_SYSTEM_LibFFI=ON|OFF       (defaults to ON)
  If set to OFF, no attempt to detect libffi libraries will be done.
  Associated python extensions are: CTYPES
  Following CMake variables can manually be set: LibFFI_LIBRARY and LibFFI_INCLUDE_DIR

USE_SYSTEM_OpenSSL=ON|OFF     (defaults to ON)
  If set to OFF, no attempt to detect OpenSSL libraries will be done.
  Associated python extensions are: HASHLIB, SSL, MD5, SHA, SHA256, SHA512
  Following CMake variables can manually be set: OPENSSL_INCLUDE_DIR, OPENSSL_LIBRARIES
  If [OPENSSL_INCLUDE_DIR, OPENSSL_LIBRARIES] are found, extensions [HASHLIB, SSL] will be built
  If [OPENSSL_INCLUDE_DIR, OPENSSL_LIBRARIES] are NOT found, extensions [SHA, SHA256, SHA512] will be built

USE_SYSTEM_TCL=ON|OFF         (defaults to ON)
  If set to OFF, no attempt to detect Tcl libraries will be done.
  Associated python extensions are: TKINTER
  Following CMake variables can manually be set: TCL_LIBRARY, TK_LIBRARY, TCL_INCLUDE_PATH, TK_INCLUDE_PATH

USE_SYSTEM_ZLIB=ON|OFF        (defaults to ON)
  If set to OFF, no attempt to detect ZLIB libraries will be done.
  Associated python extensions are: BINASCII, ZLIB
  Following CMake variables can manually be set: ZLIB_LIBRARY, ZLIB_INCLUDE_DIR, ZLIB_ROOT
  ZLIB_ROOT should be set only if USE_SYSTEM_ZLIB is ON
  If [ZLIB_LIBRARY, ZLIB_INCLUDE_DIR] are found, extensions [BINASCII] will be built with ZLIB_CRC32

USE_SYSTEM_DB=ON|OFF          (defaults to ON)
  If set to OFF, no attempt to detect DB libraries will be done.
  Associated python extensions are: BSDDB
  Following CMake variables can manually be set: DB_INCLUDE_PATH, DB_LIBRARY

USE_SYSTEM_GDBM=ON|OFF        (defaults to ON)
  If set to OFF, no attempt to detect GDBM libraries will be done.
  Associated python extensions are: DBM, GDBM
  Following CMake variables can manually be set: GDBM_INCLUDE_PATH, GDBM_LIBRARY, GDBM_COMPAT_LIBRARY, NDBM_TAG, <NDBM_TAG>_INCLUDE_PATH

USE_SYSTEM_LZMA=ON|OFF     (defaults to ON)
  If set to OFF, no attempt to detect LZMA libraries will be done.
  Associated python extensions are: LZMA
  Following CMake variables can manually be set: LZMA_INCLUDE_PATH, LZMA_LIBRARY

USE_SYSTEM_READLINE=ON|OFF    (defaults to ON)
  If set to OFF, no attempt to detect Readline libraries will be done.
  Associated python extensions are: READLINE
  Following CMake variables can manually be set: READLINE_INCLUDE_PATH, READLINE_LIBRARY

USE_SYSTEM_SQLite3=ON|OFF     (defaults to ON)
  If set to OFF, no attempt to detect SQLite3 libraries will be done.
  Associated python extensions are: SQLITE3
  Following CMake variables can manually be set: SQLite3_INCLUDE_DIR, SQLite3_LIBRARY

CMAKE_OSX_SDK                (MacOSX, default is autodetected, e.g 'macosx10.06')
  By default, the variable is automatically set running `xcrun` and/or `xcodebuild`. Note that its
  value can also be explicitly set when doing a clean configuration either by adding a cache entry in
  `cmake-gui` or by passing the argument `-DCMAKE_OSX_SDK:STRING=macosx10.6` when running `cmake`.
  Then, this variable is used to initialize `CMAKE_OSX_SYSROOT`, `CMAKE_OSX_DEPLOYMENT_TARGET`
  and `MACOSX_DEPLOYMENT_TARGET` variables.

Licenses

Materials in this repository are distributed under the following licenses:

All software is licensed under the Apache 2.0 License. See LICENSE_Apache_20 file for details.

FAQ

Why Apache 2.0 License?

From the python.org wiki, the answer to the question What if I want to contribute my code to the PSF mentions that if code is going to end up in Python or the standard library, the PSF will require you to license code under "Academic Free License" or "Apache License 2.0".

Which python versions are supported?

This project supports building multiple versions of CPython 2.7 and CPython 3. See current list of supported version in top-level CMakeLists.txt.

Since this repository is maintained separately from python/CPython itself, it needs to be manually updated whenever there is a new release of Python.

python-cmake-buildsystem's People

Contributors

adrianbroher avatar arves100 avatar billsix avatar bjia56 avatar dand-oss avatar davidsansome avatar dbrnz avatar droidmonkey avatar ericmehl avatar erlend-aasland avatar fbordignon avatar glehmann avatar gongminmin avatar ihnorton avatar jamesobutler avatar jcfr avatar jrobcary avatar khainke avatar lassoan avatar madebr avatar msmolens avatar oxbambooxo avatar shakfu avatar simonmssu avatar thewtex avatar tony avatar ycollet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-cmake-buildsystem's Issues

Why favour BUILD_{SHARED,STATIC} over BUILD_SHARED_LIBS?

I'm wondering why the project allows to build users the shared and static variant at the same time using two non-standard variables to switch on/off the static or shared build.

Also I can't think of a good reason why a regular user of this project would need to build the library shared and static at the same time.

Digging through history does give me the impression that the first implementer wasn't aware of the existence of BUILD_SHARED_LIBS.

cross-compilation using windows-x86: WindowsError in random.py

docker run -ti -v $SRC:/usr/pysrc:ro thewtex/cross-compiler-windows-x86 bash -c '\
cp /usr/pysrc/Python-2.7.11.tgz /usr/; \
cd /usr; tar -xzvf Python-2.7.11.tgz; \
cd Python-2.7.11; \
patch -p0 -i /usr/pysrc/cmake/patches-win32/01-dynload_win.patch; \
patch -p0 -i /usr/pysrc/cmake/patches-win32/02-signalmodule.patch; \
patch -p0 -i /usr/pysrc/cmake/patches-win32/03-mingw32.patch; \
patch -p0 -i /usr/pysrc/cmake/patches-win32/04-uninitialised-flags.patch; \
mkdir /usr/build; cd /usr/build; \
cmake -DBUILD_STATIC=0 -DBUILD_SHARED=1 -DENABLE_CTYPES=0 -DENABLE_SOCKET=0 -DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN_FILE ../pysrc; \
make -j4; \
ctest -R; \
bash'
1: Test command: /usr/bin/wine "/usr/build/bin/python.exe" "-Wd" "-3" "-E" "-tt" "/usr/build/Lib/test/regrtest.py" "-l" "-v" "test_abc"
1: Test timeout computed to be: 1500
1: Traceback (most recent call last):
1:   File "/usr/build/Lib/test/regrtest.py", line 160, in <module>
1:     import random
1:   File "Z:\usr\build\Lib\random.py", line 885, in <module>
1:     _inst = Random()
1:   File "Z:\usr\build\Lib\random.py", line 97, in __init__
1:     self.seed(x)
1:   File "Z:\usr\build\Lib\random.py", line 113, in seed
1:     a = long(_hexlify(_urandom(2500)), 16)
1: WindowsError: [Error -2146893801] Windows Error 0x80090017
1/1 Test #1: test_abc .........................***Failed    3.23 sec

Generate Python-X.Y.Z-pyX.Y.egg-info

File location: <CMAKE_INSTALL_PREFIX>/lib/python2.7/lib-dynload/Python-2.7.3-py2.7.egg-info

File content:

Metadata-Version: 1.1
Name: Python
Version: 2.7.3
Summary: A high-level object-oriented programming language
Home-page: http://www.python.org/2.7
Author: Guido van Rossum and the Python community
Author-email: [email protected]
License: PSF license
Description: Python is an interpreted, interactive, object-oriented programming
        language. It is often compared to Tcl, Perl, Scheme or Java.

        Python combines remarkable power with very clear syntax. It has
        modules, classes, exceptions, very high level dynamic data types, and
        dynamic typing. There are interfaces to many system calls and
        libraries, as well as to various windowing systems (X11, Motif, Tk,
        Mac, MFC). New built-in modules are easily written in C or C++. Python
        is also usable as an extension language for applications that need a
        programmable interface.

        The Python implementation is portable: it runs on many brands of UNIX,
        on Windows, DOS, OS/2, Mac, Amiga... If your favorite system isn't
        listed here, it may still be supported, if there's a C compiler for
        it. Ask around on comp.lang.python -- or just try compiling Python
        yourself.
Platform: Many
Classifier: Development Status :: 6 - Mature
Classifier: License :: OSI Approved :: Python Software Foundation License
Classifier: Natural Language :: English
Classifier: Programming Language :: C
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development

Transfer ownership to "python-cmake-buildsystem" organization

Hi David,

Hope things are going well.

To facilitate the management of the project (for example setup of continuous integration, ...) I would suggest you transfer its ownership to the python-cmake-buildsystem organization.

Let me know what you think,

Thanks
Jc

Add @jcfr as project maintainer

Hi David,

Understanding you have probably other priorities and projects to manage, what do you think of delegating some of the upstream integration ?

I would avoid to create yet an other fork of the project (named differently) that would diverge from its original. In the coming week, I would like to start integrating the new build system into existing project.

An other approach to consider, I would also like to ensure that the repository is available even if for some (very unlikely) reason you delete your github account ... since the "official" repo is currently associated with you, what do you think if instead we create the "official" repo on the "commontk" organization. It is also understood that you would be a member of the associated team with full access to the new repo.

See http://commontk.org and https://github.com/commontk

Let me know what you think.

I would like to take action by the end of next week.

Thanks
Jc

after version 2.7.4 of python sysconfigdata need to be built at compile time

after version 2.7.4 of python sysconfigdata need to be built at compile time.
Here is a patch to perform the generation of sysconfigdata.py file:

From 71c8d07c1105941e1f8f61620f32203367fc1d9a Mon Sep 17 00:00:00 2001
From: Yann Collette [email protected]
Date: Wed, 7 May 2014 16:36:36 +0200
Subject: [PATCH 1/1] generate sysconfigdata depending on python version


cmake/python/CMakeLists.txt | 7 +++++++
1 file changed, 7 insertions(+)

diff --git a/cmake/python/CMakeLists.txt b/cmake/python/CMakeLists.txt
index fd3cfa8..415f555 100644
--- a/cmake/python/CMakeLists.txt
+++ b/cmake/python/CMakeLists.txt
@@ -23,6 +23,13 @@ set_property(GLOBAL APPEND PROPERTY PYTHON_TARGETS python)
install(TARGETS python EXPORT PythonTargets RUNTIME DESTINATION ${BIN_INSTALL_DIR} COMPONENT Runtime)

if(UNIX)

  • if ("${PY_VERSION_MAJOR}.${PY_VERSION_MINOR}.${PY_VERSION_PATCH}" VERSION_GREATER "2.7.4")
  • add_custom_target(build_sysconfigdata ALL
  •                  COMMAND $<TARGET_FILE:python> "-E" "-S" "-m" "sysconfig" "--generate-posix-vars"
    
  •         COMMAND find "${CMAKE_BINARY_DIR}/build" "-name" "'*.py'" "-exec" "cp" "{}" "${CMAKE_BINARY_DIR}/${PYTHONHOME}" "\;"
    
  •                  COMMENT "Generation of sysconfigdata file")
    
  • endif ()
  • install(CODE
    "message(STATUS "Creating Python executable symlinks...")
    execute_process(

    1.9.0

Problem with compilation of python 2.7.6 under windows

I meet a problem with compilation of python 2.7.6 under windows.
Certainly due to the difference of behavior of semicolon between linux and windows.
Here is a patch which allows to compile python under windows 7 + visual 2008:

From f33adbc5c9e98fa24a14449ccb22fd071efccca1 Mon Sep 17 00:00:00 2001
From: Yann Collette [email protected]
Date: Wed, 7 May 2014 09:37:54 +0200
Subject: [PATCH 2/2] manage semicolon for windows compilation


cmake/libpython/CMakeLists.txt | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/cmake/libpython/CMakeLists.txt b/cmake/libpython/CMakeLists.txt
index d81b268..83459d4 100644
--- a/cmake/libpython/CMakeLists.txt
+++ b/cmake/libpython/CMakeLists.txt
@@ -183,6 +183,10 @@ elseif(WIN32)
set(PYTHONPATH "${PYTHONPATH}${PATHSEP}.\${PYTHONHOME_ESCAPED}\lib-tk")
endif(ENABLE_TKINTER)

  • if (WIN32)
  •  string(REPLACE ";" "\;" PYTHONPATH "${PYTHONPATH}")
    
  • endif ()
  • set_property(
    SOURCE ${SRC_DIR}/PC/getpathp.c
    PROPERTY COMPILE_DEFINITIONS

    1.9.0

Detect uint32_t and int32_t

Detect uint32_t and int32_t (defined in stdint.h on fedora 20)
Once this have been detected, add in pyconfig.h

define PY_UINT32_T uint32_t

define PY_INT_32_T int32_t

define HAVE_UINT32_T 1

define HAVE_INT32_T 1

With these modifications, I was able to compile python 2.7.6 using cmake on fedora 20.

Installation of python standard library modules despite disabled components.

This is not an issue, but rather a request for enhancement or general question.

When disabling a specific component, e.g. ENABLE_SQLITE3=OFF the coresponding .pyd file is not build. However the installation of the python standard library still contains the sqlite3 module. My naive assumption would be that in this case there would be no sqlite3 module available, because all of the sqlite3 just assumes that there is a _sqlite3 module available, which isn't the case when disabling the module. This isn't limited to the sqlite3 component but for various other components too.

Is this intended behavior? How does upstream handle this?

BZIP2_INCLUDE_DIR variable not used

can we change python-cmake-buildsystem\cmake\extensions\CMakeLists.txt line 332 from

add_python_extension(bz2
    REQUIRES BZIP2_LIBRARIES
    SOURCES bz2module.c
    LIBRARIES ${BZIP2_LIBRARIES}
)

to

add_python_extension(bz2
    REQUIRES BZIP2_LIBRARIES BZIP2_INCLUDE_DIR
    SOURCES bz2module.c
    LIBRARIES ${BZIP2_LIBRARIES}
    INCLUDEDIRS ${BZIP2_INCLUDE_DIR}
)

so that we can specify bzip2 library installation directory?

I've also attached the CMakeLists.txt file

The reason is, I will need to turn on bz2 module in python to use the python in my qt5 compilation.

thanks
-s

Not possible to compile using intel compiler

It's not possible to compile python using intel compiler.
This is due to the name of the python library.
Here is a patch which allows to compile python using intel compiler under windows + add a BUILTIN_ALL option which activate all builtins.

From f93cd14be0cd2c249e21116e5ee24b49f1de90d7 Mon Sep 17 00:00:00 2001
From: Yann Collette <[email protected]>
Date: Wed, 7 May 2014 12:15:40 +0200
Subject: [PATCH 1/1] enable compilation with intel under windows + add
 builtin_all option

---
 CMakeLists.txt         |  5 +++--
 cmake/Extensions.cmake | 15 ++++++++++-----
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index cf0d417..b844579 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -15,9 +15,9 @@ project(Python${PY_VERSION_MAJOR}${PY_VERSION_MINOR} C)

 # This is the major version number of Python
 set(LIBPYTHON_VERSION ${PY_VERSION_MAJOR}.${PY_VERSION_MINOR})
-if(MSVC)
+if(MSVC OR "${CMAKE_C_COMPILER_ID}" STREQUAL "Intel"))
     set(LIBPYTHON_VERSION ${PY_VERSION_MAJOR}${PY_VERSION_MINOR})
-endif(MSVC)
+endif()
 set(LIBPYTHON python${LIBPYTHON_VERSION})

 # Include helper functions
@@ -30,6 +30,7 @@ include(CTest)
 # Options
 option(BUILD_SHARED "Build a shared libpython library" OFF)
 option(BUILD_STATIC "Build a static libpython library" ON)
+option(BUILTIN_ALL  "Build all extensions as builtin"  OFF)
 option(USE_LIB64 "Search for dependencies and install to prefix/lib64 instead of prefix/lib" OFF)

 option(USE_SYSTEM_LIBRARIES "Use system libraries" ON)
diff --git a/cmake/Extensions.cmake b/cmake/Extensions.cmake
index 7366c99..145c0b9 100644
--- a/cmake/Extensions.cmake
+++ b/cmake/Extensions.cmake
@@ -64,11 +64,16 @@ function(add_python_extension name)
            "Controls whether the \"${name}\" extension will be built"
            ON
     )
-    option(BUILTIN_${upper_name}
-           "If this is set the \"${name}\" extension will be compiled in to libpython"
-           ${ADD_PYTHON_EXTENSION_BUILTIN}
-    )
-
+    if (BUILTIN_ALL)
+      option(BUILTIN_${upper_name}
+             "If this is set the \"${name}\" extension will be compiled in to libpython" ON}
+      )
+    else ()
+      option(BUILTIN_${upper_name}
+             "If this is set the \"${name}\" extension will be compiled in to libpython"
+             ${ADD_PYTHON_EXTENSION_BUILTIN}
+      )
+    endif ()
     # Check all the things we require are found.
     set(missing_deps "")
     foreach(dep ${ADD_PYTHON_EXTENSION_REQUIRES} ENABLE_${upper_name})
-- 
1.9.0

Build Python 32 bit on Linux 64 bits

cmake -DCMAKE_C_FLAGS=-m32 -DCMAKE_EXE_LINKER_FLAGS=-m32 -DCMAKE_INSTALL_PREFIX:PATH=/home/user/src/python-install ../python-cmake-buildsystem

I tried with the following options, but did not succeed

/home/user/src/Python-2.7.8/Include/pyport.h:886:2: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
 #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."

I can not understand what the problem.

Bump CMake minimum required version

To simplify the build system and remove some of the workaround needed to support older version of CMake, I was thinking to bump the minimum required version of CMake.

Looking at the table below (compiled using detailed data reported further down), bumping to CMake 2.8.11 will allow to use the CMake executable provided by most of the system (expect Centos 5 and 7).

Or we could bump the requirement to CMake 3.5 and expect developer to either download the latest CMake binaries or build CMake from source.

What do you all think ?

System Available CMake version
Debian stable (Jessie) 3.0.2
Ubuntu 15.10 LTS (wily) 3.2.2
Fedora 23 3.4.1
Manylinux (based of Centos 5.11) 2.6.4
CentOS 7 2.8.11
CentOS 6 2.8.12
CentOS 5 2.6.4
OpenSuse 13.X 3.2.2
RedHat EL 5,6,7 2.8.11
Windows (Chocolatey) 3.4.3
MacOSX (Homebrew) 3.5.0
MacOSX (MacPort) 3.5.1

To help make a decision, as of March 29th 2016, I collected these info:

    wheezy (oldstable) (devel): cross-platform, open-source make system
    2.8.9-1: amd64 armel armhf i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc s390 s390x sparc
    wheezy-backports (devel): cross-platform, open-source make system
    2.8.11.1-1~bpo70+1: amd64 armel armhf i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc s390 s390x sparc
    jessie (stable) (devel): cross-platform, open-source make system
    3.0.2-1: amd64 arm64 armel armhf i386 mips mipsel powerpc ppc64el s390x
    stretch (testing) (devel): cross-platform, open-source make system
    3.5.0-1: amd64 arm64 armel armhf i386 mips mipsel powerpc ppc64el s390x
    sid (unstable) (devel): cross-platform, open-source make system
    3.5.0-1: alpha amd64 arm64 armel armhf hppa hurd-i386 i386 m68k mips mipsel powerpc ppc64 ppc64el s390x sh4 sparc64 x32
    3.2.2-2+b1: kfreebsd-amd64 kfreebsd-i386
    precise (12.04LTS) (devel): cross-platform, open-source make system
    2.8.7-0ubuntu4: amd64 i386
    precise-updates (devel): cross-platform, open-source make system
    2.8.7-0ubuntu5: amd64 i386
    trusty (14.04LTS) (devel): cross-platform, open-source make system
    2.8.12.2-0ubuntu3: amd64 i386
    vivid (devel): cross-platform, open-source make system
    3.0.2-1ubuntu2: amd64 i386
    wily (devel): cross-platform, open-source make system
    3.2.2-2ubuntu3: amd64 i386
    xenial (devel): cross-platform, open-source make system
    3.3.2is3.2.2-0ubuntu5: amd64 i386
Scientific Linux 7  ftp.sourceforge.net/pub/sourceforge/s/sl/sl7-i686-project/yum/FEDOREL7/FULLMISSING/cmake-2.8.11-4.el7.i686.rpm
Scientific Linux 6  ftp.scientificlinux.org/linux/scientific/6rolling/x86_64/os/Packages/cmake-2.8.12.2-4.el6.x86_64.rpm
Scientific Linux 5  ftp.scientificlinux.org/linux/scientific/5rolling/x86_64/SL/cmake-2.6.4-5.el5.4.x86_64.rpm
RedHat EL 7     pkgs.repoforge.org/cmake/cmake-2.8.8-1.el7.rfx.x86_64.rpm
RedHat EL 6     pkgs.repoforge.org/cmake/cmake-2.8.8-1.el6.rfx.x86_64.rpm
RedHat EL 5     pkgs.repoforge.org/cmake/cmake-2.8.8-1.el5.rfx.x86_64.rpm
OpenSuSE 13.X   packman.iu-bremen.de/suse/openSUSE_13.1/Essentials/x86_64/cmake-3.2.2-200.2.x86_64.rpm
OpenSuSE 12.X   packman.iu-bremen.de/suse/openSUSE_12.3/Essentials/x86_64/cmake-3.2.2-200.1.x86_64.rpm
OpenSuSE 11.X   packman.iu-bremen.de/suse/openSUSE_11.4/Essentials/x86_64/cmake-3.1.3-184.1.x86_64.rpm
Mageia 5    distrib-coffee.ipsl.jussieu.fr/mageia/distrib/5/x86_64/media/core/updates/cmake-3.0.2-1.mga5.x86_64.rpm
Mageia 4    distrib-coffee.ipsl.jussieu.fr/mageia/distrib/4/x86_64/media/core/release/cmake-2.8.12.1-1.mga4.x86_64.rpm
Mageia 3    distrib-coffee.ipsl.jussieu.fr/mageia/distrib/3/x86_64/media/core/release/cmake-2.8.10.2-5.mga3.x86_64.rpm
Fedora 23   download.fedora.redhat.com/pub/fedora/linux/updates/23/x86_64/c/cmake-3.4.1-4.fc23.x86_64.rpm
Fedora 22   download.fedora.redhat.com/pub/fedora/linux/updates/22/x86_64/c/cmake-3.3.2-1.fc22.x86_64.rpm
Fedora 21   ftp5.gwdg.de/pub/opensuse/repositories/devel:/cloverleaf:/testing/Fedora_21/x86_64/cmake-3.2.1-3.1.fc21.x86_64.rpm
Fedora 20   ftp5.gwdg.de/pub/opensuse/repositories/Openwsman/Fedora_20/x86_64/cmake-3.0.0-142.1.x86_64.rpm
Fedora 19   ftp5.gwdg.de/pub/opensuse/repositories/Openwsman/Fedora_19/x86_64/cmake-3.0.0-142.1.x86_64.rpm
Fedora 18   ftp5.gwdg.de/pub/opensuse/repositories/Openwsman/Fedora_18/x86_64/cmake-2.8.11.2-106.1.x86_64.rpm
Fedora 17   ftp5.gwdg.de/pub/opensuse/repositories/Openwsman/Fedora_17/x86_64/cmake-2.8.11.2-106.1.x86_64.rpm
CentOS 7    ftp.centos.org/7.2.1511/os/x86_64/Packages/cmake-2.8.11-5.el7.x86_64.rpm
CentOS 6    ftp.centos.org/6.7/os/x86_64/Packages/cmake-2.8.12.2-4.el6.x86_64.rpm
CentOS 5    ftp.centos.org/5.11/os/x86_64/CentOS/cmake-2.6.4-5.el5.4.x86_64.rpm

List above obtained by:
(1) manually copying output of search illustrated below into file ~/Projects/cmake-pbone-list.txt
(2) running cat ~/Projects/cmake-pbone-list.txt | sort -r | uniq > ~/Projects/cmake-pbone-list-filtered.txt
(3) manually editing files ~/Projects/cmake-pbone-list-filtered.txt keeping the first entry of each distrib

2016 03 29_cmake_pbone_search

Update CMake files to match preferred style

Ancient CMake versions required upper-case commands.  Later command
names became case-insensitive.  Now the preferred style is lower-case.

Run the following shell code:

cmake --help-command-list |
grep -v "cmake version" |
while read c; do
    echo 's/\b'"$(echo $c | tr '[:lower:]' '[:upper:]')"'\(\s*\)(/'"$c"'\1(/g'
done >convert.sed &&
git ls-files -z -- bootstrap '*.cmake' '*.cmake.in' '*CMakeLists.txt' |
egrep -z -v '^(Utilities/cm|Source/kwsys/)' |
xargs -0 sed -i -f convert.sed &&
rm convert.sed

See Kitware/CMake@77543bd

Ancient versions of CMake required else(), endif(), and similar block
termination commands to have arguments matching the command starting the
block.  This is no longer the preferred style.

Run the following shell code:

for c in else endif endforeach endfunction endmacro endwhile; do
    echo 's/\b'"$c"'\(\s*\)(.\+)/'"$c"'\1()/'
done >convert.sed &&
git ls-files -z -- bootstrap '*.cmake' '*.cmake.in' '*CMakeLists.txt' |
egrep -z -v '^(Utilities/cm|Source/kwsys/)' |
egrep -z -v 'Tests/CMakeTests/While-Endwhile-' |
xargs -0 sed -i -f convert.sed &&
rm convert.sed

---

See Kitware/CMake@9db3116

How activate unicode ucs4

Firstly, thanks for this useful project.

I have compiled "python-cmake-buildsystem" in Ubuntu by default, no special options are activated. I am testing C++ / Python / boost-python and have this linking error:

../../../depends/boost-python-1.58.0.0/Linux_Ubuntu_trusty_x86_64/g++/Debug/lib/libboost_python-gcc49-mt-d-1_58.so: referencia a `PyUnicodeUCS4_FromEncodedObject' sin definir
../../../depends/boost-python-1.58.0.0/Linux_Ubuntu_trusty_x86_64/g++/Debug/lib/libboost_python-gcc49-mt-d-1_58.so: referencia a `PyUnicodeUCS4_AsWideChar' sin definir
collect2: error: ld returned 1 exit status

Problem here is combination of two states:

  • Python compiled with "python-cmake-buildsystem" use UCS2.
>>> import sys
>>> print sys.maxunicode
65535
  • boost-python is assuming UCS4.

How can activate ucs4? Using autotools is:

./configure --enable-unicode=ucs4

Thanks.

Remove release-2.7.x branches

As pointed by @thewtex, keeping these branches around is confusing. We should remove them.

The idea was to keep the master version of build system compatible with all stable version of python sources.

When this was not possible anymore (for example, because maintaining the system working for an older version would make the build system to complex), branch like release-2.7.x would be maintained to ensure older version of python could be build with a previous version of the build system.

For now, let's remove this branch and give up on building 2.7.1 and 2.7.2 with the build system. If there is a need, we could probably be update the test matrix on travis and appveyor to check for these and fix the build system.

For future reference, release-2.7.1 and release-2.7.2 were both pointing to ed37e55

Cross-compile with buildroot x-toolchain

Compiling this natively on Debian 8 is flawless. Attepting a cross-compile using a buildroot uclibc-based cross-toolchain for mipsel and arm is running into issues.

cmake \
-DCMAKE_INSTALL_PREFIX=$PREFIX \
-DCMAKE_C_COMPILER=$DESTARCH-linux-gcc \
-DCMAKE_CXX_COMPILER=$DESTARCH-linux-g++ \
-DCMAKE_C_FLAGS="$CPPFLAGS $CFLAGS" \
-DCMAKE_CXX_FLAGS="$CPPFLAGS $CXXFLAGS" \
-DCMAKE_EXE_LINKER_FLAGS="$LDFLAGS" \
../python-cmake-buildsystem

I cross-compile cmake itself using the cmake build system with a similar approach.
I'm getting errors for detection of some base system libraries. Perhaps this is due to using a non-glibc c-library. Just throwing a wrench in the works :)

cmath (not set: HAVE_LIBM)
crypt (not set: HAVE_LIBCRYPT)
datetime (not set: HAVE_LIBM)
HAVE_LIBUTIL

My project creates a native development environment for native compilation on arm and mipsel, and it runs into these same errors natively there as well.

It says it can't find some libraries, but it also tells me which versions it found. Also perhaps an issue of using uclibc.

-- The system name is Linux
-- The system processor is x86_64
-- The system version is 3.16.0-4-amd64
-- Could NOT find BZip2 (missing:  BZIP2_LIBRARIES) (found version "1.0.6")
-- Could NOT find Curses (missing:  CURSES_LIBRARY)
-- Could NOT find EXPAT (missing:  EXPAT_LIBRARY) (found version "2.1.0")
-- Could NOT find TCL (missing:  TCL_LIBRARY)
-- Could NOT find TCLTK (missing:  TCL_LIBRARY TK_LIBRARY)
-- Could NOT find TK (missing:  TK_LIBRARY)
-- Could NOT find ZLIB (missing:  ZLIB_LIBRARY) (found version "1.2.8")

grepping the source I don't see any instance of CMAKE_EXE_LINKER_FLAGS. I make use of it because I set a non-standard location for the dynamic linker.

Also it doesn't (and can't) detect devices like /dev/ptmx for the target, so would need a way to manually tell it whether the target devices has that or not. (Actually, probably can do this with a cmake variable I imagine).

Not sure if you're wanting to support cross-compiling for linux targets (and whether you'd worry about uclibc if so). Seems like all is in place for windows. but I really like the idea of being able to compile python without needing some parts of itself to build other parts.

Fix CMake 2.8.12.1 configuration warning

CMake Warning (dev) at CMakeLists.txt:102 (include):
  Syntax Warning in cmake code at

    /Users/fedorov/local/builds/Slicer4-Release/python/cmake/ConfigureChecks.cmake:1292:29

README says Python 2.7.1 is supported

In the master branch, the configure step expects Python/random.c to exist, but it doesn't exist in v2.7.1. Maybe the README should be updated to say that the supported version is 2.7.5?

Add logic related to MACOSX_DEPLOYMENT_TARGET

If system is >= 10.4, the deployement target will always be set to 10.4.

When build using autoconf, this lead to HAVE_STRUCT_STAT_ST_BIRTHTIME being undefined because the associated compile test fail.

Note that targeting for 10.4 will report that struct stat has no member st_birthtimespec when trying to compile the following program:

// File: cm_test_struct_stat_st_birthtime.c
#include <sys/stat.h>
int main()
{
   struct stat* tmp;
   tmp->st_birthtime;
  return 0;
}
  • 10.4 and 10.5: failed
$ MACOSX_DEPLOYMENT_TARGET=10.4 gcc cm_test_struct_stat_st_birthtime.c 
cm_test_struct_stat_st_birthtime.c: In function ‘main’:
cm_test_struct_stat_st_birthtime.c:7: error: ‘struct stat’ has no member named ‘st_birthtimespec’
  • 10.6, 10.7, 10.8: pass
MACOSX_DEPLOYMENT_TARGET=10.6 gcc cm_test_struct_stat_st_birthtime.c
$ echo $?
0

Installation of tests despite BUILD_TESTING=OFF

When installing python via make install the current master of python-cmake-buildsystem installs the python unit test in LIBDIR. I would expect that this shouldn't be the case.

To reproduce use this CMakeLists.txt file. Invoke it with cmake .. in a separate build directory.

cross-compilation using windows-x64: OverflowError: strings are too large to concat

SRC=~/Projects/python-cmake-buildsystem

docker run -ti -v $SRC:/usr/pysrc:ro thewtex/cross-compiler-windows-x64 bash -c '\
cp /usr/pysrc/Python-2.7.11.tgz /usr/; \
cd /usr; tar -xzvf Python-2.7.11.tgz; \
cd Python-2.7.11; \
patch -p0 -i /usr/pysrc/cmake/patches-win32/01-dynload_win.patch; \
patch -p0 -i /usr/pysrc/cmake/patches-win32/02-signalmodule.patch; \
patch -p0 -i /usr/pysrc/cmake/patches-win32/03-mingw32.patch; \
mkdir /usr/build; cd /usr/build; \
cmake -DCMAKE_TOOLCHAIN_FILE=$CMAKE_TOOLCHAIN_FILE ../pysrc; \
make -j4; \
ctest -R test_abc -V; \
bash'
test 1
    Start 1: test_abc

1: Test command: /usr/bin/wine "/usr/build/bin/python.exe" "-Wd" "-3" "-E" "-tt" "/usr/build/Lib/test/regrtest.py" "-l" "-v" "test_abc"
1: Test timeout computed to be: 1500
1: OverflowError: strings are too large to concat

Unable to compile datetime, socket and time extensions with shared python 2.7.9

When compiling python 2.7.9 with this project the modules mentioned in the title can't be build. CMake reports for all of them the following or a similar error message:

$HOME\python-prefix\src\python\Modules\timemodule.c(104) : warning C4273: '_PyTime_DoubleToTimet' : inconsistent dll linkage
        $HOME\python-prefix\src\python\Include\timefuncs.h(17) : see previous definition of '_PyTime_DoubleToTimet'
$HOME\python-prefix\src\python\Modules\timemodule.c(1062) : error C2491: '_PyTime_FloatTime' : definition of dllimport function not allowed

To reproduce use this CMakeLists.txt file. Invoke it with cmake -n "NMake Makefiles" .. in a separate build directory.

Fix building of readline module on MacOSX 10.6.8

[ 25%] Building C object CMakeBuild/extensions/CMakeFiles/extension_readline.dir/Users/kitware/CDashAtHome/Nightly/Python-2.7.3/Modules/readline.c.o
/Users/kitware/CDashAtHome/Nightly/Python-2.7.3/Modules/readline.c: In function 'readline_until_enter_or_signal':
/Users/kitware/CDashAtHome/Nightly/Python-2.7.3/Modules/readline.c:1024: error: 'rl_event_hook' undeclared (first use in this function)
/Users/kitware/CDashAtHome/Nightly/Python-2.7.3/Modules/readline.c:1024: error: (Each undeclared identifier is reported only once
/Users/kitware/CDashAtHome/Nightly/Python-2.7.3/Modules/readline.c:1024: error: for each function it appears in.)
make[2]: *** [CMakeBuild/extensions/CMakeFiles/extension_readline.dir/Users/kitware/CDashAtHome/Nightly/Python-2.7.3/Modules/readline.c.o] Error 1
make[1]: *** [CMakeBuild/extensions/CMakeFiles/extension_readline.dir/all] Error 2
make: *** [all] Error 2

Branch naming convention

Hi David,

Thanks for your work on the CMake-ification of python. It provides a lot flexibility in the way to build python and is well documented.

Considering that the master branch will ultimately hold the Cmake code allowing to build the latest version of python, what do you think if we create branches like:

  • release-2.7.1
  • release-2.7.2

These branch would be like the up-to-date build system for the corresponding python version.

For now, we could consider that master is dedicated to the support of the latest 2.7.x version of python. When we will be starting to support the 3 series, we will create the branch release-2.7.3

We could also help setting some dashboard using CDash so that results associated with python tests are reported. It would allow us to catch issue.

Let me know what you think,

Thanks,
Jc

Building with Visual Studio 2010

Hi JC,

To follow up on our previous thread:
commontk/PythonQt#9 (comment)

Using CMake 2.8.10 and the latest repo from today.

ENABLE_CTYPES and BUILTIN_CTYPES are missing in CMake.

Build the INSTALL project from the VS solution return:

Error 175 error MSB3073: The command "setlocal
"C:\Program Files (x86)\CMake 2.8\bin\cmake.exe" -DBUILD_TYPE=Release -P cmake_install.cmake
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd

_socket.pyd not built under windows.

_scoket.pyd is not built under windows.
I think this is due to the check of sys/socket.h which is a linux only system header (in ./cmake/ConfigureChecks.cmake)
Under windows, the header to check seems to be winsock2.h (included in some python source files).

sysv.S:383:2: error: missing .ascii/.string

I was hoping to use this to build a static python. But first I'm just trying to follow the README example.

$ mkdir scratch
$ cd ~/scratch
$ git clone git://github.com/davidsansome/python-cmake-buildsystem
$ mkdir -p python-build && mkdir -p python-install
$ cd python-build
$ cmake -DCMAKE_INSTALL_PREFIX:PATH=${HOME}/scratch/python-install ../python-cmake-buildsystem
$ make -j2
...
[  8%] Building C object CMakeBuild/extensions/CMakeFiles/extension_csv.dir/home/mudd/scratch/Python-2.7.8/Modules/_csv.c.o
/home/mudd/scratch/python-build/../Python-2.7.8/Modules/_ctypes/libffi/src/x86/sysv.S:383:2: error: missing .ascii/.string
#error missing .ascii/.string
 ^
1 error generated.
make[2]: *** [CMakeBuild/Modules/_ctypes/libffi/src/x86/sysv.o] Error 1
make[1]: *** [CMakeBuild/extensions/CMakeFiles/extension_ctypes.dir/all] Error 2

I'm running Ubuntu 14.04 with cmake 2.8.12.2 and gcc 4.8.4.

Any suggestions?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.