Seems there are no git tags here nor a PyPI release. Is a release planned? When should it be expected?

See:
https://gitlab.kitware.com/vtk/vtk/merge_requests/1511
http://review.source.kitware.com/#/c/21091/
http://review.source.kitware.com/#/c/21121/

We will have to discuss how it could be best integrated.

Cc: @thewtex @demarle

• https://github.com/matplotlib/matplotlib/blob/d527799f5e2d1ec5da528e12af168f675ef6a066/setupext.py#L563-L575
• https://github.com/matplotlib/matplotlib/blob/d527799f5e2d1ec5da528e12af168f675ef6a066/setupext.py#L1013-L1024

When Executing:
$python cmaker.py

Traceback (most recent call last):
File "cmaker.py", line 4, in
from pycmake.platform_specifics import *
ImportError: No module named platform_specifics

The idea is to be able to use url

https://ci.appveyor.com/project/scikit-build/scikit-build/branch/master

instead of

https://ci.appveyor.com/project/opadron/scikit-build/branch/master

While working on #43, I made the following observation:

The windows generator selection logic does not work near line 37:

if platform.architecture() == "x64":

The code should be able to handle tuples and other spellings of "64 bit". For example, on my home machine running windows 7, I get:

>> platform.architecture()
('64bit', 'WindowsPE')

These should be moved out of the internal test suite since they have third
party dependencies that are not satisfied by requirements-dev.txt. They should
still be tested in the project CI tests.

I am looking for information on planned shared library support.

How are RPATHs going to be handled? Is it going to have be to specified correctly at compilation time? or can it be fixed up during installation and packaging time?

How are dependent shared libraries going to be handled? If my project depends on another large project, is there an easy way to get those dependent libraries installed and packaged with the correct RPATHs?

• On Linux, specify the installation command per-distribution to install GCC.
• On Mac OSX, give instructions to install XCode.
• On Windows, give instructions to install the compiler per Python version listed in this article

Integrating nose with setup.py so that it runs using python setup.py test is not straightforward.

The simple approach is to update setup.py specifying the following:

[...]
test_suite='nose.collector',
[...]

Issue is that nose.collector doesn't detect nose plugins (e.g coverage). See here

Creating a NoseTestCommand in setup.py allow to address this. See http://fgimian.github.io/blog/2014/04/27/running-nose-tests-with-plugins-using-the-setuptools-test-command/

See https://www.python.org/dev/peps/pep-0518/

As discussed in #20, remove the pycmake package from pypi as soon as scikit-build is made available.

Cc: @opadron @msarahan

Example: http://scikit-learn.org/stable/developers/maintainer.html

• Add sample projects to scikit-build/tests/samples
• Tests will be added to python files like scikit-build/tests/test_foo.py
• Add pytest (or any other test driver of your choice) to requirements-dev.txt
• Look into testpath

Cc: @thewtex

This is evident in the wheel file name (``none-any) and in the METADATA file _Platform_ entry, which is currently UNKNOWN`.

A list with project names that link to their homepage, a short description of
the project, and skbuild capabilities it demonstrates (e.g. finding third
party dependencies, building a large C++ code base, etc.)

Projects

• symengine.py: symengine/symengine.py#219
• pyne
• simpleitk: https://github.com/SimpleITK/SimpleITKPythonPackage
• itk: https://github.com/InsightSoftwareConsortium/ITKPythonPackage
• vtk
• matplotlib
• scikit-beam: scikit-beam/scikit-beam#467
• cmake-python-distributions: https://github.com/scikit-build/cmake-python-distributions
• ninja-python-distributions: https://github.com/scikit-build/ninja-python-distributions
• https://github.com/equinor/dlisio

• Shared by @massich:
  • https://github.com/openmeeg/openmeeg
  • https://github.com/massich/replicate_pytime_timeval_error
  • https://github.com/massich/matio-cmake
  • https://github.com/mne-tools/mne-python

• Shared by @vyasr
  • https://github.com/joaander/hoomd-blue
  • https://bitbucket.org/glotzer/freud/src/master/

Currently this creates a tarbal with the contents of _skbuild/cmake-install.
Instead, it should contain the project source code so that the sdist can be
downloaded and a platform-specific binary wheel can be created from it.

Et voila.

As discussed in #85 (comment), "Reorganized master" has been forced pushed:

• History is now coherent
• Each change introduce only one set of change with a descriptive commit message

Based on your current situation, consider following the instructions reported below:

Cc: @scikit-build/developers

Existing checkout: without topic

git fetch origin
git reset --hard origin/master

If you also have a reference to your fork on github, consider also updating it:

git push name_of_your_fork master --force

Existing checkout: with topic - rebase instructions

To rebase your topic named name_of_your_topic:

git checkout master && git branch -M master-original
git fetch origin
git checkout -b master origin/master
git checkout name_of_your_topic
git rebase --onto master master-original name_of_your_topic

More details

There are no differences between the reorganized master (be4a55e) and the former one (169894b):

$ git diff master-original..master-reorganized
# No differences

$ git show master-original 
commit 169894b610d13ba6f3679ff37f7e0d7fbb071f4e
Merge: 068dbcb 8f6a585
Author: Jean-Christophe Fillion-Robin <[email protected]>
Date:   Thu Jul 21 17:36:00 2016 -0400

    Merge pull request #101 from scikit-build/94-update-appveyor-badge

    ci/appveyor: Update badges to references new account. Fixes #94

$ git show master-reorganized
commit be4a55e97cb2dbce4fb569df4708c343f7155ce6
Merge: dc2091b a85ca6b
Author: Jean-Christophe Fillion-Robin <[email protected]>
Date:   Thu Jul 21 21:37:51 2016 -0400

    Merge branch '94-update-appveyor-badge-rebased' into master-reorganized

    * 94-update-appveyor-badge-rebased:
      ci/appveyor: Update badges to reference new account. Fixes #94

Is the goal of this package to create robust setup.py for cmake projects? Is there some example how to use it?

Here is the setup.py that we use in csympy (cmake C++ project with optional thin Python wrappers): https://github.com/sympy/csympy/blob/c9d29e5ced2089934d769115e3002a5f0be089bc/setup.py

It works, but I would prefer to actually install the Python wrappers/package using make install from cmake (which is well tested in our project). That way we just make sure that make install works, and setup.py only calls it. Currently we need to test a separate installation mechanism that setup.py uses.

We should add more options in skbuild's setup() function to customize more aspects of the build.

• Document the different build type option
• Document how to set CMake variables
• Document how to set the CMake build system generator

When there is a problem building python module, report "human-friendly" error

Look into https://github.com/espdev/ITKPythonInstaller and how skbuild could help

Cc: @thewtex

Alternative

• distutils: Building C and C++ Extensions with distutils
• https://bitbucket.org/dholth/enscons
  • [Distutils] enscons, a prototype SCons-powered wheel & sdist builder
  • not a distutils extension
• https://github.com/cournape/numscons [dead project ?]

Complementary

• https://pypi.python.org/pypi/flit

Currently ctest is invoked to run test of each sample projects and we should also avoid explicitly setting PYTHONPATH. Mocking with PYTHONPATH is causing problem with tools like a tox (e.g #62)

Instead, sample project level tests:

• should be implemented as regular python test using either node or python unittest.
• sample project tests should be executed in two contexts:
  • developer mode / editable package mode
  • installed package mode

Currently outputs::

usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

error: invalid command 'bdist_wheel'

• If Makefile Unix, Ninja or Jom based generator are used: -j<number_of_core>
• If Visual Studio:
  • /MP<number_of_core>
  • /maxcpucount:<number_of_core>

Note that for complex project, it could be hard to find the right setting for /MP and /maxcpucount. For example, you want to avoid too many link steps requiring a lot of memory to happen at the same time .... Having a way to disable parallel build may be a sensible option.

More details:

• With VS >= 2010, target level parallelism can be set from command line using /maxcpucount. (Number of simultaneous MSBuild.exe processes). See [1]
• object level parallelism set using /MP (Number of simultaneous cl.exe processes). See [2]

[1] https://msdn.microsoft.com/en-us/library/bb651793%28v=vs.120%29.aspx
[2] https://msdn.microsoft.com/en-us/library/bb385193.aspx

Here is some more details of what we did for Slicer: http://massmail.spl.harvard.edu/public-archives/slicer-devel/2016/019656.html

As discussed in #89 (comment), update of test outside_project_root (in 88239c6) is now causing the file coverage.xml to be generated in

./tests/samples/fail-outside-project-root/coverage.xml

instead of

./tests/coverage.xml

This issue consolidates in a single location todo that were originally listed in #36 and #38 (later superseded by #43):

From #36 and #38:

• Extend testing coverage to include the new apps and new features.
• Currently the only way to add to the Cython module search path for add_cython_target is to use the outdated include_directories CMake command. Ideally, we would be able to use target_include_directories or something similar. This may involve the creation of an actual CMake target. This is now tracked in #243
• There is a bug when trying to specify the desired CMake build type via the --build-type option in scikit-build. It seems that some sections of the code are looking for CMAKE_BUILD_DIR/CMAKE_BUILD_TYPE as the ultimate location of the cmake build directory. I think handling multiple builds in one tree is desirable and we should probably extend this handling to the other temporary staging directories, but right now, these improvements were not expecting this. We'll need to go through the various code sections and make sure everyone's on the same page.
• Refactor the custom distutils command subclasses so that they are accessible for other packages -- in case they want to work around us just as we did distutils.
• Restore the default handling of generic data files so that it matches up with distutils' default behavior. To handle stand-alone extension modules, we'll need to add extra handling logic for the py_modules argument; we would list these modules under py_modules, and leave it to scikit-build to recognize them and place them accordingly.
• Add a final check to the manifest processing loop so that skbuild refuses to handle files installed outside of the project root.
• Add option to the add_cython_target function to override the variable name used to store the location to the generated source file.
• Remove TODO note in add_cython_target regarding C vs C++ output. So far, no problems have been encountered one way or the other during development and testing.
• Remove the FindDebugger.cmake file (used just for debugging cmake targets).
• Update the python_modules_header function so that the FORWARD_DECL_MODULES_VAR option is renamed to FORWARD_DECL_MODULES_LIST and that the list in question is passed by value.
• Add option to the python_modules_header function to override the variable names used to store the locations of the generated header file as well as that of the directory containing it.
• Clean up tower-of-babel boost module header files. Use tbabel_boost_common.h to minimize code duplication.
• Projects that attempt to CMake-install files outside of its designated project installation root should fail to install.
  • Ideally, they should fail prior to the CMake-install step (i.e.: before any problematic files are actually installed). This will need code to determine the would-be files without the aid of the install manifest.
• Each of the sample projects should have their own tests to at least ensure that they build correctly, and that any produced applications run without issue.
• Add tests for pen2_cython
• Add tests for tower-of-babel
• There are multiple bugs in the way command line arguments are processed and forwarded in scikit-build. It's not clear if they were introduced by these changes, or if they predate them.
  • On a related note, I tried to pass options directly to CMake through scikit-build using the -- argument seperator. Scikit-build is currently written to expect up to three sections of such options: the first for scikit-build options, the second for CMake config options, and the third for options to the build step. However, since CMake is used to drive the build step, options meant for the actual build tool (e.g.: make) must be separated by another -- separator. Currently scikit-build fails to properly handle this fourth separator.

I suggest we have a convenient method allowing to invoke ctest and integrate it within existing test runners like pytest, nose or unittest.

That way, we wouldn't have to directly invoke ctest like it is done here

Instead setup.py test could be done.

For reference: Similarly to what is done in pytest,

From http://pycmake.readthedocs.org to http://scikit-build.readthedocs.org

• http://docs.python-guide.org/en/latest/writing/structure/#structure-of-the-repository
• http://setuptools.readthedocs.io/en/latest/setuptools.html
• https://github.com/kennethreitz/samplemod/blob/master/setup.py

• setup.py
• README.rst
• Rename directories
• Remove old package from PyPI. Is there a way in PyPI to redirect to an other package ?
• Generate and upload renamed package

The PyPA has a guide line one the versions of compilers to use with difference versions of Python [1].

Specifically, I noticed that 2.7 was correctly compiled with VS9 2008, while py 3.5 was incorrectly compiled with VS10 2010. From the document:

Python 2.7 used Visual Studio 2008, Python 3.3 and 3.4 used Visual Studio 2010, and Python 3.5+ uses Visual Studio 2015.

[1] https://media.readthedocs.org/pdf/python-packaging-user-guide/develop/python-packaging-user-guide.pdf

As originally commented in #34 (comment) , a dedicated project could be created to facilitate the reuse of the ci/appveyor/driver.py script developed by @opadron

In its initial version, the package would provide the driver script allowing to setup the environment on appveyor.

Currently tests end up being executed using:

nosetests -w tests

Instead settings should be added to setup.cfg so that where is automatically set.

See http://nose.readthedocs.io/en/latest/usage.html#configuration

Same as #56, but we need to fix the CLI arg parsing logic.

For Python 2.7 to Python 3.2: VS2008
For Python 3.3 to Python 3.4: VS2010
For Python 3.5 and above: VS2015

Following code should be tweaked:

Greetings, this is a copy of a message I posted to the cmake mailing list, but got no love there. Since you all are closer to this topic, I hope you might know something helpful.

In compiling TBB, a conda-forge contributor ran across issues with CMake not finding the VS compiler for VS 2008 Win64: conda-forge/staged-recipes#533

The exact error is at https://ci.appveyor.com/project/conda-forge/staged-recipes/build/1.0.2118/job/t2axdaotmty35j6k#L216

Appveyor is an interesting platform, in that they install VS 2008 Express, then the VS compiler for Python (which adds the 64-bit compilation support).

I have been able to fix this in conda-build's test suite by adding some extra registry information and copying some files: https://github.com/conda/conda-build/blob/master/appveyor.yml#L71-L74

Is our version of cmake too old? Is there some other way we can teach CMake to look in the right place, so that that ugly hack of editing the registry is not necessary?

The improved modules in tests/samples/cmake-files/ should be bundled with scikit-build until they are in upstream CMake so every project does not need to copy them.

Use auditwheel to validate generated wheels in our tests and document how projects should use it in their project configuration.

The py.test framework integrates better with setuptools.

See http://docs.pytest.org/en/latest/goodpractices.html#integrating-with-setuptools-python-setup-py-test-pytest-runner

It will also allow to remove workaround discussed in #90

Hi @msarahan , @blowekamp , @scopatz and @PyCMake/owners

We were thinking of renaming the project and organization to scikit-build, that would be more descriptive of what we are trying to achieve:

Easier build for scientific packages embedding C/C++/Fortran extensions.

Let us know what you think,
@thewtex, @jcfr