Giter Club home page Giter Club logo

conan-openstudio-ruby's Introduction

Conan OpenStudio Ruby

A conan package to build ruby for OpenStudio.

Package Status

Artifactory openstudio_ruby/stable openstudio_ruby/testing
Download openstudio_ruby/stable openstudio_ruby/testing

CI is done by Travis for Linux&Mac, and AppVeyor for Windows. To ignore specific commits, please add tags in your commit message such as [skip ci] (skip all), [skip travis] or [skip appveyor].

Testing

There is a test_package folder that builds a CLI that relies on openstudio_ruby, as well as swig_installer and zlib (to embed files).

It's a CLI that has most features of openstudio's CLI, except openstudio-specific stuff (like ApplyMeasure, Run, Update).

It should hopefully help us:

  • Catch things that break early on, before we notice it on OpenStudio's side
  • Allow us to test/debug cli ruby specific issues (encodings, gem_install, interactive, etc)

If you run the conan create command, it will automatically run the tests against the package it built right after. From the root of the repo:

conan create . nrel/testing

If you only wanted to run the tests against a package you already have (downloaded or built), you can also do that like so:

conan test test_package openstudio_ruby/2.5.5@nrel/stable

Uploading to Bintray (unnecessary due to CI)

Full instructions available at Conan Docs.

Steps:

  • Get your API key by clicking on "View Profile"
  • Add remote to conan client
conan remote add <REMOTE> <YOUR_BINTRAY_REPO_URL>
# eg:
conan remote add nrel https://conan.openstudio.net/artifactory/api/conan/openstudio
  • Add your API key:
conan user -p <APIKEY> -r <REMOTE> <USERNAME>
# eg:
conan user -p <API_KEY> -r nrel commercialbuilding
  • Build your binaries locally and Upload them to your remote:
conan create . openstudio_ruby/2.5.5@nrel/testing -r nrel
conan upload openstudio_ruby/2.5.5@nrel/testing -r nrel

Copying a CCI recipe and vendoring it to NREL's remote

eg vendoring zlib/1.2.11

# Wipe dir to be sure
/bin/rm -Rf /home/julien/.conan/data/zlib/1.2.11

# Download recipe and all binary packages from CCI's remote
conan download -r conan-center zlib/1.2.11@

# Might want to make sure nothing is existing?
conan remove -r nrel zlib/1.2.11@

# Note: sometimes going to bintray and deleting the complete package helps (eg if you have inadvertantly uploaded several revisions and now you get "Upload skipped, package existing")

# Upload recipe
conan upload zlib/1.2.11@ -r nrel --all --no-overwrite recipe --parallel

# Check result?
conan search -r nrel zlib/1.2.11@
# Or output to a html table
conan search -r nrel zlib/1.2.11@ --table zlib.html
# Or a json
conan search -r nrel zlib/1.2.11@ --json zlib.json

Conan and the recipe hash: how to produce the same hash

This section is especially true should you need to manually build some packages or package configurations for example if CCI doesn't even have a package for the compiler or the recipe option you need.

eg: zlib with the minizip=True option)

Conan's hash is computed by looking at which files are exported. So you need to make sure that are using the same configuration on all machines.

Line endings

Line endings matter! This means that by the Git for windows default, you will not compute the same hash since it defaults to CRLF line endings.

git config --system core.autocrlf input
git config --system core.eol lf

If you had already checked out the repo before settings these settings, you need to tell git to pick the right line endings now:

cd conan-openstudio-ruby
git checkout-index --force --all
git rm --cached -r .  # Remove every file from git's index.
git reset --hard      # Rewrite git's index to pick up all the new line endings.

Revisions:

You conan.conf should contain general.revisions_enabled=True:

$ conan config get general.revisions_enabled
True

If not, activate it via editing ~/.conan.conf or by typing conan config set general.revisions_enabled=True

Hooks

The conan-center-index hook in particular will modify the conandata.yml if any, so if you are manually building dependencies from CCI, you need to be consistent. To activate the hook:

conan config install https://github.com/conan-io/hooks.git -sf hooks -tf hooks
conan config set hooks.conan-center

Make sure every machine returns the same:

$ conan config get hooks
attribute_checker,conan-center

Avoiding human errors for CCI packages

If you are trying to add packages for new configurations/options, instead of building from the repo, you should download then build. This is especially true for conan-center-index. Downloading the recipe and building it from cache is much better than creating or exporting the recipe from the conan-center-index repository. From the repository you need to check that you are in the same commit, that you have the same hooks, etc, and there will be human errors.

conan download -r nrel zlib/1.2.11@ --recipe
conan install zlib/1.2.11@ -b zlib -o zlib:minizip=True -s build_type=Release
conan install zlib/1.2.11@ -b zlib -o zlib:minizip=True -s build_type=Debug

Making sure you do not export a NEW recipe

Find the current revision

$ conan search -r nrel zlib/1.2.11@ -rev
0df31fd24179543f5720ec7beb2a88d7

Then specify it in the upload command! That way you're sure you're exporting the right one (= appending packages).

conan upload zlib/1.2.11@ -r nrel --all --parallel --no-overwrite all
conan upload zlib/1.2.11@:0df31fd24179543f5720ec7beb2a88d7 -r nrel --all --parallel --no-overwrite all

conan-openstudio-ruby's People

Contributors

jmarrec avatar lefticus avatar tijcolem avatar kbenne avatar

Stargazers

 avatar Chris Balbach avatar Arif Hanif avatar

Watchers

Anya Petersen avatar  avatar Alex Swindler avatar  avatar James Cloos avatar Dan Macumber avatar Michael Keck avatar National Renewable Energy Laboratory avatar Mark Adams avatar  avatar Katherine Fleming avatar Brian Ball avatar Luigi Gentile Polese avatar Elaine Hale avatar  avatar Yixing Chen avatar  avatar Chris Balbach avatar  avatar Marjorie Schott avatar Joe Robertson avatar Gort avatar Marvin avatar  avatar  avatar

conan-openstudio-ruby's Issues

Revert ruby_installer to bincrafters/stable

Had to build my own ruby_installer when updating to use Conan center index (cf #20 based on jmarrec#4)

My PR bincrafters/conan-ruby_installer#7 got merged now.

Change back to bincrafters/stable:

ruby_installer/2.5.5@jmarrec/testing # TODO: temp, revert to @bincrafters/stable

I'm not doing it right now because it involves a lengthy process (travis and appveyor to build over the course of several hours, making sure NREL/OpenStudio doesn't break in the process) and it works fine now.

Port to conan 2.0

Issue overview

Conan 2.0 has been officially released and includes a number of API breaking changes. Currently we can't build with conan 2.0

Fix or remove bundle_tests in test_package

Either we need to fix these or just disable/remove them.

Appveyor reports four failing tests on windows:

openstudio_ruby/2.5.5@nrel/testing (test package): ERROR: 4 tests Failed:
 * test\test_bundle.rb: test_bundle
 * test\test_bundle.rb: test_bundle_git
 * test\test_bundle.rb: test_bundle_no_install
 * test\test_bundle.rb: test_no_bundle

Travis reports 3 on both linux and mac:

openstudio_ruby/2.5.5@nrel/testing (test package): ERROR: 3 tests Failed:
 * test/test_bundle.rb: test_bundle
 * test/test_bundle.rb: test_bundle_git
 * test/test_bundle.rb: test_no_bundle

Patch source win32/Makefile.sub to use MSC_VER range instead of hardcoded one

win32/Makefile.sub#538

#if _MSC_VER != $(MSC_VER)
#error MSC version unmatch: _MSC_VER: $(MSC_VER) is expected.
#endif

MSVC 2019 has the range from 1920 to 1925 (currently), our current appveyor-built binary was on 1924, so if you use the latest MSVC 2019, you hit a hard error.

Abbreviation Product name [Visual Studio version] โ€ 1 VC ++ version โ€ 2 _MSC_VER _MSC_FULL_VER
2019 Update 5 Visual Studio 2019 version 16.5.1 14.25 1925 192528611
2019 Update 4 Visual Studio 2019 version 16.4.0 14.24 1924 192428314
2019 Update 3 Visual Studio 2019 version 16.3.2 14.21 1923 192328105
2019 Update 2 Visual Studio 2019 version 16.2.3 14.21 1922 192227905
2019 Update 1 Visual Studio 2019 version 16.1.2 14.21 1921 192127702
2019 Visual Studio 2019 version 16.0.0 14.20 1920 192027508
2017 Update 9 Visual Studio 2017 version 15.9.11 14.16 1916 191627030

Compute the list of .rb files at package generation time and install it in the package folder

The OpenStudio SDK CMake configure call spends an awful amount of time doing

Searching for ruby modules in '/Users/julien/.conan/data/openstudio_ruby/2.7.2/nrel/testing/package/b0b153961e79d4e95517b97d888ce78a877ec1ef/lib/**/*.rb'

https://github.com/NREL/OpenStudio/blob/c665b9fc52efde300e45b967498e2f7b9fdeb7b6/ruby/engine/CMakeLists.txt#L24-L191

We could save this time by doing it when we generate the package instead of everytime we call CMake.

The same should be done in openstudio-gems.

FYI @tijcolem (I just thought of this)

Conan Dependencies: Next Steps

Following #7 Next steps:

  • Investigate why with_libyaml=True fails to link on Windows
  • Instead of always force-building some conan dependencies in OpenStudio/ConanInstall.cmake, find a better alternative.
    • currently on mac for eg websocket, boost_asio and readline are always built each time you call cmake
    • one is to upload a nrel/readline recipe that doesn't contain binary packages, so that the first time you build OpenStudio you must build it from source
  • Investigate why conan-libffi doesn't succeed to link on Windows (I started to tweak the recipe here: https://github.com/jmarrec/conan-libffi/tree/testing/3.2.1)
  • Remove the need for CMakeLists.txt: it works on Unix but having problems on windows due to libffi. Given the urgency of our deliverables I have put this effort on hold, but it's saved on jmarrec#3
  • Missing deps on windows
    • Provide readline via conan on MSVC
    • provide gdbm
  • Check if possible to enable static builds of conan-readline on macOS
  • Check if needed to provide conan-gmp on Windows:
    • If so, check if possible to provide conan-gmp on Windows

Add test_package that mimics our CLI

Perhaps instead of waiting for things to break on OpenStudio's side, we should create a test_package (cf https://conan-docs.readthedocs.io/en/feature-cmake-toolchain-file/reference/commands/test_package.html) that would actually compile a small cpp program and link against our conan openstudio_ruby

That would automatically be run on travis/appveyor via conan-package-tools, and could help catch problems early on without having to recompile a much larger program like OpenStudio.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.