Giter Club home page Giter Club logo

mini_portile's Introduction

MiniPortile

This documents versions 2 and up, for which the require file was renamed to mini_portile2. For mini_portile versions 0.6.x and previous, please visit the v0.6.x branch.

Continuous Integration Tidelift dependencies

This project is a minimalistic implementation of a port/recipe system for developers.

Because "Works on my machine" is unacceptable for a library maintainer.

Not Another Package Management System

mini_portile2 is not a general package management system. It is not aimed to replace apt, macports or homebrew.

It's intended primarily to make sure that you, as the developer of a library, can reproduce a user's dependencies and environment by specifying a specific version of an underlying dependency that you'd like to use.

So, if a user says, "This bug happens on my system that uses libiconv 1.13.1", mini_portile2 should make it easy for you to download, compile and link against libiconv 1.13.1; and run your test suite against it.

This scenario might be simplified with something like this:

rake compile LIBICONV_VERSION=1.13.1

(For your homework, you can make libiconv version be taken from the appropriate ENV variables.)

Sounds easy, but where's the catch?

At this time mini_portile2 only supports autoconf- or configure-based projects. (That is, it assumes the library you want to build contains a configure script, which all the autoconf-based libraries do.)

As of v2.2.0, there is experimental support for CMake-based projects. We welcome your feedback on this, particularly for Windows platforms.

How to use (for autoconf projects)

Now that you know the catch, and you're still reading this, here is a quick example:

gem "mini_portile2", "~> 2.0.0" # NECESSARY if used in extconf.rb. see below.
require "mini_portile2"
recipe = MiniPortile.new("libiconv", "1.13.1")
recipe.files = ["http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.13.1.tar.gz"]
recipe.cook
recipe.activate

The gem version constraint makes sure that your extconf.rb is protected against possible backwards-incompatible changes to mini_portile2. This constraint is REQUIRED if you're using mini_portile2 within a gem installation process (e.g., extconf.rb), because Bundler doesn't enforce gem version constraints at install-time (only at run-time.

#cook will download, extract, patch, configure and compile the library into a namespaced structure.

#activate ensures GCC will find this library and prefer it over a system-wide installation.

Some keyword arguments can be passed to the constructor to configure the commands used:

gcc_command

The compiler command that is used is configurable, and in order of preference will use:

  • the CC environment variable (if present)
  • the gcc_command value passed in to the constructor
  • RbConfig::CONFIG["CC"]
  • "gcc"

You can pass it in like so:

MiniPortile.new("libiconv", "1.13.1", gcc_command: "cc")

make_command

The configuration/make command that is used is configurable, and in order of preference will use:

  • the MAKE environment variable (if present)
  • the make_command value passed in to the constructor
  • the make environment variable (if present)
  • "make"

You can pass it in like so:

MiniPortile.new("libiconv", "1.13.1", make_command: "nmake")

open_timeout, read_timeout

By default, when downloading source archives, MiniPortile will use a timeout value of 10 seconds. This can be overridden by passing a different value (in seconds):

MiniPortile.new("libiconv", "1.13.1", open_timeout: 99, read_timeout: 2)

How to use (for cmake projects)

Same as above, but instead of MiniPortile.new, call MiniPortileCMake.new.

make_command

This is configurable as above, except for Windows systems where it's hardcoded to "nmake".

cmake_command

The cmake command used is configurable, and in order of preference will use:

  • the CMAKE environment variable (if present)
  • the :cmake_command keyword argument passed into the constructor
  • "cmake" (the default)

You can pass it in like so:

MiniPortileCMake.new("libfoobar", "1.3.5", cmake_command: "cmake3")

cmake_build_type

The cmake build type is configurable as of v2.8.5, and in order of preference will use:

  • the CMAKE_BUILD_TYPE environment variable (if present)
  • the :cmake_build_type keyword argument passed into the constructor
  • "Release" (the default)

You can pass it in like so:

MiniPortileCMake.new("libfoobar", "1.3.5", cmake_build_type: "Debug")

Local source directories

Instead of downloading a remote file, you can also point mini_portile2 at a local source directory. In particular, this may be useful for testing or debugging:

gem "mini_portile2", "~> 2.0.0" # NECESSARY if used in extconf.rb. see below.
require "mini_portile2"
recipe = MiniPortile.new("libiconv", "1.13.1")
recipe.source_directory = "/path/to/local/source/for/library-1.2.3"

Directory Structure Conventions

mini_portile2 follows the principle of convention over configuration and established a folder structure where is going to place files and perform work.

Take the above example, and let's draw some picture:

mylib
  |-- ports
  |   |-- archives
  |   |   `-- libiconv-1.13.1.tar.gz
  |   `-- <platform>
  |       `-- libiconv
  |           `-- 1.13.1
  |               |-- bin
  |               |-- include
  |               `-- lib
  `-- tmp
      `-- <platform>
          `-- ports

In above structure, <platform> refers to the architecture that represents the operating system you're using (e.g. i686-linux, i386-mingw32, etc).

Inside the platform folder, mini_portile2 will store the artifacts that result from the compilation process. The library is versioned so you can keep multiple versions around on disk without clobbering anything.

archives is where downloaded source files are cached. It is recommended you avoid trashing that folder to avoid downloading the same file multiple times (save bandwidth, save the world).

tmp is where compilation is performed and can be safely discarded.

Use the recipe's #path to obtain the full path to the installation directory:

recipe.cook
recipe.path # => /home/luis/projects/myapp/ports/i686-linux/libiconv/1.13.1

How can I combine this with my compilation task?

In the simplest case, your rake compile task will depend on mini_portile2 compilation and most important, activation.

Example:

task :libiconv do
  recipe = MiniPortile.new("libiconv", "1.13.1")
  recipe.files << {
    url: "http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.13.1.tar.gz"],
    sha256: "55a36168306089009d054ccdd9d013041bfc3ab26be7033d107821f1c4949a49"
  }
  checkpoint = ".#{recipe.name}-#{recipe.version}.installed"

  unless File.exist?(checkpoint)
    recipe.cook
    touch checkpoint
  end

  recipe.activate
end

task :compile => [:libiconv] do
  # ... your library's compilation task ...
end

The above example will:

  • download and verify integrity the sources only once
  • compile the library only once (using a timestamp file)
  • ensure compiled library is activated
  • make the compile task depend upon compiled library activation

As an exercise for the reader, you could specify the libiconv version in an environment variable or a configuration file.

Download verification

MiniPortile supports HTTPS, HTTP, FTP and FILE sources for download. The integrity of the downloaded file can be verified per hash value or PGP signature. This is particular important for untrusted sources (non-HTTPS).

Hash digest verification

MiniPortile can verify the integrity of the downloaded file per SHA256, SHA1 or MD5 hash digest.

  recipe.files << {
    url: "http://your.host/file.tar.bz2",
    sha256: "<32 byte hex value>",
  }

PGP signature verification

MiniPortile can also verify the integrity of the downloaded file per PGP signature.

  public_key = <<-EOT
    -----BEGIN PGP PUBLIC KEY BLOCK-----
    Version: GnuPG v1

    mQENBE7SKu8BCADQo6x4ZQfAcPlJMLmL8zBEBUS6GyKMMMDtrTh3Yaq481HB54oR
    [...]
    -----END PGP PUBLIC KEY BLOCK-----
  EOT

  recipe.files << {
    url: "http://your.host/file.tar.bz2",
    gpg: {
      key: public_key,
      signature_url: "http://your.host/file.tar.bz2.sig"
    }
  }

Please note, that the gpg executable is required to verify the signature. It is therefore recommended to use the hash verification method instead of PGP, when used in extconf.rb while gem install.

Native and/or Cross Compilation

The above example covers the normal use case: compiling dependencies natively.

MiniPortile also covers another use case, which is the cross-compilation of the dependencies to be used as part of a binary gem compilation.

It is the perfect complementary tool for rake-compiler and its cross rake task.

Depending on your usage of rake-compiler, you will need to use host to match the installed cross-compiler toolchain.

Please refer to the examples directory for simplified and practical usage.

Supported Scenarios

As mentioned before, MiniPortile requires a GCC compiler toolchain. This has been tested against Ubuntu, OSX and even Windows (RubyInstaller with DevKit)

Support

The bug tracker is available here:

Consider subscribing to Tidelift which provides license assurances and timely security notifications for your open source dependencies, including Loofah. Tidelift subscriptions also help the Loofah maintainers fund our automated testing which in turn allows us to ship releases, bugfixes, and security updates more often.

Security

See SECURITY.md for vulnerability reporting details.

License

This library is licensed under MIT license. Please see LICENSE.txt for details.

mini_portile's People

Contributors

amatsuda avatar atul9 avatar b-dean avatar e2 avatar eagletmt avatar flavorjones avatar halfbyte avatar hanazuki avatar jtarchie avatar jvshahid avatar kamoh avatar kirikak2 avatar knu avatar larskanis avatar luislavena avatar metaskills avatar mudge avatar petergoldstein avatar quickshiftin avatar stanhu avatar watson1978 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mini_portile's Issues

Support defining pre and post events for actions

Sometimes you don't want to override an action, but you want to adjust things that action does.

For example, set environment variables like CC, LD, RANLIB and others to ensure compilation of a recipe succeed. Also could be used to configuration options conditionally at runtime instead at definition time (in case cross compilation has been enabled after the recipe was defined)

These could be defined similar to this:

recipe = MiniPortile.new("zlib", "1.2.5")
recipe.before :compile do
  ENV["CC"] = "#{host}-gcc"
  ENV["LD"] = "#{host}-ld"

  # or ...
  options[:makefile] = "win32/Makefile.gcc"
end

A basic mockup of the invocation could be:

# (MiniPortile internals)
def compile
  fire :compile do
    execute('compile', 'make')
  end
end

Space in Xcode.app pathname causes libxml2 not being able to make

This isn't really mini_portile's fault, but it'd be great to find a way to handle the case here.

You can reproduce the issue (on OSX) by

  1. mv /Applications/Xcode.app /Applications/Xcode\ 6.app
  2. gem install nokogiri

Nokogiri should fail to install. If you dump ~/.rvm/gems/ruby-2.1.1/gems/nokogiri-1.6.4/ext/nokogiri/tmp/x86_64-apple-darwin12.5.0/ports/libxml2/2.9.2/compile.log, you'll see it's because Autoconf failed to escape the path in /Applications/Xcode\ 6.app

Apparently, autconf's failure to handle spaces in path names is a known-issue http://stackoverflow.com/questions/16194124/autoconf-spaces-in-path

Can't change target

Target default (ports directory) can't be changed because target is marked as read-only.

Ideally developers should be able to bend this convention.

Support actions override

To our bad, not all the software packages out there follow a configure/compile/install strategy. Not all provide a configure script or accept the same arguments autoconf scripts does.

At this time you can code your own custom actions, but that means you loose the ability to simply invoke cook on your recipe.

It will be handy to be able to override actions so MiniPortile#cook can still be used.

An idea:

recipe = MiniPortile.new("openssl", "1.0.0d")
recipe.action :configure do
  execute("configure", "perl Configure #{options}")
end

Provided action blocks should have access to MiniPortile helpers, like execute or extract_file

Make dead-easy to generate Rake tasks

Right now, as indicated by README, you need to write 7 to 9 lines of code for each library you want to use.

This do not scale properly when you have 3 or 4 recipes and need to update them (because you change your mind in what could be cool or better approach)

A suggestion: create something like MiniPortile::Task that used in combination with the recipe it generates the ports:recipe Rake task for you.

use MakeMakefile.find_executable in place of MiniPortile#which

MiniPortile currently has a private methods, which, that searches the $PATH environment variable for an executable, and makes sure to do it safely on Windows as well as POSIX systems.

But Ruby's MakeMakefile module has a method find_executable that does the same thing. Let's use that instead and delete some code.

ERROR 'xf' is not recognized

require "mini_portile"
recipe = MiniPortile.new("libiconv", "1.14")
recipe.files = ["http://ftp.gnu.org/pub/gnu/libiconv/libiconv-
1.14.tar.gz"]
recipe.cook
recipe.activate

err:

Extracting libiconv-1.14.tar.gz into tmp//ports/libiconv/1.14... ERROR
'xf' is not recognized as an internal or external command,
operable program or batch file.
C:/Ruby200/lib/ruby/gems/2.0.0/gems/mini_portile-0.5.1/lib/mini_portile.rb:234:i
n `extract_file': Failed to complete extract task (RuntimeError)
        from C:/Ruby200/lib/ruby/gems/2.0.0/gems/mini_portile-0.5.1/lib/mini_por
tile.rb:34:in `block in extract'
        from C:/Ruby200/lib/ruby/gems/2.0.0/gems/mini_portile-0.5.1/lib/mini_por
tile.rb:32:in `each'
        from C:/Ruby200/lib/ruby/gems/2.0.0/gems/mini_portile-0.5.1/lib/mini_por
tile.rb:32:in `extract'
        from C:/Ruby200/lib/ruby/gems/2.0.0/gems/mini_portile-0.5.1/lib/mini_por
tile.rb:98:in `cook'
        from a.rb:5:in `<main>'

I'think it is a bug :)

Trigger configure process if configure options change

Right now the markers for configured? are configure script and generated makefile.

Any change in the configure options (config_options) will not trigger a configure again.

Thinking MD5 the options and store it, compare again.

For that to work will also require we evaluate configure options like --prefix too.

Applying patch: git apply, worktrees and invalid gitdir

I'm using recently released worktrees git feature, and having issues with applying patches.
Also I use Vagrant, and it renders gitdir path within worktree invalid.

Here is an example.

mkdir /tmp/git_apply_issue
cd /tmp/git_apply_issue
# Simulate worktree with invalid path
echo 'gitdir: /nonexistent' > .git
echo "source 'https://rubygems.org'\n\ngem 'nokogiri'" > Gemfile
bundle install --path ./.bundle

# Extracting libxml2-2.9.2.tar.gz into tmp/x86_64-apple-darwin15.2.0/ports/libxml2/2.9.2... OK
# Running git apply with /tmp/git_apply_issue/.bundle/ruby/2.3.0/gems/nokogiri-1.6.7.2/patches/libxml2/0001-Revert-Missing-initialization-for-the-catalog-module.patch... ERROR, review '/tmp/git_apply_issue/.bundle/ruby/2.3.0/gems/nokogiri-1.6.7.2/ext/nokogiri/tmp/x86_64-apple-darwin15.2.0/ports/libxml2/2.9.2/patch.log' to see what happened. Last lines are:
# ========================================================================
# fatal: Not a git repository: /nonexistent
# ========================================================================
...

I believe it is caused by behavior of git apply itself. Apparently --work-tree=. does not prevent git from traversing up and looking for git repo files.

git apply command is being executed within /tmp/git_apply_issue/.bundle/ruby/2.3.0/gems/nokogiri-1.6.7.2/ext/nokogiri.

Is there a way to avoid this error?

Consistently use host (compared against rake-compiler)

MiniPortile uses RbConfig::CONFIG["arch"] as host reference, and that is inconsistent with rake-compiler, which uses host instead.

This do not present a issue when doing configure --host=i686-darwin10.6.0 on OSX, but is a problem when natively compiling on Windows and the compiler toolchain used is mingw-w64 and their executables are prefixed by the platform triplet.

Example, configure --host=i386-mingw32 will attempt to use mingw's GCC instead of i686-w64-mingw32 (mingw-w64 one)

Host reference needs to be consistent, so I recommend usage of RbConfig::CONFIG["host"] instead.

NetBSD's tar doesn't recognize compress/gzip/bzip2 file automatically.

On NetBSD, /bin/tar doesn't recognize compress/gzip/bzip2 compression automatically. I got the following erros when I tried "gem install nokogiri".

Extracting libxml2-2.8.0.tar.gz into tmp/x86_64--netbsd/ports/libxml2/2.8.0... ERROR
tar: Cannot identify format. Searching...
tar: Cpio file name length 10951 is out of range
tar: Invalid header, starting valid header search.
tar: Cpio file name length 53995 is out of range
tar: Cpio file name length 30875 is out of range

[snip]

tar: Cpio file name in header is corrupted
tar: End of archive volume 1 reached
tar: Unexpected EOF on archive file
*** extconf.rb failed ***

If I have gnu tar or bsdtar (I can easily install them from pkgsrc, NetBSD package system), the patch works fine to me. I'm not sure if it's the right fix.

--- mini_portile.rb.orig 2013-10-12 19:38:21.000000000 -0700
+++ mini_portile.rb 2013-10-14 13:09:45.000000000 -0700
@@ -186,7 +186,7 @@

def tar_exe
@@tar_exe ||= begin

  •  %w[tar bsdtar basic-bsdtar].find { |c|
    
  •  %w[gtar bsdtar tar basic-bsdtar].find { |c|
     which(c)
    
    }
    end

Figure out why jruby-9.0.0.0 is failing tests

Currently I get this failure:

Running git apply with /home/flavorjones/code/oss/mini_portile/test/assets/patch 1.diff...
fatal: can't open patch '/home/flavorjones/code/oss/mini_portile/test/assets/patch': No such file or directory

the contents of the command are an array:

["git", "--work-tree=.", "apply", "/home/flavorjones/code/oss/mini_portile/test/assets/patch 1.diff"]

which indicates there's some kind of whitespace-handling that needs to be done. However, this test appears to pass on jruby-9.0.0.0:

  def test_system_with_space_in_arg
    FileUtils.mkdir_p 'test_dir'
    File.open('test_dir/patch 1.diff', 'w') { |f| f.write "Hello, world." }
    system("ls test_dir")
    command = ["git", "--work-tree=.", "apply", File.expand_path("test_dir/patch 1.diff")]
    puts command.inspect
    assert(system(*command))
  ensure
    FileUtils.rm_rf 'test_dir'
  end

so I'm mystified as to what's going on.

#tar_exe doesn't find tar on IllumOS (omnios)

I encountered this problem on OmniOS while attempting to install the nokogiri gem.

Extracting libxml2-2.8.0.tar.gz into tmp//ports/libxml2/2.8.0... ERROR
sh: line 1: xf: not found

I tracked it down to this. The #tar_exe method attempts to run:

tar --version

However, "solaris" tar (as used on IllumOS distributions) does not have a --version argument.

root@menthe:~# tar --version
tar: s: unknown function modifier
Usage: tar {c|r|t|u|x}[BDeEFhilmnopPTvw@/[0-7]][bf][X...] [j|J|z|Z] [blocksize] [tarfile] [size] [exclude-file...] {file | -I include-file | -C directory file}...
root@menthe:~# echo $?
1

It would be better to use a #which method that checks the $PATH for the specified binary. For example, mislav posted this answer on stackoverflow.

This is likely a bug in the #tar_exe method for other IllumOS derivatives (like SmartOS) and possibly Solaris 10 & 11.

test both `git apply` and `patch`

Currently, the test suite picks git apply if git is installed, else patch.

On both Travis and Appveyor, as well as any likely developer machine, this means that git is tested but patch is not.

I bring this up because gnuwin32's patch.exe on Windows does not handle embedded whitespace in a patch's candidate file name, and so TestCook#test_patch and TestCookWithBrokenGitDir#test_patch both fail on Windows when git is not installed.

I would like to take over maintenance of `mini_portile`

Hey Luis,

Since Nokogiri makes great use of mini_portile, in some ways not originally intended, I guess I'm as good a candidate as any to take it over. Let me know how you'd like to proceed. At the very least, I'd like to get a braindump from you with respect to any outstanding urgent issues and what you think should be on the roadmap.

It was great meeting you in person that one strange day in New York. I'm very sorry to see your farewell, and I can only hope that it isn't forever; or that I at least get to buy you another beer.

Cheers,
-m

running as root causes tar to try and set owner/group

We run mini_portile in a docker container, which defaults to a root user.
When tar (on ubuntu lucid) is asked to untar a src distribution, it defaults to setting owner and group, which it cannot do inside the container. Our workaround is to pass --no-same-owner to tar. --no-same-owner is the default behaviour in tar when the user is not root.

Currently we monkey patch mini_portile - do you have any recommendations?

There are a lot of integration environments for mini_portile, so not sure how reliable this patch would be for all users of mini_portile.

configure.md5 hashing with MD5 is not FIPS compliant

It looks like an MD5 digest is used in MiniPortile#configure and MiniPortile#configured?. It looks like security is not a concern given what it's used for. However, this will fail on an OS running in FIPS mode as MD5 is not an accepted digest algorithm. One option is to use a supported algorithm, such as Digest::SHA256, but that may not even be necessary as you might even be okay to use computed_options.to_s without hashing it. Thoughts?

For reference, the failure can be demonstrated on a FIPS-enabled system as follows:

irb(main):001:0> require 'digest/md5'
=> true
irb(main):002:0> Digest::MD5.hexdigest("anything")
md5_dgst.c(78): OpenSSL internal error, assertion failed: Digest MD5 forbidden in FIPS mode!
Aborted (core dumped)

"RuntimeError: Digest::Base cannot be directly inherited in Ruby"

This happend after updating all gems to the latest version:
https://hub.docker.com/r/wpscanteam/wpscan/builds/b5xh8jdtu33s3u9hcj8alec/

RuntimeError: Digest::Base cannot be directly inherited in Ruby
An error occurred while installing mini_portile2 (2.3.0), and Bundler cannot
continue.
Make sure that `gem install mini_portile2 -v '2.3.0'` succeeds before bundling.

This seems to be introduced in this commit: c8d3442#diff-03dcf58e9c53b59fc2bf7d60d421e400L7

File extraction should accept options

Some packages like OpenSSL contains symlinks. While symlinks are not a problem on POSIX systems, they are for extracting tools on Windows.

MSYS tar or bsdtar choke and return an error when extracting them, which causes the extract action to fail:

openssl-0.9.8q/apps/md4.c: Can't create 'openssl-0.9.8q/apps/md4.c'
openssl-0.9.8q/include/openssl/aes.h: Can't create 'openssl-0.9.8q/include/openssl/aes.h'
openssl-0.9.8q/include/openssl/asn1.h: Can't create 'openssl-0.9.8q/include/openssl/asn1.h'

Will be great if extract_file supported options like :ignore_errors => true

cannot compile due to a `spawn': no implicit conversion of nil into String

I cannot get my gems to compile because nokogiri fails to build and it fails with an error from mini_portile. do you guys have insight on this?

Env: clean Fedora 26, ruby 2.4.1p111. I think I have install all the other libraries and packages required. This process has worked before for a few months on Fedora 25 (with possibly older versions of the same gems)

..vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:344:in `spawn': no implicit conversion of nil into String (TypeError)
	from /home/cpg/hda-platform/html/vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:344:in `block in execute'
	from /home/cpg/hda-platform/html/vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:337:in `chdir'
	from /home/cpg/hda-platform/html/vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:337:in `execute'
	from /home/cpg/hda-platform/html/vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:331:in `extract_file'
	from /home/cpg/hda-platform/html/vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:57:in `block in extract'
	from /home/cpg/hda-platform/html/vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:56:in `each'
	from /home/cpg/hda-platform/html/vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:56:in `extract'
	from /home/cpg/hda-platform/html/vendor/bundle/ruby/2.4.0/gems/mini_portile2-2.1.0/lib/mini_portile2/mini_portile.rb:147:in `cook'
	from extconf.rb:364:in `block (2 levels) in process_recipe'
	from extconf.rb:257:in `block in chdir_for_build'
	from extconf.rb:256:in `chdir'
	from extconf.rb:256:in `chdir_for_build'
	from extconf.rb:363:in `block in process_recipe'
	from extconf.rb:262:in `tap'
	from extconf.rb:262:in `process_recipe'
	from extconf.rb:547:in `<main>'

MiniPortile + configure in $PATH + non bourne /bin/sh + libxml2 = kaboom

Testcase:

  • Ensure /bin/sh is not a bourne shell (/bin/dash, for example)
  • Create a "configure" file with the following content:
#! /bin/sh
echo foo
exit 1
  • Make it executable and place it in a directory in your $PATH, or add the directory containing it in your $PATH
  • gem install nokogiri

The install fails with a message saying:
Running 'configure' for libxml2 2.9.2... ERROR, review '(...)/configure.log' to see what happened.
That configure.log contains "foo".

Why is there a configure in the PATH will you ask? Well, there are plenty of reasons you can end up with this (like building git yourself and adding the build directory in your PATH, not knowing you should add the bin-wrappers subdirectory instead).

The reason this is happening is that nokogiri's build script use MiniPortfile to compile libxml2, and MiniPortfile calls "sh configure" to run libxml2's configure. Presumably because Windows wouldn't know how to run "configure" alone. Now, this would be fine is libxml2's configure was not trying to reexecute itself with a bourne shell. To do so (this is an autoconf thing), it munges $0 and runs bash $the_result_of_the_munging.
The munging of $0 is done as follows:

  • if it contains a / (like ./configure or $some_relative_path/configure would), keep $0 intact
  • otherwise, scan $PATH to find a file named $0 (which is just "configure"). So it will expand "configure" to the path of that "configure" script in your $PATH and execute that.

The following would fix this:

diff --git a/lib/mini_portile/mini_portile.rb b/lib/mini_portile/mini_portile.rb
index 6913400..4598e21 100644
--- a/lib/mini_portile/mini_portile.rb
+++ b/lib/mini_portile/mini_portile.rb
@@ -63,7 +63,7 @@ class MiniPortile
     digest   = Digest::MD5.hexdigest(computed_options)
     File.open(md5_file, "w") { |f| f.write digest }

-    execute('configure', %Q(sh configure #{computed_options}))
+    execute('configure', %Q(sh ./configure #{computed_options}))
   end

   def compile

Downloading Github releases (which redirect to S3) give an odd error

Reported by @dmikusa-pivotal via direct email:

require 'mini_portile'
recipe = MiniPortile.new("rabbitmq-c", "0.5.2")
recipe.files = ["https://github.com/alanxz/rabbitmq-c/releases/download/v0.5.2/rabbitmq-c-0.5.2.tar.gz"]
recipe.cook

gives us this error once it's redirected to the S3 bucket:

Error: Client Error: #<Net::HTTPForbidden 403 Forbidden readbody=false>

`git apply` fails depending on git configuration

My ~/.gitconfig had the setting autocrlf = input.

For some reason this caused mini_portile to fail to apply patches with git apply. I could not install nokogiri due to the problem. sparklemotion/nokogiri#1095

The solution in this case was to change my git configuration by removing the autocrlf setting.

Is there a better way to address this issue?

mini_portile-0.6.1 identification error

I want to install redmine_dmsf but when run rake command to migrate database into "bitnami redmine stack 2.6.0-2 / windows 7 / ruby version 2.0.0", I have met mini_portile-0.6.1 is required but missing.

And "bundle check" command shows that gem is not installed and recommend to execute "bundle install" command. But "bundle install" command showed not any effect., just same situation.

Actually mini_portile-0.6.1 was installed successfully, can be found by "gem list mini" command.

Could you help me to install this gem?

Eager memoization of paths is affecting cross compilation

Combined with rake-compiler, we need to use MiniPortile#path to be able to supply options to the extension configure process.

Because of that, the later change of MiniPortile#host is not reflected.

This is a side-effect that could be solved by Issue #6.

Putting this here as reminder.

Long installation time?

Installing this gem takes very long time on relatively fast machines? I don't see nothing in the code indicating it should take that long?

EDIT: it takes very long only if it is a part of a bundle. gem install mini_portile is fast, but bundle hangs on Installing mini_portile for a very long time. Could it be some misunderstanding between bundler and this gem, or am I completely wrong?

[Feature Request] Checksum mechanism needed

If files are to be downloaded on user's site, size & checksum information for each download file should be included in a recipe and the files must be checked upon before being used.

What I have in mind is something like this:

recipe.distinfo = {
  "file or url" => {
    size: 1048576,
    sha256: '35c5803b469bd178a70c060ed83ba22e52fc9e4e754fefa53f783ce242098953',
  },
}

Support for projects requiring out of source cmake builds

Some projects (in my case OSRM) has prevented in-source builds. Some projects do it in order to avoid conflicts with files and otherwise keep the build data separate from source files.

It seems that mini_portile does in-source builds only. Would it be possible to at least allow configuration of the build dir?

RailsInstaller Compatibility

Luis, from our conversation, this is a reminder that when I installed RailsInstaller 1.0.5 I did not have all that I needed to use MiniPortile.

I had to download bsdtar and place it inside C:\RailsInstaller\DevKit\mingw\bin as tar.exe

I also had to execute C:\RailsInstaller\DevKit\devkitvars.bat

Checksums should be verified at extraction time (in addition to download time)

I'd like to eliminate the possibility that files are being changed on disk after download, but before extraction (particularly because downloaded files are cached and used later if the compiled version is removed).

I would like to re-use the same checksum verification logic that's currently being used at download time.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.