Why depend on both vips and mini_magick? about image_processing HOT 10 CLOSED

Is it possible that the docker distro you are using comes with libvips pre-installed

When I ran ruby -e "require 'libvips'" as the Docker command (last line of Dockerfile), I see the same error you posted, which indicated to me libvips was not installed.

Regardless since it does seem at least at some point there is a C extension involved, it would make sense to consider this change for the next major version in any case.

Once I understand why this error happens at install time on Debian, I'm willing to consider changing this.

from image_processing.

I am also serious about having to install 500MB of packages to install libvips.

This is an issue with the debian package -- it include the libvips GUI in the recommends section. This might be a good idea for desktop use of libvips, but it's not useful on a server.

If you install the package without recommended dependencies, it's a much less insane size:

$ docker run --rm -it debian:bullseye
root@6d72f5065bae:/# apt update -qq
7 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@6d72f5065bae:/# apt install libvips42 --no-install-recommends
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  fontconfig fontconfig-config fonts-dejavu-core imagemagick-6-common libaec0
  libaom0 libbrotli1 libbsd0 libcairo-gobject2 libcairo2 libcfitsio9
  libcurl3-gnutls libdatrie1 libdav1d4 libde265-0 libdeflate0 libexif12
  libexpat1 libfftw3-double3 libfontconfig1 libfreetype6 libfribidi0
  libgdk-pixbuf-2.0-0 libgdk-pixbuf-xlib-2.0-0 libgdk-pixbuf2.0-0
  libgdk-pixbuf2.0-common libgif7 libglib2.0-0 libgomp1 libgraphite2-3
  libgsf-1-114 libgsf-1-common libharfbuzz0b libhdf5-103-1 libheif1 libicu67
  libilmbase25 libimagequant0 libjbig0 libjpeg62-turbo liblcms2-2
  libldap-2.4-2 liblqr-1-0 libltdl7 libmagickcore-6.q16-6 libmatio11 libmd0
  libnghttp2-14 libnspr4 libnss3 libnuma1 libopenexr25 libopenjp2-7
  libopenslide0 liborc-0.4-0 libpango-1.0-0 libpangocairo-1.0-0
  libpangoft2-1.0-0 libpixman-1-0 libpng16-16 libpoppler-glib8 libpoppler102
  libpsl5 librsvg2-2 librtmp1 libsasl2-2 libsasl2-modules-db libsqlite3-0
  libssh2-1 libsz2 libthai-data libthai0 libtiff5 libwebp6 libwebpdemux2
  libwebpmux3 libx11-6 libx11-data libx265-192 libxau6 libxcb-render0
  libxcb-shm0 libxcb1 libxdmcp6 libxext6 libxml2 libxrender1 sensible-utils
  shared-mime-info ucf
Suggested packages:
  libfftw3-bin libfftw3-dev liblcms2-utils libmagickcore-6.q16-6-extra
  librsvg2-bin
Recommended packages:
  ca-certificates libgdk-pixbuf2.0-bin libglib2.0-data xdg-user-dirs
  libldap-common ghostscript gsfonts poppler-data publicsuffix librsvg2-common
  libsasl2-modules nip2
The following NEW packages will be installed:
  fontconfig fontconfig-config fonts-dejavu-core imagemagick-6-common libaec0
  libaom0 libbrotli1 libbsd0 libcairo-gobject2 libcairo2 libcfitsio9
  libcurl3-gnutls libdatrie1 libdav1d4 libde265-0 libdeflate0 libexif12
  libexpat1 libfftw3-double3 libfontconfig1 libfreetype6 libfribidi0
  libgdk-pixbuf-2.0-0 libgdk-pixbuf-xlib-2.0-0 libgdk-pixbuf2.0-0
  libgdk-pixbuf2.0-common libgif7 libglib2.0-0 libgomp1 libgraphite2-3
  libgsf-1-114 libgsf-1-common libharfbuzz0b libhdf5-103-1 libheif1 libicu67
  libilmbase25 libimagequant0 libjbig0 libjpeg62-turbo liblcms2-2
  libldap-2.4-2 liblqr-1-0 libltdl7 libmagickcore-6.q16-6 libmatio11 libmd0
  libnghttp2-14 libnspr4 libnss3 libnuma1 libopenexr25 libopenjp2-7
  libopenslide0 liborc-0.4-0 libpango-1.0-0 libpangocairo-1.0-0
  libpangoft2-1.0-0 libpixman-1-0 libpng16-16 libpoppler-glib8 libpoppler102
  libpsl5 librsvg2-2 librtmp1 libsasl2-2 libsasl2-modules-db libsqlite3-0
  libssh2-1 libsz2 libthai-data libthai0 libtiff5 libvips42 libwebp6
  libwebpdemux2 libwebpmux3 libx11-6 libx11-data libx265-192 libxau6
  libxcb-render0 libxcb-shm0 libxcb1 libxdmcp6 libxext6 libxml2 libxrender1
  sensible-utils shared-mime-info ucf
0 upgraded, 91 newly installed, 0 to remove and 7 not upgraded.
Need to get 40.6 MB of archives.
After this operation, 146 MB of additional disk space will be used.
Do you want to continue? [Y/n]

from image_processing.

Thanks @jcupitt for additional info.

Since I wasn't able to reproduce the original issue, and since there haven't been other people reporting this issue, I'm closing this.

from image_processing.

However because of the hard dep on ruby-vips, bundle will fail unless libvips is installed.

I didn't know that 😮. On my Mac, the ruby-vips gem installed just fine without having libvips installed, because it doesn't have any C extensions. What OS are you using?

Wouldn't it be better for this gem to not depend on any of these, so you can choose yourself?

I chose this because I found it challenging to utilize features from newer mini_magick and ruby-vips versions, while still having to supporting old versions. Without making them dependencies, I couldn't bump their versions in a backwards-compatible way. Given that mini_magick has 0.1MB, and ruby-vips 0.5MB, I thought it's an OK trade-off to always install both.

from image_processing.

@janko when bundle installing my app on Debian 10 (Buster), I had to install libvips to be able to successfully bundle install. From Gemfile.lock, the only obvious reason for that would be ruby-vips.
The error was something like this:
Could not open library 'libvips.so.42': libvips.so.42: cannot open shared object file: No such file or directory

I don't think you're correct about ruby-vips not having deps on C extensions, see: libvips/ruby-vips#219

While I understand your point about version requirements on these gems, I am also serious about having to install 500MB of packages to install libvips. It probably depends on some Xorg stuff or something so while libvips itself might be small, all the dependencies aren't. Dependencies that it seems are not needed for example for imagemagick.

There are some rails gems that do this kind of thing (not explicitly depend on gems you can choose, so you have to add them to Gemfile yourself, one really obvious one is ActiveRecord with pg/mysql etc). As far as version goes you could probably check which gem is loaded and the version of that gem on runtime. Maybe not ideal but a totally workable solution.

from image_processing.

The error was something like this:
Could not open library 'libvips.so.42': libvips.so.42: cannot open shared object file: No such file or directory

Yeah, that sounds like ruby-vips trying to load a dynamic library, which I thought happens only when requiring libvips.

I don't think you're correct about ruby-vips not having deps on C extensions, see: libvips/ruby-vips#219

Did this issue talk about installing ruby-vips or requiring it? ruby-vips doesn't appear to have C extensions in terms of having C code in its codebase, which means there is nothing to compile at install time (there is no "building native extensions" message when installing the gem). I'm just trying to understand why anything gets run on install time, I don't see anything in ruby-vips.gemspec that would trigger it 🤔

While I understand your point about version requirements on these gems, I am also serious about having to install 500MB of packages to install libvips.

Yes, having to install libvips if you're going to use ImageMagick is unacceptable. I will test it out with a Debian docker image.

There are some rails gems that do this kind of thing (not explicitly depend on gems you can choose, so you have to add them to Gemfile yourself, one really obvious one is ActiveRecord with pg/mysql etc). As far as version goes you could probably check which gem is loaded and the version of that gem on runtime. Maybe not ideal but a totally workable solution.

Yeah, that's an alternative, it's just that this way I wouldn't be able to bump dependency versions, at least not in minor releases of image_processing (examples 1, 2, 3). It might be a good trade-off though, and it can be changed in version 2.0.

from image_processing.

I'm just trying to understand why anything gets run on install time, I don't see anything in ruby-vips.gemspec that would trigger it
I am using Spree (Rails e-commerce), and it does have a require 'ruby-vips' in it (in spree-core gem, lib/spree/core.rb). However as far as I know, gems don't actually get loaded during bundle install? In that case this can't be the reason. I don't have exact logs anymore but I am sure the error happened during bundle install (not during launching the app). The spree gemspec doesn't mention vips.

from image_processing.

I added this Gemfile, ran bundle install, and ran bundle lock --add-platform aarch64-linux.

source "https://rubygems.org"

gem "spree"

I then created this Dockerfile, which should allegedly be based on Debian:

FROM ruby:3.1.2-bullseye

# throw errors if Gemfile has been modified since Gemfile.lock
RUN bundle config --global frozen 1

WORKDIR /usr/src/app

COPY Gemfile Gemfile.lock ./
RUN bundle install

COPY . .

CMD ["cat", "/etc/issue"]

Running docker build . ran the bundle install just fine. It would be helpful if you could reproduce the issue on Docker, because I don't have access to a Debian machine. I believe you that it happens on bundle install, I just need to understand why before taking further steps.

from image_processing.

Hm that sounds like it should reproduce the issue.
Is it possible that the docker distro you are using comes with libvips pre-installed? Or it uses libvips from the host (I'm not sure exactly how docker works)? The VPS I am using probably came with a minimal distribution of Debian without libvips.
I don't remember having to install anything on my Mac, only had the issue when deploying to the VPS.
I'm not really versed with Docker, although I am willing to try it, it would probably take quite a while for me to get to it. Regardless since it does seem at least at some point there is a C extension involved, it would make sense to consider this change for the next major version in any case. If runtime checks are not ok you could also make the gem only depend on vips (seems like it's "default" now in Rails), and just make another gemspec for image_processing_magick or something.

from image_processing.

When I ran ruby -e "require 'libvips'" as the Docker command (last line of Dockerfile), I see the same error you posted, which indicated to me libvips was not installed.

I see, that's interesting. So weird.

from image_processing.