Giter Club home page Giter Club logo

exrm_deb's Introduction

Elixir Release Manager DEB generator

Coverage Status Build Status Hex version Inline docs

Adds simple Debian Package (DEB) generation to the exrm package manager.

Functionality list

  1. Able to build debian packages:
    1. With changelog
    2. With control file
  2. Ability to add in pre/post install/remove scripts
  3. Validates configuration before completing the build
  4. Add ability for you to replace file templates with your own
  5. Distillery support
  6. Automatically builds init scripts:
    1. Upstart
    2. Systemd
    3. SysV
  7. Handle functionality for Hot Upgrades
  8. Merge debian to-be-deployed files with your own structure

External dependencies

Before using exrm-deb, you'll need the following commands installed and in your path:

  • tar (or gtar if you're on a mac - you can brew install gnu-tar if you don't already have it)
  • ar
  • uname

Configuration

Exrm-deb relies on the following data in the mix.exs file being set:

defmodule Testapp.Mixfile do
   use Mix.Project

   def project do
      [app: :testapp,
      version: "0.0.1",
      elixir: "~> 1.0",
+     description: "Create a deb for your elixir release with ease",
      build_embedded: Mix.env == :prod,
      start_permanent: Mix.env == :prod,
-     deps: deps]
+     deps: deps,
+     package: package]
   end

The package function must be set as per the hex guidelines, but with some extra lines:

def package do
   [
+     external_dependencies: [],
+     codename: lsb_release(),
+     license_file: "LICENSE",
      files: [ "lib", "mix.exs", "README*", "LICENSE"],
+     config_files: ["/etc/init/api.conf"],
      maintainers: ["John Hamelink <[email protected]>"],
      licenses: ["MIT"],
      vendor: "John Hamelink",
      links:  %{
        "GitHub" => "https://github.com/johnhamelink/testapp",
        "Docs" => "hexdocs.pm/testapp",
+       "Homepage" => "https://github.com/johnhamelink/testapp"
      }
   ]
end
def lsb_release do
  {release, _} = System.cmd("lsb_release", ["-c", "-s"])
  String.replace(release, "\n", "")
end

A list of configuration options you can add to package/0:

  • config_file
    • Array of Strings
    • Should contain the absolute path of the config file to be overwritten.
  • licenses
    • Array of Strings
    • Can be something like ["Copyright <date> <company_name>"] if you are building private packages.
  • maintainers
    • Array of Strings
    • Should be in the format name <email>
  • external_dependencies
    • Array of Strings
    • Should be in the format of package-name (operator version_number) where operator is either <<, <=, =, >=, or >> - read more about this here.
  • maintainer_scripts
    • A keyword list of Strings
    • The keyword should be one of: :pre_install, :post_install, :pre_uninstall, or :post_uninstall
    • The keyword should point to the path of a script you want to run at the moment in question.
  • vendor
    • String
    • The distribution vendor that's creating the debian package. I normally just put my name or company name.
  • owner
    • A keyword list of Strings
    • If set, requires both user and group keys to be set.
    • This is used when building the archive to set the correct user and group
    • Defaults to root for user & group.
  • codename
    • String
    • Should contain the distribution codename to be chained to version number.

Additional details about codename

This configuration can be very useful in case you want to package the same version of the app for different distributions dynamically, without modifying the version in Distillery configuration.

A typical use case can be an environment where you have different Docker containers, and a different OS: each container compiles and packages the application in the running OS, in order to avoid startup problems in production.

With codename, at the end of the process, you obtain a package in the form myapp-1.2.1~xenial_amd64.deb. Also the control script in the deb file is packaged with the correct version like 1.2.1~xenial. At this point, it's easier to manage the packages loaded in a repository, because they are versioned also by distribution.

Usage

Building Deb file

Distillery

You can build a deb by adding plugin ExrmDeb.Distillery to your rel/config.exs file. With distillery, the name and version is taken from the rel/config.exs file as opposed to the mix.exs file.

Exrm

You can build a deb at the same time as building a release by adding the --deb option to release.

mix release --deb

This task first constructs the release using exrm, then generates a deb file for the release. The deb is built from scratch, retrieving default values such as the name, version, etc using the mix.exs file.

The _build/deb directory tree, along with the rest of the release can be removed with mix release.clean

Please visit exrm for additional information.

Customising deb config files

You can customise the debs that are being built by copying the template files used and modifying them:

mix release.deb.generate_templates

When you next run mix release --deb, your custom templates will be used instead of the defaults inside the plugin.

Installation

The package can be installed as:

  1. Add exrm_deb to your list of dependencies in mix.exs:

    def deps do [{:exrm_deb, "~> 0.0.1"}] end

  2. Ensure exrm_deb is started before your application:

    def application do [applications: [:exrm_deb]] end

exrm_deb's People

Contributors

0chroma avatar eraserewind avatar hermanverschooten avatar johnhamelink avatar michalmuskala avatar samuelebistoletti avatar tverlaan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

exrm_deb's Issues

Undefined function :swab.sync/2

==> Failed to execute after_release hook for Elixir.ReleaseManager.Plugin.Deb!
** (UndefinedFunctionError) undefined function :swab.sync/2 (module :swab is not available)
:swab.sync([convert: :gunzip, tar: :fakeroot, convert: :gzip], <<31, 139, 8, [..snip..]>>
   lib/exrm_deb/utils/compression.ex:36: ExrmDeb.Utils.Compression.compress/3
    lib/exrm_deb/data.ex:20: ExrmDeb.Data.build/2
    lib/exrm_deb.ex:26: ReleaseManager.Plugin.Deb.start_build/1
    lib/mix/tasks/release.ex:262: anonymous fn/2 in Mix.Tasks.Release.execute_after_hooks/1
    (elixir) lib/enum.ex:1473: Enum."-reduce/3-lists^foldl/2-0-"/3
    lib/mix/tasks/release.ex:80: Mix.Tasks.Release.do_run/1
    (mix) lib/mix/cli.ex:58: Mix.CLI.run_task/2

Systemd script not working

Systemd script is not working when trying to start the application packaged with distillery.

I got this error:
Active: failed (Result: exit-code) since lun 2017-01-23 12:10:53 CET; 3min 17s left
Process: 593 ExecStop=/opt/app/bin/app stop (code=exited, status=1/FAILURE)
Process: 487 ExecStart=/opt/app/bin/app start (code=exited, status=0/SUCCESS)
Main PID: 591 (code=exited, status=0/SUCCESS)

If I try to start the app with: /opt/app/bin/app start, it works, so the problem must be systemd script itself.

Created .deb not installable since switch to erl_tar

The latest release of exrm_deb (0.0.5) produces an installable .deb, however it seems to ignore the external_dependencies present in my package configuration. I tested with the latest commit on master and while external_dependencies are properly included in the control file, I am now seeing:
unable to create/opt/ace/erts-7.3/lib/liberts_r.a.dpkg-new' (while processing opt/ace/erts-7.3/lib/liberts_r.a'): No such file or directory when attempting to install the resulting .deb.

My investigations pointed me to this issue on Stack Overflow describing the same issue with a hand-rolled .deb: http://stackoverflow.com/questions/4564434/why-does-my-hand-created-deb-package-fails-at-install-with-unable-to-create-on

The issue, it seems, is that "It turns out that the list of files must also include each directory to create as well." - tracing through recent commits I've narrowed this regression down to commit 7b518e9 (use erl_tar instead of system (g)tar).

erl_tar does not produce a tar that includes the folders, only their included files. This causes the same issue described in the linked stack overflow question and results in a deb that is not installable (at least on Debian Jessie).

Ideas/suggestions

Hey! Just today I came across your deb packager for exrm. It is about a month ago I wanted to start on this myself but didn't find the time. It's really cool that in the mean time someone else started on this. It looks very promising (!) and I used it just now to see how it comes along.

I ran into an error about a missing description in the deb, should the latest github version work? The hex version had a problem with finding the changelog template, but that's already fixed in master. fixed

I also have some ideas or maybe suggestions and would like to know your thoughts about it:

  • configuration of the release in /etc/APP_NAME (together with optional conform file)
  • start script in /usr/bin
  • everything else in /usr/lib/APP/VERSION/FILES_HERE
    This should make it look more like a regular debian package, correct?

I'm also interested in replacing tar with erl_tar or looking at how to implement upgrades. Can I just start working on those topics or is there some way in which you would like contributions? merged

Lintian errors and warnings to clean up

  • W: exrmdeb: missing-depends-line
  • W: exrmdeb: changelog-not-compressed-with-max-compression changelog.gz
  • E: exrmdeb: file-in-etc-not-marked-as-conffile etc/init/exrmdeb.conf
  • E: exrmdeb: file-in-etc-not-marked-as-conffile etc/systemd/system/multi user.target.wants/exrmdeb.service
  • E: exrmdeb: control-file-has-bad-permissions preinst 0644 != 0755
  • E: exrmdeb: control-file-has-bad-owner preinst 1000/1000 != root/root
  • E: exrmdeb: no-copyright-file
  • W: exrmdeb: description-starts-with-leading-spaces
  • E: exrmdeb: unknown-control-interpreter control/preinst #!/usr/bin/env
  • E: exrmdeb: systemd-service-file-outside-lib etc/systemd/system/multi-user.target.wants/exrmdeb.service
  • E: exrmdeb: service-key-has-whitespace etc/systemd/system/multi-user.target.wants/exrmdeb.service at line 9

deb file keeps growing with each new release

I have been using exrm_deb in combination with distillery for some time now.
I noticed that uploading my .deb file seems to take longer and longer every time, but I didn't take the time to look into this. But yesterday I deployed on a new server, and lo and behold, all my versions up to now are in the releases folder. So it seems that my deb file is now approaching 487Mb, because it includes all previous versions.

Is this by design?

-rw-rw-r-- 1 herman herman  57M feb  7 17:11 gwserver-2.0.0.deb
-rw-rw-r-- 1 herman herman  65M feb 28 09:57 gwserver-2.0.1.deb
-rw-rw-r-- 1 herman herman  93M feb 28 16:32 gwserver-2.0.2.deb
-rw-rw-r-- 1 herman herman 121M mrt  1 15:19 gwserver-2.0.3.deb
-rw-rw-r-- 1 herman herman 178M mrt  2 08:14 gwserver-2.0.4.deb
-rw-rw-r-- 1 herman herman 178M mrt  3 07:45 gwserver-2.0.5.deb
-rw-rw-r-- 1 herman herman 207M mrt  8 17:50 gwserver-2.0.6.deb
-rw-rw-r-- 1 herman herman 239M mrt  8 18:52 gwserver-2.0.7.deb
-rw-rw-r-- 1 herman herman 267M mrt  8 19:11 gwserver-2.1.0.deb
-rw-rw-r-- 1 herman herman 296M mrt  8 19:31 gwserver-2.1.1.deb
-rw-rw-r-- 1 herman herman 352M mrt  9 00:16 gwserver-2.1.2.deb
-rw-rw-r-- 1 herman herman 380M mrt  9 12:09 gwserver-2.1.3.deb
-rw-rw-r-- 1 herman herman 380M mrt  9 12:12 gwserver-2.1.4.deb
-rw-rw-r-- 1 herman herman 459M mrt 25 08:50 gwserver-2.1.5.deb
-rw-rw-r-- 1 herman herman 459M mrt 25 10:03 gwserver-2.1.6.deb
-rw-rw-r-- 1 herman herman 487M mrt 29 17:29 gwserver-2.1.7.deb

Generated deb package could not install

I ran into error when I executed sudo dpkg --install to generated deb file which using default template.

environment: same as #6
Here is a log:

exrm_deb_sandbox$ sudo dpkg --install rel/exrmdebsandbox-0.0.1.deb 
dpkg: warning: parsing file '/var/lib/dpkg/tmp.ci/control' near line 8 package 'exrmdebsandbox':
 missing description
dpkg: error processing archive rel/exrmdebsandbox-0.0.1.deb (--install):
 parsing file '/var/lib/dpkg/tmp.ci/control' near line 16:
 several package info entries found, only one allowed
Errors were encountered while processing:
 rel/exrmdebsandbox-0.0.1.deb

Failed to execute after_release hook for Elixir.ReleaseManager.Plugin.Deb

I ran into error when I executed exrm release --deb in my sandbox.
Here is a log:

src$ cat /etc/debian_version
8.3

src$ uname -a
Linux packer-virtualbox-iso-1457657143 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u4 (2016-02-29) x86_64 GNU/Linux

src$ elixir -v
Erlang/OTP 18 [erts-7.2] [source] [64-bit] [async-threads:10] [kernel-poll:false]

Elixir 1.2.3

src$ mix -v
Mix 1.2.3

src$ mix new --sup exrm_deb_sandbox
* creating README.md
* creating .gitignore
* creating mix.exs
* creating config
* creating config/config.exs
* creating lib
* creating lib/exrm_deb_sandbox.ex
* creating test
* creating test/test_helper.exs
* creating test/exrm_deb_sandbox_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd exrm_deb_sandbox
    mix test

Run "mix help" for more commands.

src$ cd exrm_deb_sandbox/

src/exrm_deb_sandbox$  git init && git add . && git commit -am "First Import"

src/exrm_deb_sandbox$ (...edit mix.exs...)

src/exrm_deb_sandbox$ git show
commit b6e5ed019ea196358d37baaa2443e7358ce6a64b
Author: niku <[email protected]>
Date:   Mon Mar 14 20:11:28 2016 +0900

    Use exrm_deb

diff --git a/mix.exs b/mix.exs
index 6ee8bb4..246a39b 100644
--- a/mix.exs
+++ b/mix.exs
@@ -7,14 +7,16 @@ defmodule ExrmDebSandbox.Mixfile do
      elixir: "~> 1.2",
      build_embedded: Mix.env == :prod,
      start_permanent: Mix.env == :prod,
-     deps: deps]
+     deps: deps,
+     description: "A sandbox of exrm_deb",
+     package: package]
   end

   # Configuration for the OTP application
   #
   # Type "mix help compile.app" for more information
   def application do
-    [applications: [:logger],
+    [applications: [:logger, :exrm_deb],
      mod: {ExrmDebSandbox, []}]
   end

@@ -28,6 +30,20 @@ defmodule ExrmDebSandbox.Mixfile do
   #
   # Type "mix help deps" for more examples and options
   defp deps do
-    []
+    [{:exrm_deb, "0.0.4"}]
+  end
+
+  defp package do
+    [
+      external_dependencies: [],
+      license_file: "LICENSE",
+      maintainers: ["niku <[email protected]>"],
+      licenses: ["MIT"],
+      vendor: "niku",
+      links:  %{
+        "GitHub" => "https://github.com/niku/exrm_deb_sandbox",
+        "Homepage" => "https://github.com/niku/exrm_deb_sandbox"
+      }
+    ]
   end
 end

src/exrm_deb_sandbox$ mix do deps.get, release --deb
(...snip...)
==> exrm_deb
Compiled lib/exrm_deb.ex
Compiled lib/exrm_deb/control.ex
Compiled lib/exrm_deb/data.ex
Compiled lib/exrm_deb/deb.ex
Compiled lib/exrm_deb/generators/changelog.ex
Compiled lib/exrm_deb/config.ex
Compiled lib/exrm_deb/generators/control.ex
Compiled lib/exrm_deb/generators/systemd.ex
Compiled lib/exrm_deb/generators/template_finder.ex
Compiled lib/exrm_deb/generators/upstart.ex
Compiled lib/exrm_deb/utils/compression.ex
Compiled lib/exrm_deb/utils/config.ex
Compiled lib/exrm_deb/utils/file.ex
lib/mix/tasks/release.deb.generate_templates.ex:25: warning: variable source is unused
Compiled lib/mix/tasks/release.deb.generate_templates.ex
Generated exrm_deb app
==> exrm_deb_sandbox
Compiled lib/exrm_deb_sandbox.ex
Generated exrm_deb_sandbox app
Consolidated Timex.Date.Convert
Consolidated Vex.Validator.Source
Consolidated Vex.Extract
Consolidated Vex.Blank
Consolidated List.Chars
Consolidated String.Chars
Consolidated Collectable
Consolidated Enumerable
Consolidated IEx.Info
Consolidated Inspect
Building release with MIX_ENV=dev.
==> Failed to execute after_release hook for Elixir.ReleaseManager.Plugin.Deb!
** (FunctionClauseError) no function clause matching in IO.chardata_to_string/1
    (elixir) lib/io.ex:346: IO.chardata_to_string(nil)
    (elixir) lib/path.ex:467: Path.join/2
    (elixir) lib/path.ex:449: Path.join/1
    lib/exrm_deb/generators/changelog.ex:14: ExrmDeb.Generators.Changelog.build/2
    lib/exrm_deb/data.ex:10: ExrmDeb.Data.build/2
    lib/exrm_deb.ex:26: ReleaseManager.Plugin.Deb.start_build/1
    lib/mix/tasks/release.ex:262: anonymous fn/2 in Mix.Tasks.Release.execute_after_hooks/1
    (elixir) lib/enum.ex:1473: Enum."-reduce/3-lists^foldl/2-0-"/3

src/exrm_deb_sandbox$ cat mix.lock 
%{"bbmustache": {:hex, :bbmustache, "1.0.4"},
  "certifi": {:hex, :certifi, "0.4.0"},
  "cf": {:hex, :cf, "0.2.1"},
  "combine": {:hex, :combine, "0.7.0"},
  "earmark": {:hex, :earmark, "0.2.1"},
  "erlware_commons": {:hex, :erlware_commons, "0.19.0"},
  "ex_doc": {:hex, :ex_doc, "0.11.4"},
  "exrm": {:hex, :exrm, "1.0.2"},
  "exrm_deb": {:hex, :exrm_deb, "0.0.4"},
  "getopt": {:hex, :getopt, "0.8.2"},
  "hackney": {:hex, :hackney, "1.5.1"},
  "idna": {:hex, :idna, "1.1.0"},
  "metrics": {:hex, :metrics, "1.0.1"},
  "mimerl": {:hex, :mimerl, "1.0.2"},
  "providers": {:hex, :providers, "1.6.0"},
  "relx": {:hex, :relx, "3.18.0"},
  "ssl_verify_hostname": {:hex, :ssl_verify_hostname, "1.0.6"},
  "timex": {:hex, :timex, "1.0.2"},
  "tzdata": {:hex, :tzdata, "0.5.6"},
  "vex": {:hex, :vex, "0.5.5"}}

dpkg refuses to write to directories that don't exist

root@e8e72219b3cc:/testapp/apps/testapp/rel# dpkg -i testapp-0.0.1.deb 
(Reading database ... 28502 files and directories currently installed.)
Preparing to unpack testapp-0.0.1.deb ...
Unpacking testapp (0.0.1) ...
dpkg: error processing archive testapp-0.0.1.deb (--install):
 unable to create `/usr/share/doc/testapp/changelog.gz.dpkg-new' (while processing `usr/share/doc/testapp/changelog.gz'): No such file or directory
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Errors were encountered while processing:
 testapp-0.0.1.deb
root@e8e72219b3cc:/testapp/apps/testapp/rel# dpkg -c testapp-0.0.1.deb  | head -n1
-rw-r--r-- root/root       242 2016-06-26 03:43 usr/share/doc/testapp/changelog.gz

this seems to happen because /usr/share/doc/testapp was not already created.

Same thing happens with every other directory inside the package.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.