xolox / python-deb-pkg-tools Goto Github PK

That PR that I submitted in the past was partly to try and eliminate the SUDO prompts. However, in the end, I was only able to reduce them. I'll still be required to one, every time I build.

I need this to be completely automated. A newer application will not be attached to a terminal.

What is it that is requiring it?

Hello. I am receiving the following error when trying to set a field in the control file:

deb-pkg-tools -p foo.deb -s "Version: 0.1"
2020-03-16 11:43:43 rothbard deb_pkg_tools.cli[29169] ERROR An error occurred! Aborting..
Traceback (most recent call last):
  File "/venv/lib/python3.6/site-packages/deb_pkg_tools/cli.py", line 215, in main
    action()
  File "/venv/lib/python3.6/site-packages/deb_pkg_tools/control.py", line 158, in patch_control_file
    defaults = Deb822(handle)
  File "/venv/lib/python3.6/site-packages/debian/deb822.py", line 633, in __init__
    self._internal_parser(iterable, fields, strict)
  File "/venv/lib/python3.6/site-packages/debian/deb822.py", line 784, in _internal_parser
    self._skip_useless_lines(sequence), strict):
  File "/venv/lib/python3.6/site-packages/debian/deb822.py", line 1093, in gpg_stripped_paragraph
    return cls.split_gpg_and_payload(sequence, strict)[1]
  File "/venv/lib/python3.6/site-packages/debian/deb822.py", line 1038, in split_gpg_and_payload
    for line in sequence:
  File "/venv/lib/python3.6/site-packages/debian/deb822.py", line 740, in _skip_useless_lines
    for line in sequence:
  File "/venv/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 133: invalid start byte

The above is on Python 3.6.9. The following is on Python 2.7.10:

2020-03-16 11:39:27 rothbard deb_pkg_tools.cli[27113] ERROR An error occurred! Aborting..
Traceback (most recent call last):
  File "/venv/local/lib/python2.7/site-packages/deb_pkg_tools/cli.py", line 235, in main
    action()
  File "/venv/local/lib/python2.7/site-packages/deb_pkg_tools/control.py", line 175, in patch_control_file
    defaults = Deb822(handle)
  File "/venv/local/lib/python2.7/site-packages/debian/deb822.py", line 633, in __init__
    self._internal_parser(iterable, fields, strict)
  File "/venv/local/lib/python2.7/site-packages/debian/deb822.py", line 785, in _internal_parser
    line = self.decoder.decode(linebytes)
  File "/venv/local/lib/python2.7/site-packages/debian/deb822.py", line 2346, in decode
    return value.decode(result['encoding'])
TypeError: decode() argument 1 must be string, not None

I've tried deb-pkg-tools 6.1 and 7.0.

Hi. First of all, thank you for an awesome lib.

But it has some drawbacks. I mean its mandatory dependencies.

1. python-memcached - do I really need memcached to work with deb packages using this lib? Can this be made optional?
2. humanfriendly and coloredlogs - could these be made optional?
3. sh is more pythonic alternative to executor.

As an interested may-be user, I would like the README to tell (me),

• what this tool really does (I assume it packages python packages as DEBs?), and
• how I would use it.

I suggest to expand the README with a short description of what the tool does, and to provide a bare-minimum sample for how to use it

When I run a scan_packages() or an update_repository() I get a Packages and a Packages.gz file that contain a "Filename" field for each package in the repo; however, the field only contains the name of the file and not the path to the file. If I manually run a dpkg-scanpackages -m ${dir} | gzip -9c > ${dir}/Packages.gz on the repo the Packages.gz that it creates has a relative path to the .deb file in the "Filename" field. Is there support already for creating Packages files that contain relative paths to the archives in their repos?

hi @xolox , thank you for building the package and distribution it under the MIT license. We are evaluating it, but found, that you are using python-debian, which is distibuted under GPL2 or later. Did you evaluate, how it works together? Thank you in advance

Several objects and the objects in deps.py in particular are plain (small) objects and they do not have the equivalent of a namedtuple._asdict() or an attrs .asdict() method or similar to return plain collections and/or built-in types for easier integration such as for instance serializing to JSON. This would be a great addition. In contrast most of the deb822 objects have a mapping-like behavior (though that does not make them JSON-serializable either by default). And since they are collections.Mapping they are easier to convert at once to actual built-in types.

I encountered this error:

expression = 'libtdb-dev (>> 1.1) [!hurd-any]'
parse_relationship(expression)
 ValueError: Corrupt package relationship expression: Splitting name from relationship resulted in more than two tokens! (expression: libtdb-dev (>> 1.1) [!hurd-any], tokens: [u'libtdb-dev', u'>> 1.1', u'[!hurd-any]'])

as original found in this Ubuntu Xenial package:

Package: libcanberra
Binary: libcanberra0, libcanberra0-dbg, libcanberra-dev, libcanberra-pulse, libcanberra-pulse-dbg, libcanberra-gstreamer, libcanberra-gstreamer-dbg, libcanberra-gtk0, libcanberra-gtk0-dbg, libcanberra-gtk-common-dev, libcanberra-gtk-dev, libcanberra-gtk-module, libcanberra-gtk-module-dbg, libcanberra-gtk3-0, libcanberra-gtk3-0-dbg, libcanberra-gtk3-dev, libcanberra-gtk3-module, gnome-session-canberra, libcanberra-gtk3-module-dbg, libcanberra-doc
Version: 0.30-2.1ubuntu1
Priority: optional
Section: libs
Maintainer: Ubuntu Developers <[email protected]>
Original-Maintainer: Debian GNOME Maintainers <[email protected]>
Build-Depends: debhelper (>= 8.1.3~), cdbs (>= 0.4.93~), dh-autoreconf, gnome-pkg-tools, m4, libltdl-dev | libltdl7-dev (>= 2.2.6), dpkg-dev (>= 1.14.13), libasound2-dev [linux-any], libvorbis-dev, libglib2.0-dev (>= 2.32), libgtk-3-dev (>= 3.0.0), libgtk2.0-dev (>= 2.22), libx11-dev, libtdb-dev (>> 1.1) [!hurd-any], gtk-doc-tools, libpulse-dev (>= 0.9.11), libgstreamer1.0-dev
Architecture: any all
Standards-Version: 3.9.4
Format: 3.0 (quilt)
Directory: pool/main/libc/libcanberra

currently everything in /etc/ directory gets treated as conffiles. There are siuations where you dont want this to happen. You should update the function "update_conffiles(directory) in the package.py to accept a boolean that defines if /etc/ is treated as conffiles or not. You should also make it default to yes for legacy support.

Hi @xolox,
I have a very specific need (TBH I expected this to be quite common but turns out it's not).
I have a debian repository hosted on Amazon S3, and I would like the ability to upload new packages (more specifically, new versions of a package) without re-creating all the repo metadata from .deb files all over again.
So the idea would be to either extend repo.update_repository() with a flag to read the existing Packages file beforehand, or somehow add the possibility to merge two existing repositories.

Would you accept a PR for this kind of feature?

python 2 compatibility is preventing from using type hints and pathlib

Hello! First of all, thank you for providing deb_pkg_tools, it's a library that we find immensely useful. :-)

This is just a quick query with regards to support for the "Recommends" (and other) control field when parsing dependencies, as it appears that it currently isn't, based on the DEPENDS_LIKE_FIELDS list:

DEPENDS_LIKE_FIELDS = (
    'Breaks',
    'Build-Depends',
    'Conflicts',
    'Depends',
    'Pre-Depends',
    'Provides',
    'Replaces',
    'Suggests',
)

This caused a problem for us when expecting the Recommends field to be a parsed relationship set, and was unexpectedly a string instead (unparsed). The Debian policy documentation mentions control fields in various sections that probably should belong in this list, such as:

• Section 7.1:
  • Recommends
• Section 7.2:
  • Enhances
• Section 7.7:
  • Build-Depends
  • Build-Depends-Indep
  • Build-Depends-Arch
  • Build-Conflicts
  • Build-Conflicts-Indep
  • Build-Conflicts-Arch
• Section 7.8:
  • Built-Using

An example of a debian package / control file that has Recommends is the virtualbox-5.1 deb file:

Package: virtualbox-5.1
Version: 5.1.14-112924Ubuntutrusty
Architecture: i386
Maintainer: Oracle Corporation [email protected]
Installed-Size: 168128
Pre-Depends: debconf (>= 1.1) | debconf-2.0
Depends: libc6 (>= 2.15), libcurl3 (>= 7.16.2), libdevmapper1.02.1 (>= 2:1.02.20), libfontconfig1 (>= 2.9.0), libfreetype6 (>= 2.2.1), libgcc1 (>= 1:4.1.1), libgl1-mesa-glx | libgl1, libglib2.0-0 (>= 2.12.0), libice6 (>= 1:1.0.0), libpng12-0 (>= 1.2.13-4), libpython2.7 (>= 2.7), libsdl1.2debian (>= 1.2.11), libsm6, libssl1.0.0 (>= 1.0.0), libstdc++6 (>= 4.6), libvpx1 (>= 1.0.0), libx11-6, libx11-xcb1, libxcb1 (>= 1.8), libxcursor1 (>> 1.1.2), libxext6, libxinerama1, libxml2 (>= 2.7.4), libxmu6, libxrender1, libxt6, zlib1g (>= 1:1.1.4), psmisc, adduser
Recommends: libasound2, libpulse0, libsdl-ttf2.0-0, kmod | kldutils | module-init-tools, linux-headers-generic | linux-headers-generic-pae | linux-headers-686-pae | linux-headers-amd64 | linux-headers-2.6-686 | linux-headers-2.6-amd64 | linux-headers, linux-image, gcc, make | build-essential | dpkg-dev, binutils, pdf-viewer, libgl1
Conflicts: virtualbox, virtualbox-guest-additions-iso, virtualbox-ose
Replaces: virtualbox
Provides: virtualbox
Section: contrib/misc
Priority: optional
Description: Oracle VM VirtualBox
VirtualBox is a powerful PC virtualization solution allowing you to run a
wide range of PC operating systems on your Linux system. This includes
Windows, Linux, FreeBSD, DOS, OpenBSD and others. VirtualBox comes with a broad
feature set and excellent performance, making it the premier virtualization
software solution on the market.
Python-Version: 2.7

We'd be fairly happy to contribute a PR if you agree that these should be parsed. Thanks!

https://dev.gnupg.org/source/gpgme/browse/master/lang/python/