lvc / pkgdiff Goto Github PK

I am using pkgdiff to compare two zip files containing multiple jars. One of the jar is build (dedup-0.001-SNAPSHOT.jar) and rest are downloaded from maven repo and then all are added in zip as part of maven build. pkgdiff is reporting change when there is nothing changed in dedup-0.001-SNAPSHOT.jar

dmanna-m01:es-plugins dmanna$ pkgdiff  ~/Desktop/hbase-co/1/dedup-0.001-SNAPSHOT.zip ~/Desktop/hbase-co/2/dedup-0.001-SNAPSHOT.zip
reading packages ...
comparing packages ...
creating report ...
result: CHANGED (0.000002%)
report: pkgdiff_reports/dedup/0.001-SNAPSHOT_to_0.001-SNAPSHOT/changes_report.html

The detailed report is empty.

dmanna-m01:details dmanna$ pwd
/Users/dmanna/code/github/vnera/main/es-plugins/pkgdiff_reports/dedup/0.001-SNAPSHOT_to_0.001-SNAPSHOT/details
dmanna-m01:details dmanna$ ls
dmanna-m01:details dmanna$

The surprising part is if I unzip and then compare the jar which is reported as different then pkgdiff reports that the there is no change.

dmanna-m01:hbase-co dmanna$ pkgdiff 1/dedup-0.001-SNAPSHOT/dedup-0.001-SNAPSHOT.jar 2/dedup-0.001-SNAPSHOT/dedup-0.001-SNAPSHOT.jar
reading packages ...
comparing packages ...
creating report ...
result: UNCHANGED
report: pkgdiff_reports/dedup/0.001-SNAPSHOT_to_0.001-SNAPSHOT/changes_report.html
dmanna-m01:hbase-co dmanna$

Is this some issue on pkgdiff ? How can I know what exactly is differing in dedup-0.001-SNAPSHOT.jar when it is part of zip file?

Hi Andrey!

I currently packaged rfcdiff for Debian and was wondering to which extend your custom rfcdiff script is necessary for pkgdiff to work properly?

Could your changes to it, be merged into upstream rfcdiff?

Greetings
Peter

Hello,

I want to express my appreciation for the impressive tool you've provided.
While utilizing it to compare two folders, I've observed that the report accurately captures changes in header or code files, as expected.
However, I'm reaching out for your assistance in reducing some unnecessary noise in the report.

As an example:
Copyright (c) 1999-2021 or its subsidiaries. All Rights Reserved.
is marked as changed to:
Copyright (c) 1999-2023 or its subsidiaries. All Rights Reserved.

The sole modification in header or C code files is the alteration in the copyright year, and I would like to exclude such instances from being listed in my changes_report.html. Is there an option available to ignore specific strings using regex? If not, could you please consider my enhancement request?

When one of the packages to compare is a symlink we get this:

ERROR: unknown format "rpm"

At the end of the changes_report.html file there's a link supposed to point to the homepage of the project (http://pkgdiff.github.com/pkgdiff/), however when clicking it a 404 page is displayed (the correct URL being actually https://github.com/lvc/pkgdiff).

Firstly, many thanks for your developing so cool tool to compare linux archives. It works perfectly for me while I just tried it.
And, I am also seeking a way to get the reclusive visual view report for all inner archives, for example,

• I have two version archives, such as a-1.0.war, and a-1.1.war
• pkgdiff detects one jar file change, yes, this jar can be reported, actually, at this time, I want to also understand what the exact change for this jar file. I expect there would be a "diff" link which is able to redirect me to another detailed report for this different inner archive.

Best regards - Hut

This is more of notes, not an issue. can be added to the documentation. I had a requirement that needed run pkgdiff in jenkins step, and put together a Dockerfile for it. The community can use it too.

FROM fedora:36 as builder

ARG SYS_ROOT=/mnt/sys-root

RUN mkdir $SYS_ROOT; \
    dnf install \
    --installroot $SYS_ROOT \
    --releasever 36 \
    --nodocs -y \
    diffutils \
    gawk \
    perl-File-LibMagic \
    pkgdiff \
    wdiff \
    dpkg \
    binutils \
    tar \
    xz \
    gzip \
    which \
    java-17-openjdk-headless \
    abi-compliance-checker \
    abi-dumper; \
    dnf --installroot $SYS_ROOT clean all; 

# Some standard cleanup to reduce the size of image    
RUN rm -rf $SYS_ROOT/var/log/dnf* $SYS_ROOT/var/log/yum.* $SYS_ROOT/var/cache/dnf $SYS_ROOT/var/lib/dnf/repos; \
    rm -rf $SYS_ROOT/var/lib/dnf/history* $SYS_ROOT/var/log/hawkey.log $SYS_ROOT/boot $SYS_ROOT/dev/null $SYS_ROOT/run/*; \
    rm -f $SYS_ROOT/etc/machine-id; \
    touch $SYS_ROOT/etc/machine-id; \
    mkdir -p $SYS_ROOT/work

# Second stage build to reduce size
FROM scratch
COPY --from=builder /mnt/sys-root/ /

WORKDIR /work

ENTRYPOINT ["pkgdiff"]

CMD ["--help"]

Published image available at srbala/build-tools:pkgdiff from dockerhub.

docker pull srbala/build-tools:pkgdiff
docker run --privileged --rm -ti -v $PWD:/work srbala/build-tools:pkgdiff fileOld.tar.xz fileNew.tar.xz

Add an option to hide unchanged files in the final report: -hide-unchanged

html is nice et al, but what I often want is just to get a quick overview on how two things differ. Now for little comparisons it is a multistage procedure requiring running pkgdiff, opening the browser, closing the browser, removing the file. If there was a mode just to print information in the output of running of pkgdiff, it would be great.

E.g. debdiff in Debian world lists files which are present in one package but not in the other, and vise versa. The same could be done by pkgdiff.

I suggest to add changes rate in percents to the report summary (i.e. "75% Changed").

This is my understanding that pkgdiff doesn't support the gradle so that's why it is taking this under python.The next thing which I noticed is it is showing some gradle file under Text File header.

Hi,

The pkgdiff was working fine a while before but recently I see the report alway says UNCHANGED. I am running it on a Amazon EC2

Regards

Hello,

Your program is very useful is almost all cases but in some rare cases, where the package are built unconventionally, the diff report is not usable.

Let me explain:

In my company, we are building our software/web services "unconventionally", and we are installing each version in a separate folder:

/opt/my_company/my_product/1.0.0/all_my_files...
/opt/my_company/my_product/1.1.0/all_my_files...
/opt/my_company/my_product/1.2.0/all_my_files...
/opt/my_company/my_product/1.3.0/all_my_files...

Between version 1.2.0 and version 1.3.0, it may have few changes.

When I'm running pkgdiff on both RPM (we are using RPM-based Linux servers), ALL files are reported as moved (yes, it's true) with a Delta = 0%

My RFE is to hide these results when we are using the -hide-unchanged option to keep only files with real delta, even if they are moved.

To reach this point, I've changed this part of code:

https://github.com/lvc/pkgdiff/blob/master/pkgdiff.pl#L2171

2169    if($HideUnchanged)
2170    {
2171        if($Info{"Status"} eq "unchanged")
2172        {  # do not show unchanged files
2173            next;
2174        }
2175    }

to add:

2169    if($HideUnchanged)
2170    {
2171        if($Info{"Status"} eq "unchanged" or show_number($Info{"Rate"}*100) eq 0)
2172        {  # do not show unchanged files
2173            next;
2174        }
2175    }

This let me know which files are modified between two packages ignoring moved file without any modification.

Thank you!

It is needed, when the files are either file descriptors or temporary files

It would be good to provide a report not only in HTML style.
Another output like standart text would be welcome.
Standart text can be used for parsing by another programs.

Current status is not so friendly for parsing.

First I want to say that I am a very happy user of "pkgdiff". On a server I provide are currently roughly 7.700 software package comparisons available with nearly 1.900.000 diffs reports for package member files, all generated with a slightly adapted version of "pkgdiff" (though based on version 1.6.2 with some small corrections taken from 1.6.4)!

Now I think about using the current 1.7.2 release (it's not easy for me since I am a Perl layman) and found that "pkgdiff" is stlll based on the "rfcdiff" bash script version 1.41. Meanwile exists "rfcdiff" 1.45 so my issue is to suggest the usage of that newer script (if meaningful) for an optional new "pkgdiff" release since some additions are there done (viewable for e.g. via https://fossies.org/diffs/rfcdiff/1.41_vs_1.45/).

It would be nice to use the file command to determine the type of the pkg being compared rather than extension.

The use case I hit is

1. wget
2. wget <url to same package, different url>

Both had the same name but different sizes, but the second one now ends in .1 pkgdiff complains that it doesn't understand .1 format. When file reports Zip archive data, at least v2.0 to extract

hi,
I am using 1.8 version and I am trying to compare jar files with the following options
-check-byte-code -full-method-diffs

The main report shows the % change in the changed Java Classes. However, if I click on the link "diff" , it redirects me to a blank page.
The HTML files are created for each of the Changed classes but they are empty.

Can you please advise what could be the reason?

BTW, I get a following warning when I run the tool
WARNING: perl-File-LibMagic is not installed
I think, this should not be the reason for the empty reports.

Please help.

code version: 95c88b3

OS version:
macOS high Sierra, 10.13.6

Perl version:

perl -v

This is perl 5, version 18, subversion 2 (v5.18.2) built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)

Command
sudo make install prefix=/usr

error log

perl Makefile.pl -install -prefix "/usr"
INSTALL PREFIX: /usr
-- Installing /usr/bin/pkgdiff
can't open file '/usr/bin/pkgdiff': Operation not permitted
make: *** [install] Error 1

A fossies.org user points me to the fact that diifs for plist-files ("Property list"; XML-like) are detected but are not shown as a visual diffs file. Adding in modules/FileType.xml within the according entry

  <format>
      Text
  </format>

seems to help.

Some links in the "Examples" section of the file doc/index.html are now stale and must be updated probably by

http://lvc.github.com/pkgdiff/ -> https://lvc.github.io/pkgdiff/

I try to fix my problem but still w/o success.
What I would like to achive?

I would like to add to files.xml percentage values how much the file was changed.
What do you think about it? In HTML report the percentage are mentioned.

files.xml is computer readable and therefore it would be good to have those information in the files too.

I suggest to add color legend at the top of the changes report.

Can pkgdiff be used for comparing jars? I am asking this as https://lvc.github.io/pkgdiff/ does not mention anything about jars.

If yes how does it compare jars? I am asking this as to test pkgdiff I created a dummy jar with some java files and then added some comments in one of the java file and then created the jar again. pkgdiff correctly recognized this and reported the jars as unchanged. What all things are checked to determine the jars are different ?

I have two packages:

$ for f in */pdns-backend-sqlite3_* ; do echo == $f ; dpkg-deb -c $f | grep /schema.sqlite3.sql ; done
== pdns-4.4.0.16.relauth44x.gc8b6f52df-debian-buster/pdns-backend-sqlite3_4.4.0+relauth44x.16.gc8b6f52df-1pdns.buster_amd64.deb
-rw-r--r-- root/root      3043 2021-01-12 16:29 ./usr/share/doc/pdns-backend-sqlite3/schema.sqlite3.sql
== pdns-4.4.0.21.debianupdate44.g1613283c0-debian-buster/pdns-backend-sqlite3_4.4.0+debianupdate44.21.g1613283c0-1pdns.buster_amd64.deb
-rw-r--r-- root/root      3043 2021-01-29 08:34 ./usr/share/pdns-backend-sqlite3/schema/schema.sqlite3.sql
lrwxrwxrwx root/root         0 2021-01-29 08:34 ./usr/share/doc/pdns-backend-sqlite3/schema.sqlite3.sql -> ../../pdns-backend-sqlite3/schema/schema.sqlite3.sql

The file was moved, and its old location was symlinked to the new one. However, pkgdiff reports the new location as 'added' and the old location as 'unchanged' - presumably because the content is still the same. I'd rather have seen some note that there was in fact a change, so I would know to have a look.

(By the way, thank you for pkgdiff, it's such an awesome tool!)

The tool is not working properly under FreeBSD. The issue is that basic rfcdiff tool trying to create temporary directories using "mktemp -d" command which is not working under FreeBSD. The following command should used instead:

mktemp -d -t rfcdiff.XXXXXXXX

Version 1.8 is mentioned in the changelog and in the README, however there's no tag and it's not mentioned in the Downloads. Please either add a tag and download for 1.8 or do not mention non-existing version, as it creates confusion for users: https://repology.org/metapackage/pkgdiff/versions

I suggest to add --size-limit option to skip checking of some large files.

The --details option enables detailed checks only for shared libraries and header files for now. Additional detailed checks should be added for other file formats too: python/perl/ruby modules, DBUS interfaces, QML interfaces, CLI and others.

Hello

When comparing two packages, I request for a feature which also checks for any change in file permission (like 750 to 755 or no setuid to setuid u+s) or change in file owner or group i.e. httpd to root.

This will help detect any security lapse or accidental change in such ACLs.

So please add this feature.

Thank you

The tool should print estimated change rate to the console.

hi. I have 2 java file,
in first file have
JCO.Repository("Hell1", connClient);

in second file
JCO.Repository("Hell2", connClient);

but where I was compared pkgdiff don't show change.

The current master produces lots of sh syntax errors. I think it's this change:

-    system($Cmd." >\"".$SPath."\" 2>$TMP_DIR/null");
+    
+    my $TmpFile = $TMP_DIR."/null";
+    qx/$Cmd." >\"".$SPath."\" 2>$TmpFile/;

Since qx/…/ is a command literal, using . to concatenate inside it is nonsense. I think it should be:

qx/$Cmd >"$SPath" 2>"$TmpFile"/;

Compiled and installed on RHEL 8.4.
When trying to run pkgdiff -h I get

ERROR: can't find modules

There is not enough context to know what the problem is. Any recommendations to troubleshoot?

Add sorting by status, delta and file name to the report tables by clicking on the table headers.

The standard arguments of the tool are:

pkgdiff -old OLD.pkg -new NEW.pkg [options]

The tool should accept also:

pkgdiff OLD.pkg NEW.pkg [options]

👀 Some source code analysis tools can help to find opportunities for improving software components.
💭 I propose to increase the usage of combined operators accordingly.

diff --git a/pkgdiff.pl b/pkgdiff.pl
index 82ae2f5..06bf614 100644
--- a/pkgdiff.pl
+++ b/pkgdiff.pl
@@ -2038,7 +2038,7 @@ sub showOp($)
         $Op="";
     }
     if($Op) {
-        $Op = $Op." ";
+        $Op .= " ";
     }
     return $Op;
 }

Arch Linux recently added support for using zstd to compress packages.

I am running pkgdiff on an ubuntu docker container:
root@d73ffc6c819a:/# pkgdiff spark-core_v1.jar spark-core_v2.jar
reading packages ...
comparing packages ...
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found
sh: 1: file: not found

creating report ...
result: CHANGED (0.4%)
report: pkgdiff_reports/spark-core/v1_to_v2/changes_report.html

The -open option should be added to open changes report in the system default browser.

I have a user trying pkgdiff 1.7.0 but abi-compliance-checker (1.99.13) seems to not like the options being passed to it.

https://bugzilla.redhat.com/show_bug.cgi?id=1277606

I suggest to add <skip_files> section of XML-descriptor to be able to skip checking of some files in the package.

I'm comparing jar files, and with -quick I get a difference in the META.INF, that I don´t get if I run it without -quick

If /tmp system directory is a symlink, then the tool prints additional "//home/.../tmp/content1/" prefixes to files in the report.

Hi,

In order to get changelog diff & pre/postinstall scripts diff, i suggest the following modification:

diff --git a/pkgdiff.pl b/pkgdiff.pl
index e6024b4..89ffc1a 100644
--- a/pkgdiff.pl
+++ b/pkgdiff.pl
@@ -3187,7 +3187,7 @@ sub readPackage($$)
                 $TotalDeps{$Kind." ".$N} = 1;
             }
         }
-        $PackageInfo{$Attr{"Name"}}{"V$Version"} = queryRPM($Path, "--info");
+        $PackageInfo{$Attr{"Name"}}{"V$Version"} = queryRPM($Path, "--info --changelog --scripts");
         $Group{"Format"}{$Format} = 1;
     }
     elsif($Format eq "ARCHIVE")

Best regards,
David

Add support for more archive formats: TAR.LZMA, TAR.LZ, WAR, EAR, TBZ, ...

I have compared two war files. It was not working and then I have renamed to zip and it works. But it is not comparing the class file. I made the change but it says unchanged.