agardiner / csv-diff-report Goto Github PK
View Code? Open in Web Editor NEWGenerate diff reports in HTML and XLSX formats using CSV-Diff
License: BSD 2-Clause "Simplified" License
Generate diff reports in HTML and XLSX formats using CSV-Diff
License: BSD 2-Clause "Simplified" License
is there a way to get ALL fields of the row to show in the Excel file for an update? I really can't figure out the pattern to what fields show up by some of my key fields don't show and it makes it nearly impossible to find the row in the original source (this is because some of the files have dupes and the diff ignores dupe rows so the row field in the output isn't accurate.. at least that's what I'm thinking is causing the row number to be wrong).. at any rate.. having the fields there in the diff would be a huge win for me. I spend a ton of time tracking down the data in the source files..
thanks
Gary
When comparing two UTF-8 CSVs, I use csvdiff one.csv two.csv --encoding utf-8
. The outputted HTML does not render German Umlaute properly: e.g. instead of ü
you get ü
.
Example CSV:
one.csv
:
Aufgabe
Änderung übernehmen
Übergang planen
two.csv
:
Aufgabe
Camp-Farbkombi überlegen
Übergang planen
This exception may occur under, for example, Japanese Windows environment.
In the case, the encoding of the string got via ARGV may be not UTF-8 but Windows 31J (a.k.a. Codepage 932).
# demo1.rb
p ARGV.first.encoding
> ruby demo1.rb foo
#<Encoding:Windows-31J>
Generally, two strings which do not contain any non-ASCII character can be joined, even if one is in UTF-8 and the other is in Windows 31J.
p ["foo".encode("UTF-8"), "bar".encode("CP932")].join
# Not raised.
However, if these strings contain a non-ASCII character, Encoding::CompatibilityError occurs.
p ["あ".encode("UTF-8"), "う".encode("CP932")].join
# => `join': incompatible character encodings: UTF-8 and Windows-31J (En
coding::CompatibilityError)
So, if one of the file paths and a cell value of the CSV have any non-ASCII character, the same exception may occur here.
I think that converting the file paths into UTF-8 may solve the problem.
Hello,
I'm trying to save a diff as a text file, but I'm getting a lot of errors.
I've copied my input in het terminal and the corresponding result:
ruby /users/broex/.rvm/gems/ruby-2.2.1/bin/csvdiff prod.csv new.csv --parent-fields data_uri --child-fields data_uri --format text
Performing file diff:
From File: prod.csv
To File: new.csv
Opening FROM file 'prod.csv'... 43907 lines read
Opening TO file 'new.csv'... 43907 lines read
Found 43907 differences: 43907 Updates
/Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/lib/csv-diff-report/text.rb:31:in text_diff': undefined local variable or method
csv' for #CSVDiff::Report:0x007f8e71a1e350 (NameError)
from /Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/lib/csv-diff-report/text.rb:17:in block (2 levels) in text_output' from /Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/lib/csv-diff-report/text.rb:16:in
each'
from /Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/lib/csv-diff-report/text.rb:16:in block in text_output' from /users/broex/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/csv.rb:1273:in
open'
from /Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/lib/csv-diff-report/text.rb:15:in text_output' from /Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/lib/csv-diff-report/report.rb:121:in
output'
from /Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/lib/csv-diff-report/cli.rb:116:in process' from /Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/lib/csv-diff-report/cli.rb:78:in
run'
from /Users/broex/.rvm/gems/ruby-2.2.1/gems/csv-diff-report-0.3.1/bin/csvdiff:7:in <top (required)>' from /users/broex/.rvm/gems/ruby-2.2.1/bin/csvdiff:23:in
load'
from /users/broex/.rvm/gems/ruby-2.2.1/bin/csvdiff:23:in `
In the README, under the 'Usage' section, it says in order to run the help menu to enter the command 'csv-diff --help'...this produces the error '-bash: csv-diff: command not found' whereas removing the hyphen and running the command 'csvdiff --help' produces the desired result of generating the help menu.
When using keyfields of parent-child fields there is a difference in the time it takes to diff two csv's.
With a small number of records this is not a big issue, however, with 20.000+ record using the keyfields function takes a lot more time.
Is there a reason the keyfield function takes more time, and is there a way to speed up this process?
When settings --include-matched True
, the not changed columns are not added to the HTML file
If I add new = d
below this line:
csv-diff-report/lib/csv-diff-report/html.rb
Line 163 in 65e0a44
Hello, first off great tool, works a lot faster then the previous tool I used.
But I have a question, when comparing two CSV's and exporting it to a HTML file it takes about 1 minute to complete a compare of around 40000 records. However when exporting the same compare to XLSX it takes quite a lot longer when there are a lot off differences (maybe 20 minutes).
Where does this difference come from and is it possible to make the xlsx export faster?
When I try this tool with UTF8 encoded file I get error
"invalid column name as key"
It seems to work fine when the file is not UTF8 encoded.
I'm running into this error when using csvdiff, and wondering if you could help resolve it?
csvdiff a.csv b.csv
'
Performing file diff:
From File: a.csv
To File: b.csv
Opening FROM file 'a.csv'...
An error occurred opening file a.csv: wrong number of arguments (given 3, expected 1..2)
/usr/local/Cellar/ruby/3.0.0/lib/ruby/3.0.0/csv.rb:1412:inopen': wrong number of arguments (given 3, expected 1..2) (ArgumentError) from /usr/local/lib/ruby/gems/3.0.0/gems/csv-diff-0.6.1/lib/csv-diff/csv_source.rb:60:in
initialize'
from /usr/local/lib/ruby/gems/3.0.0/gems/csv-diff-report-0.3.6/lib/csv-diff-report/report.rb:262:innew' from /usr/local/lib/ruby/gems/3.0.0/gems/csv-diff-report-0.3.6/lib/csv-diff-report/report.rb:262:in
open_source'
from /usr/local/lib/ruby/gems/3.0.0/gems/csv-diff-report-0.3.6/lib/csv-diff-report/report.rb:219:indiff_file' from /usr/local/lib/ruby/gems/3.0.0/gems/csv-diff-report-0.3.6/lib/csv-diff-report/report.rb:87:in
diff'
from /usr/local/lib/ruby/gems/3.0.0/gems/csv-diff-report-0.3.6/lib/csv-diff-report/cli.rb:131:inprocess' from /usr/local/lib/ruby/gems/3.0.0/gems/csv-diff-report-0.3.6/lib/csv-diff-report/cli.rb:101:in
run'
from /usr/local/lib/ruby/gems/3.0.0/gems/csv-diff-report-0.3.6/bin/csvdiff:7:in<top (required)>' from /usr/local/bin/csvdiff:23:in
load'
from /usr/local/bin/csvdiff:23:in `
Hello.
I'm trying to generate csv diff report as html.
But, when csv file path is absolute, it does not work.
it just works by specifying the csv file as a relative path.
$ csvdiff test.csv test_old.csv
/Library/Ruby/Gems/2.0.0/gems/arg-parser-0.3.0/lib/arg-parser/dsl.rb:70: warning: instance variable @arg_opts not initialized
Performing file diff:
From File: test.csv
To File: test_old.csv
Opening FROM file 'test.csv'... 3 lines read
Opening TO file 'test_old.csv'... 3 lines read
Found 1 differences: 1 Updates
Diff report saved to './Diff_test_to_test_old.html'
$ csvdiff /Users/maeken/csv/test.csv /Users/maeken/csv/test_old.csv
/Library/Ruby/Gems/2.0.0/gems/arg-parser-0.3.0/lib/arg-parser/dsl.rb:70: warning: instance variable @arg_opts not initialized
USAGE: ruby /usr/local/bin/csvdiff FROM TO [PATTERN] [OPTIONS]
Specify the /? or --help option for more detailed help
It can be useful to show only the fields that have been changed when comparing two CSVs.
Currently the rows are filtered this way, but the columns are not.
Apologies in advance if the tool already has options to support this but I am not seeing it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.