evan / ccsv Goto Github PK

View Code? Open in Web Editor NEW

64.0 64.0 52.0 2.55 MB

A pure-C CSV parser for Ruby

Home Page: http://blog.evanweaver.com/files/doc/fauna/ccsv/

License: Academic Free License v3.0

Ruby 50.96% Shell 0.56% C 48.48%

ccsv's People

Contributors

Stargazers

Watchers

Forkers

dougcole rianawi pronix crossman zhum superp wwood abates jonmidhir inf1k venki911 clod81 lifemoveson cyborgjourney

ccsv's Issues

Double free security issue

When attempting to parse a file with the following bytes, a double free occurs.
BD 22 5C 0A 0A

Tested on ubuntu, with ruby 2.4.2.

gdb debug:
gdb --batch -q --ex=r --ex 'back' --ex 'disass $pc, $pc+16' --ex 'info reg' --ex 'quit' --args /usr/local/bin/ruby /data/ccsv/ext/test.rb file_containing_crash_bytes 0</dev/null

gdb output:

*** Error in `/usr/local/bin/ruby': double free or corruption (fasttop): 0x0000000002116b80 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fb6195c57e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fb6195ce37a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fb6195d253c]
/data/ccsv/ext/ccsv.so(+0x2f0d)[0x7fb618d33f0d]
/usr/local/lib/libruby.so.2.4(+0x596a98)[0x7fb61a810a98]
/usr/local/lib/libruby.so.2.4(+0x58869b)[0x7fb61a80269b]
/usr/local/lib/libruby.so.2.4(+0x583741)[0x7fb61a7fd741]
/usr/local/lib/libruby.so.2.4(+0x583510)[0x7fb61a7fd510]
/usr/local/lib/libruby.so.2.4(+0x58319b)[0x7fb61a7fd19b]
/usr/local/lib/libruby.so.2.4(+0x5411bf)[0x7fb61a7bb1bf]
/usr/local/lib/libruby.so.2.4(+0x578756)[0x7fb61a7f2756]
/usr/local/lib/libruby.so.2.4(rb_iseq_eval_main+0x838)[0x7fb61a7f5218]
/usr/local/lib/libruby.so.2.4(ruby_run_node+0x339)[0x7fb61a3ff479]
/usr/local/bin/ruby[0x4011d7]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fb61956e830]
/usr/local/bin/ruby(_start+0x29)[0x401059]
======= Memory map: ========
00400000-00402000 r-xp 00000000 00:32 300                                /usr/local/bin/ruby
00601000-00602000 r--p 00001000 00:32 300                                /usr/local/bin/ruby
00602000-00603000 rw-p 00002000 00:32 300                                /usr/local/bin/ruby
00603000-00613000 rw-p 00000000 00:00 0 
01dae000-02123000 rw-p 00000000 00:00 0                                  [heap]
7fb614000000-7fb614021000 rw-p 00000000 00:00 0 
7fb614021000-7fb618000000 ---p 00000000 00:00 0 
7fb618b1b000-7fb618b31000 r-xp 00000000 00:32 1495                       /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb618b31000-7fb618d30000 ---p 00016000 00:32 1495                       /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb618d30000-7fb618d31000 rw-p 00015000 00:32 1495                       /lib/x86_64-linux-gnu/libgcc_s.so.1
7fb618d31000-7fb618d35000 r-xp 00000000 00:32 1326                       /data/ccsv/ext/ccsv.so
7fb618d35000-7fb618f34000 ---p 00004000 00:32 1326                       /data/ccsv/ext/ccsv.so
7fb618f34000-7fb618f35000 r--p 00003000 00:32 1326                       /data/ccsv/ext/ccsv.so
7fb618f35000-7fb618f36000 rw-p 00004000 00:32 1326                       /data/ccsv/ext/ccsv.so
7fb618f36000-7fb618f46000 r-xp 00000000 00:32 335                        /usr/local/lib/ruby/2.4.0/x86_64-linux/stringio.so
7fb618f46000-7fb619146000 ---p 00010000 00:32 335                        /usr/local/lib/ruby/2.4.0/x86_64-linux/stringio.so
7fb619146000-7fb619147000 r--p 00010000 00:32 335                        /usr/local/lib/ruby/2.4.0/x86_64-linux/stringio.so
7fb619147000-7fb619148000 rw-p 00011000 00:32 335                        /usr/local/lib/ruby/2.4.0/x86_64-linux/stringio.so
7fb619148000-7fb61914a000 r-xp 00000000 00:32 318                        /usr/local/lib/ruby/2.4.0/x86_64-linux/enc/trans/transdb.so
7fb61914a000-7fb619349000 ---p 00002000 00:32 318                        /usr/local/lib/ruby/2.4.0/x86_64-linux/enc/trans/transdb.so
7fb619349000-7fb61934a000 r--p 00001000 00:32 318                        /usr/local/lib/ruby/2.4.0/x86_64-linux/enc/trans/transdb.so
7fb61934a000-7fb61934b000 rw-p 00002000 00:32 318                        /usr/local/lib/ruby/2.4.0/x86_64-linux/enc/trans/transdb.so
7fb61934b000-7fb61934d000 r-xp 00000000 00:32 316                        /usr/local/lib/ruby/2.4.0/x86_64-linux/enc/encdb.so
7fb61934d000-7fb61954c000 ---p 00002000 00:32 316                        /usr/local/lib/ruby/2.4.0/x86_64-linux/enc/encdb.so
7fb61954c000-7fb61954d000 r--p 00001000 00:32 316                        /usr/local/lib/ruby/2.4.0/x86_64-linux/enc/encdb.so
7fb61954d000-7fb61954e000 rw-p 00002000 00:32 316                        /usr/local/lib/ruby/2.4.0/x86_64-linux/enc/encdb.so
7fb61954e000-7fb61970e000 r-xp 00000000 00:32 43                         /lib/x86_64-linux-gnu/libc-2.23.so
7fb61970e000-7fb61990e000 ---p 001c0000 00:32 43                         /lib/x86_64-linux-gnu/libc-2.23.so
7fb61990e000-7fb619912000 r--p 001c0000 00:32 43                         /lib/x86_64-linux-gnu/libc-2.23.so
7fb619912000-7fb619914000 rw-p 001c4000 00:32 43                         /lib/x86_64-linux-gnu/libc-2.23.so
7fb619914000-7fb619918000 rw-p 00000000 00:00 0 
7fb619918000-7fb619a20000 r-xp 00000000 00:32 139                        /lib/x86_64-linux-gnu/libm-2.23.so
7fb619a20000-7fb619c1f000 ---p 00108000 00:32 139                        /lib/x86_64-linux-gnu/libm-2.23.so
7fb619c1f000-7fb619c20000 r--p 00107000 00:32 139                        /lib/x86_64-linux-gnu/libm-2.23.so
7fb619c20000-7fb619c21000 rw-p 00108000 00:32 139                        /lib/x86_64-linux-gnu/libm-2.23.so
7fb619c21000-7fb619c2a000 r-xp 00000000 00:32 305                        /lib/x86_64-linux-gnu/libcrypt-2.23.so
7fb619c2a000-7fb619e29000 ---p 00009000 00:32 305                        /lib/x86_64-linux-gnu/libcrypt-2.23.so
7fb619e29000-7fb619e2a000 r--p 00008000 00:32 305                        /lib/x86_64-linux-gnu/libcrypt-2.23.so
7fb619e2a000-7fb619e2b000 rw-p 00009000 00:32 305                        /lib/x86_64-linux-gnu/libcrypt-2.23.so
7fb619e2b000-7fb619e59000 rw-p 00000000 00:00 0 
7fb619e59000-7fb619e5c000 r-xp 00000000 00:32 41                         /lib/x86_64-linux-gnu/libdl-2.23.so
7fb619e5c000-7fb61a05b000 ---p 00003000 00:32 41                         /lib/x86_64-linux-gnu/libdl-2.23.so
7fb61a05b000-7fb61a05c000 r--p 00002000 00:32 41                         /lib/x86_64-linux-gnu/libdl-2.23.so
7fb61a05c000-7fb61a05d000 rw-p 00003000 00:32 41                         /lib/x86_64-linux-gnu/libdl-2.23.so
7fb61a05d000-7fb61a075000 r-xp 00000000 00:32 85                         /lib/x86_64-linux-gnu/libpthread-2.23.so
7fb61a075000-7fb61a274000 ---p 00018000 00:32 85                         /lib/x86_64-linux-gnu/libpthread-2.23.so
7fb61a274000-7fb61a275000 r--p 00017000 00:32 85                         /lib/x86_64-linux-gnu/libpthread-2.23.so
7fb61a275000-7fb61a276000 rw-p 00018000 00:32 85                         /lib/x86_64-linux-gnu/libpthread-2.23.so
7fb61a276000-7fb61a27a000 rw-p 00000000 00:00 0 
7fb61a27a000-7fb61a905000 r-xp 00000000 00:32 303                        /usr/local/lib/libruby.so.2.4.2
7fb61a905000-7fb61ab05000 ---p 0068b000 00:32 303                        /usr/local/lib/libruby.so.2.4.2
7fb61ab05000-7fb61ab0b000 r--p 0068b000 00:32 303                        /usr/local/lib/libruby.so.2.4.2
7fb61ab0b000-7fb61ab0e000 rw-p 00691000 00:32 303                        /usr/local/lib/libruby.so.2.4.2
7fb61ab0e000-7fb61ab1e000 rw-p 00000000 00:00 0 
7fb61ab1e000-7fb61ab44000 r-xp 00000000 00:32 36                         /lib/x86_64-linux-gnu/ld-2.23.so
7fb61ac04000-7fb61ad3b000 rw-p 00000000 00:00 0 
7fb61ad3c000-7fb61ad3d000 rw-p 00000000 00:00 0 
7fb61ad3d000-7fb61ad3e000 ---p 00000000 00:00 0 
7fb61ad3e000-7fb61ad43000 rw-p 00000000 00:00 0 
7fb61ad43000-7fb61ad44000 r--p 00025000 00:32 36                         /lib/x86_64-linux-gnu/ld-2.23.so
7fb61ad44000-7fb61ad45000 rw-p 00026000 00:32 36                         /lib/x86_64-linux-gnu/ld-2.23.so
7fb61ad45000-7fb61ad46000 rw-p 00000000 00:00 0 
7ffc1a668000-7ffc1ae67000 rw-p 00000000 00:00 0                          [stack]
7ffc1ae6a000-7ffc1ae6c000 r--p 00000000 00:00 0                          [vvar]
7ffc1ae6c000-7ffc1ae6e000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
During startup program terminated with signal SIGABRT, Aborted.

Doesn't handle quoted fields at all

Parses "bar\"baz" as "\"bar\"\"baz\""
Parses "bar,baz" as "\"bar" and "baz\""
Parses empty quoted field as "\"\"" instead of empty string ("")
Doesn't fail on unescaped double quote, e.g. "foo"x"bar"
Doesn't fail on unmatched quote, e.g. "Foo","Bar","Baz
Doesn't fail on whitespace before quoted field, e.g. "Foo","Bar", "Baz"
See #7

Multiple character deliminator support

Please add.

Add option to read from IO objects in addition to files

Fails to parse multiline CSV

multiline.csv

foo,"bar
baz",bzz

Expected output

CSV.foreach('multiline.csv').to_a  # [["foo", "bar\nbaz", "bzz"]]

Actual output

rows = []
Ccsv.foreach(filename) {|row| rows << row}
rows  # [["foo", "\"bar"], ["baz\"", "bzz"]]

Misparses CRLF line endings

CCSV:

[["foo\r"], ["bar\r"], ["baz\r"]]

CSV and all other CSV libraries tested:

[["foo"], ["bar"], ["baz"]]

Row separators should be escaped by double-quoting the entire field, not by backslash-escaping the row separator

No CSV specification escapes fields like in this test https://github.com/evan/ccsv/blob/master/spec/ccsv_spec.rb#L111

Similarly, column separators are not escaped with backslashes, but by using double-quoting https://github.com/evan/ccsv/blob/master/spec/ccsv_spec.rb#L76

Reads empty string for blank row

Try with this file:

foo

Add gemspec

Can't install gem from GitHub without a gemspec.

Adding Encoding options to foreach loop

Currently if the file is encoded in non-UTF-8 format, it doesnt write the rows to the file. Is this something which can be handled ? like CSV.foreach(#{csvfilename},encoding:ISO-8859-1).

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.