nreco / csv Goto Github PK

View Code? Open in Web Editor NEW

139.0 10.0 33.0 22 KB

Fast C# CSV parser

Home Page: https://www.nrecosite.com

License: MIT License

C# 100.00%

csv csv-parser csv-reader dotnet dotnet-core csv-parsing csharp

csv's Introduction

NReco.Csv

Ultra-fast C# CSV parser: implements stream reader and writer.

very fast: x2-x4 times faster than JoshClose's CSVHelper
memory efficient: uses only single circular buffer, no allocations in heap for CSV of any size
lightweight: bare csv parser with simple API
tolerant to not-fully correct CSV files, you can control max length of CSV file (useful for processing end-user CSV uploads)
can be used for stream processing of many-GB CSV files
supports all .NET versions: Framework 4.5+, .NET Core, NET6+

How to use

Parse CSV stream:

using (var streamRdr = new StreamReader(inputStream)) {
  var csvReader = new CsvReader(streamRdr, ",");
  while (csvReader.Read()) {
    for (int i=0; i<csvReader.FieldsCount; i++) {
      string val = csvReader[i];
    }
  }
}

Generate CSV to stream:


using (var streamWr = new StreamWriter(outputStream)) {
  var csvWriter = new CsvWriter(streamWr);
  // write line
  csvWriter.WriteField("Value with double quote\"");
  csvWriter.WriteField("And with\nnew line");
  csvWriter.WriteField("Normal");
  csvWriter.NextRecord();
}

Who is using this?

NReco.Csv is in production use at SeekTable.com and PivotData microservice.

License

Distributed under the MIT license

csv's People

Contributors

Stargazers

Watchers

csv's Issues

Possible StackOverflow Exception if file with many empty lines in a row is passed.

The Read() method calls itself when an empty line is encountered. If there are sufficient empty lines in a file, this has the possibility of causing a stack overflow exception, killing the entire process. Could you please make it so that the Read() method works without recursion?

Add IDataReader implementation

In some cases it might be useful to read CSV stream as System.Data IDataReader.
This unlocks simple way to read CSV to DataTable or to list of POCO-models - say, this is possible with DataReaderResult class from NReco.Data library.

In the initial implementation, lets expose all CSV line fields as 'string' FieldType, and add option to skip first (headers) line.

Custom quote

Why there is no possibility to accept a different quote?

Something like this
var csvReader = new CsvReader(stream, delimiter, quote); (eventually also comment char....)

This lib is very fast but also incomplete with only delimiter. I hope you add this feature.

Add API for ReadOnlySpan for efficient values handling

In .NET Core 2.1 and upcoming netstandard 2.1 ReadOnlySpan can be used for zero-allocation handling of string values - without need to create 'string' object from the buffer.

To support this it is enough to add ProcessValueInBuffer overload or maybe add method ReadOnlySpan<char> GetSpanValue(int idx).

Getting error intermittently "Object reference not set to an instance of an object."

Stack trace:
at NReco.Csv.CsvReader.GetOrAddField(Int32 startIdx)
at NReco.Csv.CsvReader.Read()

Using version 1.0.1

CsvWriter: add an option to disable double quotes for values with leading/trailing spaces

Currently CsvWriter always wraps with double quotes values that start and/or end with space. This is allowed by RFC4180, however double quotes are not required in that case.

Let's add an option to control this behaviour.

Encoding issue in readme

Example in Readme has < instead of <

for (int i=0; i&lt;csvReader.FieldsCount; i++) {

for (int i=0; i<csvReader.FieldsCount; i++) {

Feature for getting count of read and skipped lines.

Introduce 2 new properties - SkippedLines and ReadLines or similarly named that gets the point across.

SkippedLines - returns the number of skipped lines of the file.
ReadLines - returns the number of read lines of the file.

I'll attempt to tackle this now as I'm in need of it. Let me know if this is something you'd like to see implemented in the library!

JSON field support

File contains 3 columns:
AddressId, JsonData, UpdatedAt
1,"{"address": {"zip": null, "city": null, "state": null, "country": "USA", "street1": null, "street2": null, "street3": null}}","2021-03-31T23:36:04"
2,"{"guardian": {"name": "Abhishek", "phone": true, "address": {"zip": "6204", "city": "York", "state": "PA", "country": "USA", "street1": "62115B Greenwood Rd", "street2": null, "street3": null}, "relationship": "Son"}}","2021-03-31T23:36:04"

I am not able to find "UpdatedAt" field at csvReader[2]. Please add json support also. thanks

readme error about "StreamWrite"

in readme,how to use, "StreamWrite" should be "StreamWriter"

[Question] Does NReco.Csv write fully-correct CSV files?

It seems the library is capable of reading not fully correct CSV files. if you can expand on the differences, as I'm guessing your model is specified in RFC4180?

With that said, does the library itself produce CSV files conforming to the above RFC standard?