dr3s / csv-wrangler Goto Github PK

As a developer, I would like to be able to specify data types when parsing fields, so that I can enforce the range of values on the output.

As a developer, I would like a streaming API, so that I can process large files without running out of memory.

As a developer, I would like to be able to transform an object stream, so that I can map inputs to a contract of my choice.

Ex:
Rename Order Number → OrderID and parse as Integer

As a developer, I would like to specify a regex or other pattern for a source field, so that I can validate the input data matches expectations

Ex:
Product Number [A-Z0-9]+
Year YYYY

As a developer I would like to read a CSV file as a stream of objects, so that I can manipulate the data in an incremental fashion

Input:
CSV file path

Output:
Stream of objects

As a developer, I would like to add new fields to the output with fixed values, so that I can set static content.

Ex:
Add a new column with the fixed value "kg" → Unit and parse as String

As a developer, I would like to be able to apply a function on one or more source fields, so that I can derive a new field.

Ex:
Add a new column, concatenating Year, Month, Day → OrderDate and parse as
DateTime

As a developer, I would like feedback on invalid rows, so that I can handle errors on parsing or transforming source data incrementally.

As a developer, I would like to configure the library via a YAML file, so that I can control the library parsing and transformation without writing code.

Ex:

transforms:
  -
    input:
      -
        name: 'Order Number'
        pattern: d+
    output:
      -
        name: OrderId
        type: Integer
        value: 'value(Order Number)'
  -
    input:
      -
        name: Year
        pattern: dddd
      -
        name: Month
        pattern: dd
      -
        name: Day
        pattern: dd
    output:
      -
        name: OrderDate
        type: DateTime
        value: 'dateTime(Year, Month, Day)'

As a developer, I would like documentation of the library, so that I can understand it's architecture and API usage