Giter Club home page Giter Club logo

Comments (10)

yruslan avatar yruslan commented on May 27, 2024

Please, attach a small (10-15 Kb) example data file and the corresponding copybook. We will take a look,

from cobrix.

swami2018 avatar swami2018 commented on May 27, 2024

the tricky part is there is no Cobol copybook for the full file. We have copybook for each record type and unfortunately we can't share the file or copybook.

from cobrix.

yruslan avatar yruslan commented on May 27, 2024

Several copybooks for each record type is fine. The way Cobrix works you are going to need to combine several copybooks into one such as each record type redefines the others using REDEFINES keyword. A video from our DataWorks'2019 presentation will be available on youtube soon. It shows how you can combine copybooks. We can assist you with this as well.

Still, we need an example to implement what you ask. It may not be a production data file / copybook. It can be a generated or obfuscated synthetic example that has the same structure, e.g. several record types and no RDW headers. We need to understand how these 3 letter and 1 letter keywords are used to determine record sizes.

from cobrix.

swami2018 avatar swami2018 commented on May 27, 2024

If we can combine multiple copybooks to one then that solves the problem.
The file looks like the below.
H00..................
R01...................
R05...................
T00...................

Each record type have a fixed length. H00 have a length of 100 and R01 have a length of 3000 etc. Each record type have a copybook and we have option to combine that is great. If this works, then we dont need a byte reader option itself.

from cobrix.

yruslan avatar yruslan commented on May 27, 2024

It is more clear now. Are there any separators between records, e.g. new lines, or the file is pure binary?

Still, if you can provide us with a sample file, it will make our job a lot easier and you can get the feature faster.

from cobrix.

swami2018 avatar swami2018 commented on May 27, 2024

no there are no separator and the file is pure binary.

from cobrix.

swami2018 avatar swami2018 commented on May 27, 2024

Just checking if this DataWorks'2019 presentation video is posted online.

from cobrix.

yruslan avatar yruslan commented on May 27, 2024

Not yet. We are checking it regularly. Will post a link here when the video is available.

from cobrix.

yruslan avatar yruslan commented on May 27, 2024

Here are Cobrix videos:

DataWorks:
https://youtu.be/o_up7X3ZL24

Spark Summit:
https://youtu.be/BOBIdGf3Tm0

(Will be put to the README page shortly)

from cobrix.

yruslan avatar yruslan commented on May 27, 2024

This could be solved using custom record header parsers introduced in #68. We will have a PR for it soon.

from cobrix.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.