Comments (10)
Please, attach a small (10-15 Kb) example data file and the corresponding copybook. We will take a look,
from cobrix.
the tricky part is there is no Cobol copybook for the full file. We have copybook for each record type and unfortunately we can't share the file or copybook.
from cobrix.
Several copybooks for each record type is fine. The way Cobrix works you are going to need to combine several copybooks into one such as each record type redefines the others using REDEFINES keyword. A video from our DataWorks'2019 presentation will be available on youtube soon. It shows how you can combine copybooks. We can assist you with this as well.
Still, we need an example to implement what you ask. It may not be a production data file / copybook. It can be a generated or obfuscated synthetic example that has the same structure, e.g. several record types and no RDW headers. We need to understand how these 3 letter and 1 letter keywords are used to determine record sizes.
from cobrix.
If we can combine multiple copybooks to one then that solves the problem.
The file looks like the below.
H00..................
R01...................
R05...................
T00...................
Each record type have a fixed length. H00 have a length of 100 and R01 have a length of 3000 etc. Each record type have a copybook and we have option to combine that is great. If this works, then we dont need a byte reader option itself.
from cobrix.
It is more clear now. Are there any separators between records, e.g. new lines, or the file is pure binary?
Still, if you can provide us with a sample file, it will make our job a lot easier and you can get the feature faster.
from cobrix.
no there are no separator and the file is pure binary.
from cobrix.
Just checking if this DataWorks'2019 presentation video is posted online.
from cobrix.
Not yet. We are checking it regularly. Will post a link here when the video is available.
from cobrix.
Here are Cobrix videos:
DataWorks:
https://youtu.be/o_up7X3ZL24
Spark Summit:
https://youtu.be/BOBIdGf3Tm0
(Will be put to the README page shortly)
from cobrix.
This could be solved using custom record header parsers introduced in #68. We will have a PR for it soon.
from cobrix.
Related Issues (20)
- copybook meta data for RDBMS HOT 5
- ADLS support HOT 1
- Mainframe Condensed data HOT 1
- Is it possible to flatten a nested schema so all values are the root? HOT 5
- COMP-3 field is being read with a value 3 less than expected value HOT 3
- Df to sas7bdat file writer HOT 3
- Installing Cobrix Libraries HOT 1
- record_format VB file fails with length of BDW block is too big HOT 7
- File start/end offset issue #601 HOT 4
- Make project Spark 3.5 compatible.
- File start/end offset issue for VB file HOT 5
- RDW headers should never be zero (0,0,0,0). Found zero size record at 4078719. HOT 1
- Stream processing with Flink HOT 1
- Not able to run simple cobol app with java HOT 1
- Process ASCII file with fixed length format HOT 5
- US ASCII file with newline character present within data HOT 1
- ebcdic_code_page for German character Γ€,Γ,ΓΌ HOT 9
- Can I get the raw record bytes from ebcdic file w/out parsing HOT 4
- BBBB in copybook HOT 3
- Is it possible to read a nested Binary Field? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cobrix.