Comments (7)
I had a look and tried to process the *_04_GCPL_C16.mpr
file with both the development version of yadg as well as with yadg-4.2.4
, see attached picture:
I don't think the cycle number
is stored in the .mpr
file. Which is also why the NetCDF has NaN
s. It might be calculated by EC-Lab once the .mpr
file is opened there; see #41.
Could you please clarify if this is a regression introduced in yadg-5.x
or whether it's also missing from the json files on your end with yadg-4.2.x
?
from yadg.
@PeterKraus
I have updated all of my environment to yadg5 since we transitioned to yadg5 last week. I tried to make a new conda environment with version 4.2 and tried to run this file but stragely got the following error
Traceback (most recent call last):
File "C:\Users\plnu\Anaconda3\envs\yadg4\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\plnu\Anaconda3\envs\yadg4\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\plnu\Anaconda3\envs\yadg4\Scripts\yadg.exe\__main__.py", line 4, in <module>
File "C:\Users\plnu\Anaconda3\envs\yadg4\lib\site-packages\yadg\__init__.py", line 2, in <module>
from .main import run_with_arguments
File "C:\Users\plnu\Anaconda3\envs\yadg4\lib\site-packages\yadg\main.py", line 8, in <module>
from . import subcommands
File "C:\Users\plnu\Anaconda3\envs\yadg4\lib\site-packages\yadg\subcommands.py", line 8, in <module>
from dgbowl_schemas import to_dataschema, DataSchema
ImportError: cannot import name 'to_dataschema' from 'dgbowl_schemas' (C:\Users\plnu\Anaconda3\envs\yadg4\lib\site-packages\dgbowl_schemas\__init__.py)
Anyhow, since you have tried on your end with both stable and development version of yadg 4.2 and found that there is no record of cycle number, I doubt it would be different on my end.
I investigated this matter a bit further. I used EC-lab (version<11.5) to export this *_04_GCPL_C16.mpr
file into .mpt and found that indeed this very mpr file was re-written. The size of the file increases from 14 mb to 23 mb suggesting that EC-lab may have add some columns into this. You can find the file after conversion here I used yadg5 to process this new mpr file again, but still the cycle number still remain to be 1 and NaN.
When I used EC-lab (version >11.5) to perform mpr to mpt converion, the resulting mpr file after conversion can no longer be processed using yadg5. Despite this file being recorded using EC-lab version <11.5. This strongly suggest that EC-lab does re-write the mpr file after mpt conversion.
I have sent an email to Biologic contact person (Corsin was pissed about them changing mpr format without informing us, so we organized a meeting once). I asked him about whether or not Cycle number is recorded in the original mpr file. I will get back to you If I got any further information.
from yadg.
The Biologic representative has responded my email and said that he cannot comment on the content of the mpr files as they are intellectual property of Biologic SAS.
from yadg.
@NukP Alright. I'll close the issue as there's nothing we can do about "missing" data in yadg. Perhaps the cycle numbers could be computed (e.g. from the half cycle
column), but that should then be implemented in dgpost, as it's post-processing, not extraction. Feel free to open a feature request there.
With regards to installing yadg-4.2
, I suggest you try and install yadg-4.2.5
(which is the latest in that branch) - I just tried it on my machine and it works without issues; if you run into problems, please open a new issue.
from yadg.
@PeterKraus Everything seems to point into the direction that the cycle number is not recorded in mpr file in the first place. One small thing that I don't get is that if the cycle number is not there in the first place, why does yadg read this column name with some of the row having value 1? If this is being recorded, then cycle number should not even be appeared in the extracted column at all.
Anyhow, there is nothing we can do about this and I agree this would be best resolved by calculating the cycle number in dgpost as you suggested.
from yadg.
You're probably merging different techniques into one NetCDF file in your preset (perhaps even with --ignore-merge-errors
). Missing values get back-filled with np.NaN
.
from yadg.
Ah I see, that's probably the reason why. Thanks.
from yadg.
Related Issues (20)
- nvm
- Harcoded String Handling HOT 4
- loss of <I> column in the converted json from the mpr file in GCPL technique using yadg 4 HOT 5
- Wrong Filename extension in extract subcommand? HOT 1
- Rework GitHub Actions HOT 1
- Implement parsing of `ezchrom.dat` files HOT 3
- Error in parsing `.mpr` files with `CA` technique: HOT 1
- Implement support for `CVA` technique
- Issue with yadg yaml recipe file HOT 5
- `basic.csv`: `/` substitution screws up units HOT 1
- EC-Lab 11.50 files not compatible with yadg HOT 2
- Error: Unsipported locale settings HOT 10
- `eclab.mpt`: Parsing of header is prone to crashes.
- `eclab.*`: Better uncertainty estimation for derived quantities
- `eclab.mpt`: add "Analog OUT/V" into columns
- `original_metadata`: Consistent namespacing of metadata HOT 1
- incompatible with eclab 11.60 HOT 9
- Missing value into parameter map HOT 1
- Stepdefaults.py failling to set default locale and raising an error HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yadg.