Giter Club home page Giter Club logo

publicqcforum's Introduction

Argo Quality Control forum

This repository is here to build a knowledge database and a forum about Quality Control (QC) of Argo data.

You can engage with the Argo QC community by posting issues at:
https://github.com/euroargodev/publicQCforum/issues

You can share use cases and your expertise with codes (through fork and pull-request) and wiki pages: https://github.com/euroargodev/publicQCforum/wiki

This forum is about everything related to Argo QC, for instance:

  • Requests for expertise on difficult data
  • Questions about QC information propagation in the Argo database (eg: greylist, netcdf fields)
  • Illustration of Argo data errors and how to fix them
  • Access and content of reference database
  • etc ...

publicqcforum's People

Contributors

gmaze avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

publicqcforum's Issues

< example of sensor failure> Kistler pressure sensor issue

The Argo 3901931 float was deployed in 2017. It is an Arvor float equipped with an SBE41CP (8497) and a Kistler pressure sensor (4940374).

This float had received a MIN/MAX warning starting around cycle 105 with fresher salinity values (-0.35 psu at depth). After analysis, it appears that there is a problem with the pressure sensor, which reports higher pressure values than the actual values.

image
Temperature (left) and Salinity (right) for float 3901931 in function of pressure. Cycles 99 to 112

image
Theta/S diagram for float 3901931. Cycles 99 to 112

Pressure problem appears clearly on these three plots: note the large temperature shift in the thermocline, the salinity minimum shift from 850db to 1150db, the shift in the theta/S diagram with the profiles acquired after cycle 105 that are no longer parallel to the previous profiles.
Pressure does not seem correctable with a simple pressure offset as the magnitude of the pressure error increased with depth.

Based on email exchanges between K. Martini and R. Cancouet, this is probably related to a defect detected in the Kistler pressure sensors in 2016 : a sudden shift in the pressure span (the calibration slope of pressure). The problem concerns CTDs built between January and July 2016, and this particular CTD was built in May 2016.
The magnitude of the pressure span shift, is 1-30 %, pivoting at 0 pressure, and always one sign – resulting in higher reported pressure than actual pressure. Although the pressure sensors were checked during the routine manufacturing process for the defect, there was a small chance that the check would not catch all the affected sensors. More information can be found here:
http://www.argo.ucsd.edu/Kistler_report_SBE.pdf

Because pressure is not correctable, we put this float into the grey list for PRES, PSAL, et TEMP with a flag 4 from cycle 104.

Have you found similar cases?

[expert] FORTRAN code for RTQC of bio-optical variables

Is your QC expertise request related to a problem? Please describe.
Argo Canada is deploying floats equipped with bio-optics (CHLA, BBP, CDOM) next year. To this point, we have only worked with DOXY in terms of BGC variables, and so we do not have a RTQC method set up for these "new to us" variables. Since our DOXY code is written in FORTRAN, we plan/hope to stay consistent with that. Can anyone share or point me to existing code to do this that we could adapt to our DAC?

Describe the Argo data you'd like to validate
Incoming data stream to the MEDS DAC.

Describe QC methods you've already considered
I'm familiar with the RTQC tests for these variables, just don't want to re-write something I am quite sure already exists!

Additional context
N/A

fast salinity drift

Dear all

I have started a report on all my files which I would like to share with you. It contains all the floats in my dmqc-responsibility which shown this failure mode and it would be great if we could make it a more complete list for europe. I added a few of you to the assignees list, I probably forget people. Please share

Liste_mit_fastsaltydrift.pdf

User Acceptance Test: Python version of the OWC tool

BODC Argo team with consultation with other Argo partners took initiative to convert the OWC software from Matlab to Python. This initiative has been undertaken based on the output from the international survey about the methods and tools used in DMQC core data.

The following forum is dedicated for reporting any issues, bugs and recommendations arrived during the user acceptance testing process.

Area restriction hist data OW

Hi all,

I'm doing the DM for float 6901263.

Regarding the historical data, I'd like OW to do not consider data from the Alboran Sea (Western Mediterranean). Indeed it is a different basin from the Atlantic and with higher salinity values, so the analysis can be misinterpreted. I've talking with Cecile about how we can restrict the Med sea area through changing some scripts (get_region_ow.m or find_besthist.m?) and found convenient to ask here if anyone has been in this situation before and how they have solved the problem.

Thanks!

Screenshot 2023-02-03 at 12 54 13

COVID19 impact on Argo QC activities: suggestions and/or advice

Dear colleagues,
I hope this post will find you well.
With national and international exceptional times due to COVID19, we start to see some impact on the Argo program with regard to deployments.
I'd like to open an issue here with regard to the impact on QC activities.
Working from home I guess will highlight the role for online forum and collaboration tools that we develop here at @euroargodev/all
So please, I invite you to shoot in comments here your suggestions or advice on how to keep going with good Argo QC from home and what are the tools you miss.
Take care.

DMQC and enclosed SSH budget

Here are analysis files one of our own Arvor floats from the upwelling area in the South Atlantic. It is very stable since the beginning but has always shown an negative offset at the lower range of our expected mapping uncertainty. In the past I have always acted on the credo ‘trust your float’ and have not corrected offsets when they were smaller than +-0.01. But since the discussion at las AST I am worried about the unclosed SSH budget. But I am also worried about overcorrecting and forcing all the float data towards climatology, knowing how imperfect the climatology is.
3901670_1
3901670_2
3901670_3
3901670_6
altimeter_comparison_3901670
deloyment_compar_sal3901670
deloyment_compar_sal3901670_detail
deloyment_compar_temp3901670
deloyment_compar_temp3901670_detail
pt_s_anom_map_3901670
For this float a deployment CTD exists with calibrated data and this comparison does not show a need for a negative correction of the float data. If ever the float is too fresh. And I would not have a good explanation why the lab calibration should have been so bad, that the float measures wrongly from the start with a bias of ~-0.01. Therefor I have applied no correction, and since I trust my float I have also left the Qc at 1. But I would like to get your opinion on this. And maybe we need to communicate with the other dm operators how to deal with situations like this.

Record optimal CPcor value for Deep floats

For those who are processing Deep Argo floats in delayed mode, I have created a google sheet where you can record the optimal CPcor value along with the CTD serial number for each of your deep floats. This will help the deep Argo working group track CPcor values to see if there is any batch dependence and possibly re-evaluate the recommended standard CPcor_new values.

An optimized estimate of CPcor can be obtained in delayed-mode by comparing a deep float profile to a reference profile (e.g. deployment CTD casts).
A Matlab routine (COMPUTE_new_CPcor_brian.m) is available on https://github.com/ArgoDMQC/DM_CPcor.
Please, refer to section 3.10 of the Argo Quality Control manual for more information on Deep Argo delayed mode procedure.

The google sheet is available here:
https://docs.google.com/spreadsheets/d/1ai1I0gzyHHRv_n6t2M3BMWVBp1F9XO4L2XB1YhBni9U/edit?usp=sharing

Thanks,

Collection of examples of floats for which the salinity shows a TBTO-like adjustment during the first few profiles.

During the last DMQC discussions, several examples were reported of floats for which the salinity was too fresh in the first cycle and then gradually recovered. The purpose of this issue is to start collecting examples of floats where the same behaviour has been observed.

If you have an example to share, please comment on this issue. Don't forget to include the float number, a brief description of its programming (cycle length...) and any figures you think might be useful (e.g. Figure 3 & 6 from OWC, Theta/S diagram...).

Is there an exhaustive list of situations where N_PROF > 1?

Is your QC expertise request related to a problem? Please describe.
In a given cycle, the netCDF dimension N_PROF is typically 1 (1 cycle, 1 profile), but is sometimes greater than 1 (1 cycle, multiple profiles). In my experience, this is usually when there are observations recorded on the descent, and so there are two profiles for that cycle (files R[wmo]_[cycle].nc R[wmo]_[cycle]D.nc to give an example. There are some situations where there are even more than 2 profiles, and from the file I can't seem to discern from the file why that is (see next section for data). I also have not found an exhaustive lists in any Argo handbook, but hoping I just haven't been looking in the right place and someone could point me to a list or table.

Describe the Argo data you'd like to validate
An example of such a file is here: https://data-argo.ifremer.fr/dac/coriolis/6901494/profiles/BD6901494_352.nc

Describe QC methods you've already considered
Not so much a QC method, but this is a more general question related to this issue in the argoFloats R package: ArgoCanada/argoFloats#413

Additional context
Some python code looking at the netCDF file:

Click to unroll code 

from netCDF4 import Dataset

# get the file - not showing the local path on my machine, just assuming same directory
nc = Dataset('BD6901494_352.nc')
# show the dimension
print(nc.dimensions['N_PROF'])
# <class 'netCDF4._netCDF4.Dimension'>: name = 'N_PROF', size = 4
# variables in this file
print(nc.variables.keys())
# dict_keys(['DATA_TYPE', 'FORMAT_VERSION', 'HANDBOOK_VERSION', 
# 'REFERENCE_DATE_TIME', 'DATE_CREATION', 'DATE_UPDATE', 'PLATFORM_NUMBER', 
# 'PROJECT_NAME', 'PI_NAME', 'STATION_PARAMETERS', 'CYCLE_NUMBER', 'DIRECTION', 
# 'DATA_CENTRE', 'DC_REFERENCE', 'DATA_STATE_INDICATOR', 'DATA_MODE', 
# 'PARAMETER_DATA_MODE', 'PLATFORM_TYPE', 'FLOAT_SERIAL_NO', 'FIRMWARE_VERSION', 
# 'WMO_INST_TYPE', 'JULD', 'JULD_QC', 'JULD_LOCATION', 'LATITUDE', 'LONGITUDE', 
# 'POSITION_QC', 'POSITIONING_SYSTEM', 'PROFILE_RAW_DOWNWELLING_IRRADIANCE380_QC', 
# 'PROFILE_RAW_DOWNWELLING_IRRADIANCE412_QC', 
# 'PROFILE_RAW_DOWNWELLING_IRRADIANCE490_QC', 'PROFILE_RAW_DOWNWELLING_PAR_QC', 
# 'PROFILE_DOWN_IRRADIANCE380_QC', 'PROFILE_DOWN_IRRADIANCE412_QC', 
# 'PROFILE_DOWN_IRRADIANCE490_QC', 'PROFILE_DOWNWELLING_PAR_QC', 
# 'PROFILE_FLUORESCENCE_CHLA_QC', 'PROFILE_BETA_BACKSCATTERING700_QC', 
# 'PROFILE_FLUORESCENCE_CDOM_QC', 'PROFILE_CHLA_QC', 'PROFILE_BBP700_QC', 
# 'PROFILE_CDOM_QC', 'VERTICAL_SAMPLING_SCHEME', 'CONFIG_MISSION_NUMBER', 'PRES', 
# 'TEMP_STD', 'PSAL_STD', 'PRES_MED', 'TEMP_MED', 'PSAL_MED', 
# 'RAW_DOWNWELLING_IRRADIANCE380', 'RAW_DOWNWELLING_IRRADIANCE380_QC', 
# 'RAW_DOWNWELLING_IRRADIANCE412', 'RAW_DOWNWELLING_IRRADIANCE412_QC', 
# 'RAW_DOWNWELLING_IRRADIANCE490', 'RAW_DOWNWELLING_IRRADIANCE490_QC', 
# 'RAW_DOWNWELLING_PAR', 'RAW_DOWNWELLING_PAR_QC', 
# 'RAW_DOWNWELLING_IRRADIANCE380_STD', 'RAW_DOWNWELLING_IRRADIANCE412_STD', 
# 'RAW_DOWNWELLING_IRRADIANCE490_STD', 'RAW_DOWNWELLING_PAR_STD', 
# 'RAW_DOWNWELLING_IRRADIANCE380_MED', 'RAW_DOWNWELLING_IRRADIANCE412_MED', 
# 'RAW_DOWNWELLING_IRRADIANCE490_MED', 'RAW_DOWNWELLING_PAR_MED', 
# 'DOWN_IRRADIANCE380', 'DOWN_IRRADIANCE380_QC', 'DOWN_IRRADIANCE380_ADJUSTED', 
# 'DOWN_IRRADIANCE380_ADJUSTED_QC', 'DOWN_IRRADIANCE380_ADJUSTED_ERROR', 
# 'DOWN_IRRADIANCE412', 'DOWN_IRRADIANCE412_QC', 'DOWN_IRRADIANCE412_ADJUSTED', 
# 'DOWN_IRRADIANCE412_ADJUSTED_QC', 'DOWN_IRRADIANCE412_ADJUSTED_ERROR', 
# 'DOWN_IRRADIANCE490', 'DOWN_IRRADIANCE490_QC', 'DOWN_IRRADIANCE490_ADJUSTED', 
# 'DOWN_IRRADIANCE490_ADJUSTED_QC', 'DOWN_IRRADIANCE490_ADJUSTED_ERROR', 
# 'DOWNWELLING_PAR', 'DOWNWELLING_PAR_QC', 'DOWNWELLING_PAR_ADJUSTED', 
# 'DOWNWELLING_PAR_ADJUSTED_QC', 'DOWNWELLING_PAR_ADJUSTED_ERROR', 
# 'FLUORESCENCE_CHLA', 'FLUORESCENCE_CHLA_QC', 'BETA_BACKSCATTERING700', 
# 'BETA_BACKSCATTERING700_QC', 'FLUORESCENCE_CDOM', 'FLUORESCENCE_CDOM_QC', 
# 'FLUORESCENCE_CHLA_STD', 'BETA_BACKSCATTERING700_STD', 'FLUORESCENCE_CDOM_STD', 
# 'FLUORESCENCE_CHLA_MED', 'BETA_BACKSCATTERING700_MED', 'FLUORESCENCE_CDOM_MED', 
# 'CHLA', 'CHLA_QC', 'CHLA_ADJUSTED', 'CHLA_ADJUSTED_QC', 'CHLA_ADJUSTED_ERROR', 
# 'BBP700', 'BBP700_QC', 'BBP700_ADJUSTED', 'BBP700_ADJUSTED_QC', 
# 'BBP700_ADJUSTED_ERROR', 'CDOM', 'CDOM_QC', 'CDOM_ADJUSTED', 'CDOM_ADJUSTED_QC', 
# 'CDOM_ADJUSTED_ERROR', 'HISTORY_INSTITUTION', 'HISTORY_STEP', 'HISTORY_SOFTWARE', 
# 'HISTORY_SOFTWARE_RELEASE', 'HISTORY_REFERENCE', 'HISTORY_DATE', 'HISTORY_ACTION', 
# 'HISTORY_PARAMETER', 'HISTORY_START_PRES', 'HISTORY_STOP_PRES', 
# 'HISTORY_PREVIOUS_VALUE', 'HISTORY_QCTEST', 'PARAMETER', 
# 'SCIENTIFIC_CALIB_EQUATION', 'SCIENTIFIC_CALIB_COEFFICIENT', 
# 'SCIENTIFIC_CALIB_COMMENT', 'SCIENTIFIC_CALIB_DATE'])
# pressures for each profile
for i in range(nc.dimensions['N_PROF'].size):
    print(nc['PRES'][:].data[i,:])
# [2.1000e+00 2.6000e+00 3.5000e+00 4.6000e+00 5.4000e+00 6.5000e+00
#  7.5000e+00 8.6000e+00 9.7000e+00 1.0700e+01 1.1500e+01 1.2400e+01
#  1.3500e+01 1.4600e+01 1.5600e+01 1.6700e+01 1.7500e+01 1.8300e+01
#  1.9400e+01 2.0600e+01 2.1600e+01 2.2600e+01 2.3400e+01 2.4400e+01
#  2.5700e+01 2.6400e+01 2.7300e+01 2.8300e+01 2.9400e+01 3.0600e+01
#  3.1700e+01 3.2600e+01 3.3300e+01 3.4300e+01 3.5600e+01 3.6700e+01
#  3.7700e+01 3.8400e+01 3.9400e+01 4.0700e+01 4.1700e+01 4.2600e+01
#  4.3400e+01 4.4500e+01 4.5600e+01 4.6600e+01 4.7300e+01 4.8300e+01
#  4.9600e+01 5.0600e+01 5.1600e+01 5.2600e+01 5.3600e+01 5.4300e+01
#  5.5700e+01 5.6500e+01 5.7500e+01 5.8700e+01 5.9600e+01 6.0300e+01
#  6.1400e+01 6.2700e+01 6.3600e+01 6.4300e+01 6.5500e+01 6.6600e+01
#  6.7300e+01 6.8300e+01 6.9500e+01 7.0500e+01 7.1400e+01 7.2500e+01
#  7.3500e+01 7.4600e+01 7.5700e+01 7.6500e+01 7.7300e+01 7.8300e+01
#  7.9500e+01 8.0500e+01 8.1700e+01 8.2700e+01 8.3700e+01 8.4700e+01
#  8.5700e+01 8.6500e+01 8.7300e+01 8.8400e+01 8.9500e+01 9.0700e+01
#  9.1500e+01 9.2300e+01 9.3500e+01 9.4600e+01 9.5500e+01 9.6300e+01
#  9.7400e+01 9.8500e+01 9.9400e+01 1.0030e+02 1.0130e+02 1.0250e+02
#  1.0340e+02 1.0430e+02 1.0560e+02 1.0650e+02 1.0740e+02 1.0860e+02
#  1.0940e+02 1.1040e+02 1.1160e+02 1.1240e+02 1.1330e+02 1.1460e+02
#  1.1570e+02 1.1660e+02 1.1740e+02 1.1840e+02 1.1950e+02 1.2060e+02
#  1.2160e+02 1.2270e+02 1.2350e+02 1.2440e+02 1.2530e+02 1.2650e+02
#  1.2760e+02 1.2830e+02 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04]
# [6.0000e-01 1.5000e+00 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04]
# [5.0000e-01 1.5000e+00 2.2000e+00 2.9000e+00 3.7000e+00 4.4000e+00
#  5.1000e+00 6.0000e+00 6.8000e+00 7.6000e+00 8.2000e+00 8.9000e+00
#  9.6000e+00 1.0300e+01 1.1000e+01 1.1800e+01 1.2600e+01 1.3600e+01
#  1.4500e+01 1.5400e+01 1.6300e+01 1.7100e+01 1.8000e+01 1.9000e+01
#  2.0000e+01 2.1100e+01 2.2300e+01 2.3400e+01 2.4500e+01 2.5700e+01
#  2.6900e+01 2.8000e+01 2.9200e+01 3.0300e+01 3.1500e+01 3.2600e+01
#  3.3800e+01 3.4900e+01 3.6100e+01 3.7200e+01 3.8400e+01 3.9500e+01
#  4.0700e+01 4.1900e+01 4.3100e+01 4.4200e+01 4.5400e+01 4.6600e+01
#  4.7700e+01 4.8900e+01 5.0100e+01 5.1200e+01 5.2300e+01 5.3400e+01
#  5.4500e+01 5.5600e+01 5.6700e+01 5.7800e+01 5.8900e+01 6.0000e+01
#  6.1100e+01 6.2200e+01 6.3200e+01 6.4300e+01 6.5300e+01 6.6500e+01
#  6.7600e+01 6.8700e+01 6.9900e+01 7.0900e+01 7.1800e+01 7.2500e+01
#  7.3100e+01 7.3800e+01 7.4400e+01 7.5100e+01 7.5800e+01 7.6500e+01
#  7.7200e+01 7.7900e+01 7.8600e+01 7.9300e+01 8.0100e+01 8.0800e+01
#  8.1600e+01 8.2500e+01 8.3300e+01 8.4100e+01 8.5000e+01 8.5800e+01
#  8.6700e+01 8.7600e+01 8.8500e+01 8.9400e+01 9.0200e+01 9.1100e+01
#  9.2100e+01 9.3000e+01 9.3900e+01 9.4900e+01 9.5900e+01 9.6900e+01
#  9.7900e+01 9.8800e+01 9.9800e+01 1.0080e+02 1.0180e+02 1.0280e+02
#  1.0380e+02 1.0490e+02 1.0590e+02 1.0690e+02 1.0790e+02 1.0900e+02
#  1.1000e+02 1.1110e+02 1.1220e+02 1.1320e+02 1.1440e+02 1.1540e+02
#  1.1660e+02 1.1770e+02 1.1890e+02 1.1990e+02 1.2110e+02 1.2220e+02
#  1.2330e+02 1.2440e+02 1.2540e+02 1.2630e+02 1.2700e+02 1.2740e+02
#  1.2750e+02 1.2760e+02 1.2770e+02 1.2780e+02 1.2790e+02 1.2790e+02
#  1.2800e+02 1.2810e+02 1.2820e+02 1.2820e+02 1.2820e+02 1.2830e+02
#  1.2830e+02 1.2830e+02 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04 9.9999e+04
#  9.9999e+04 9.9999e+04 9.9999e+04]
# [  0.5   1.1   1.2   1.3   1.5   1.6   1.9   2.1   2.4   2.6   2.8   3.1
#    3.2   3.5   3.7   3.8   4.    4.    4.2   4.4   4.9   5.3   5.4   5.7
#    5.9   6.2   6.3   6.6   6.8   7.    7.1   7.3   7.7   8.    8.    8.2
#    8.3   8.5   8.8   8.9   9.    9.2   9.3   9.4  10.1  10.8  11.6  12.5
#   13.4  14.4  15.2  16.1  16.9  17.8  18.8  19.8  20.9  22.   23.2  24.3
#   25.5  26.6  27.7  28.9  30.   31.2  32.4  33.5  34.7  35.8  37.   38.2
#   39.3  40.5  41.7  42.8  44.   45.2  46.3  47.5  48.7  49.8  51.   52.
#   53.2  54.2  55.3  56.5  57.6  58.7  59.8  60.9  61.9  63.   64.   65.1
#   66.2  67.3  68.5  69.6  70.7  71.6  72.4  73.   73.7  74.3  75.   75.7
#   76.3  77.1  77.8  78.4  79.2  79.9  80.7  81.5  82.3  83.1  84.   84.8
#   85.6  86.5  87.4  88.3  89.2  90.1  90.9  91.9  92.8  93.8  94.7  95.7
#   96.7  97.7  98.6  99.7 100.6 101.7 102.6 103.6 104.7 105.7 106.7 107.7
#  108.8 109.8 110.9 111.9 113.  114.1 115.2 116.4 117.5 118.6 119.7 120.8
#  122.  123.1 124.2 125.2 126.1 126.9 127.3 127.5 127.6 127.7 127.8 127.8
#  127.9 128.  128.1 128.1 128.2 128.2 128.2 128.3 128.3]
# for all N_PROF except the last one, biological variables (CHLA, BBP, CDOM) are all fillvalues
for i in range(nc.dimensions['N_PROF'].size):
    print(nc['CHLA'][:].data[i,:])
# Out:
# [99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999.]
# [99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999.]
# [99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999. 99999.
#  99999. 99999. 99999. 99999. 99999. 99999. 99999.]
# [-0.1533  0.073   0.365   0.1898  0.4745  0.5256  0.2555  0.1679  0.2117
#  -0.2555 -0.0657 -0.0146  0.3723  0.0876  0.0219  0.5183  0.2701 -0.0438
#   0.2774  0.2847 -0.1825 -0.0657  0.0584 -0.0949  0.2044  0.0584 -0.2409
#   0.1898 -0.0511  0.1752  0.4088  0.146   0.6132  0.3431  0.5548  0.0438
#   0.2701 -0.1606  0.5183  0.292   0.6497  0.0584  0.9052 -0.3504  0.1679
#   0.1679  0.1971  0.1971  0.1971  0.219   0.1898  0.219   0.2263  0.2482
#   0.2555  0.2263  0.1971  0.1825  0.1752  0.1898  0.2117  0.1971  0.2117
#   0.219   0.2263  0.2336  0.2628  0.2701  0.2774  0.2774  0.2628  0.2701
#   0.292   0.3066  0.3066  0.3139  0.3358  0.3504  0.3431  0.4161  0.4307
#   0.4818  0.4818  0.4891  0.5037  0.5256  0.5913  0.6862  0.6716  0.7008
#   0.7665  0.6862  0.6643  0.6351  0.5986  0.511   0.5402  0.4891  0.4818
#   0.438   0.5037  0.4453  0.4453  0.4307  0.4672  0.4161  0.4088  0.4599
#   0.4088  0.4015  0.4088  0.4234  0.4307  0.4234  0.4088  0.4453  0.4453
#   0.4599  0.438   0.4891  0.4234  0.4453  0.4307  0.4161  0.4307  0.3942
#   0.3869  0.3796  0.4161  0.4015  0.3796  0.3577  0.3139  0.3139  0.3285
#   0.3066  0.3066  0.3139  0.3066  0.3139  0.3066  0.3139  0.2993  0.2774
#   0.2774  0.2482  0.2482  0.2409  0.219   0.219   0.2044  0.1825  0.1752
#   0.2336  0.2117  0.1898  0.1606  0.1679  0.1606  0.1533  0.146   0.1533
#   0.1533  0.146   0.1679  0.1606  0.1606  0.1387  0.1679  0.1533  0.1533
#   0.146   0.1752  0.1606  0.1825  0.146   0.1606]
</p>
</details>

North Atlantic weird float

Is your QC expertise request related to a problem? Please describe.
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Describe the Argo data you'd like to validate
A clear and concise description of the Argo data you want QC expertise for (eg: location, model, wmo, etc ...)

Status of Machine Learning for Argo QC

I'd like to open a discussion thread to get the status of developments with regard to the use of Machine Learning techniques in Argo QC procedures.

Different groups may have started to explore this possibility and it would be constructive to get here the status of these efforts, to avoid duplicates and to get feedback.

This could include a description of:

  • the target variables (eg: QC flag for one TEMP measure, QC flag for one PSAL profile,...)
  • the choice of features, explanatory variables
  • the ML method (eg: random forest)
  • the dataset used
  • the overall performance or difficulties encountered
  • anything you think relevant wrt this topic

2nd European Argo/7th International Argo Delayed-mode QC Workshop for CTD data

In 2018, two Argo DMQC Workshops for CTD data were held: the 1st European Argo DMQC Workshop in April 2018 and the 6th International Argo DMQC Workshop in December 2018. These two workshops aimed to bring all delayed-mode operators towards the same level of knowledge of the QC procedures and understanding of the Argo data system. Sharing of community tools on Github and a DMQC cookbook were initiated as a result.

In 2019, the 2nd Deep-Argo Workshop was held in May 2019 and revealed sufficient expertise in the Deep-Argo community to start formulating some QC procedures. The annual ADMT meeting in October 2019 saw the beginning of the BODC project to convert the OWC tool from Matlab to Python. The Seabird CTD salty drifts remained an ongoing issue.

In 2020, the 2nd European Argo and the 7th International Argo Delayed- mode QC Workshop for CTD data will be jointly held in May 2020, hosted by BODC in Liverpool, UK. The workshop will be held over 3 to 4 days, depending on how the agenda unfolds, during the week starting 11 May 2020.

The agenda of the workshop will focus on the following topics:

  • A full day of interactive session dedicated to looking at examples and best practices for salty-drifting CTDs and other regional cases, including marginal seas
  • Deep-Argo QC procedures
  • Introduction of BODC's Python version of the OWC tool
  • Compilation of the DMQC Cookbook
  • The collaborative framework for DMQC: how to use and contribute to
    online tools (repositories, Github issues) and QC forum
  • Miscellaneous DMQC discussions: machine learning methods
    development, access to and content of reference data, other shared tools (visualization), how to keep track of delayed-mode operators, etc.

~ from Brian King, @gmaze , Annie Wong, John Gilson

More information to follow.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.