Port cron to WCOSS2 that generates 48 hours old aircraft dump and prepbufr files that are 100% non-restricted about obsproc HOT 21 OPEN

Working from branch: feature/48hr-old-aircraft-nr-33 in obsproc github repo.
Working from branch: feature/48hr-old-aircraft-nr-33 in prepobs github repo.

from obsproc.

Began to incorporate changes to remorest.f into feature/48hr-old-aircraft-nr-33.
Stopped at line 165.
Reference this diff to view changes that will be worked in from vlab/git.
https://vlab.noaa.gov/redmine/projects/emc-obsproc/repository/emc_obsproc_shared_bufr_remorest/diff?utf8=%E2%9C%93&rev=0f34052a816dd62c11eb9684063864665f8de80c&rev_to=d6b9879af07f4ed25de4e06e623c609a355121c4

from obsproc.

Continue to incorporate changes to remorest.f into feature/48hr-old-aircraft-nr-33
Stopped at line 402.

from obsproc.

Merged develop into feature/48hr-old-aircraft-nr-33 to get the latest updates.

from obsproc.

Continue to incorporate changes to remorest.f into feature/48hr-old-aircraft-nr-33
Stopped at line 739.

from obsproc.

Completed changes to remorest.f in featuer/48hr-old-aircraft-nr-33

from obsproc.

Began to incorporate changes to exdump_post.sh into feature/48hr-old-aircraft-nr-33.
Stopped at line 332.

from obsproc.

Continue to incorporate changes to exdump_post.sh into feature/48hr-old-aircraft-nr-33
Stopped at line 523.

from obsproc.

Completed changes to exdump_post.sh in feature/48hr-old-aircraft-nr-33

from obsproc.

Completed changes JOBSPROC_GLOBAL_DUMP_POST and JOBSPROC_GLOBAL_PREP_POST in feature/48hr-old-aircraft-nr-33

from obsproc.

Was able to build and install on cactus and dogwood: /lfs/h2/emc/obsproc/noscrub/Shelley.Melchior/install/obsproc-nr-33

from obsproc.

Will this become a PR?

from obsproc.

Eventually. Well after go-live. This is a functionality that should go into operations. It has been supported for GDA and UCAR users for years. I will consider adding GPS-RO first, and then visit the possibility of PR and operational implementation. We will discuss in greater detail once I have this functionality working on wcoss2.

from obsproc.

JOBSPROC_GLOBAL_DUMP_POST was run successfully and re-processed the 48-hr old aircar and aircft bufr_d files so that previously restricted reports are now fully available in the nr bufr_d file. For example,
On 20220518 the aircar and aircft bufr_d files from 20220516 are re-processed so that a new nr bufr_d file is generated that includes all reports that were previously eliminated from the nr file because they were still restricted.

Next up, JOBSPROC_GLOBAL_PREP_POST ...

from obsproc.

JOBSPROC_GLOBAL_PREP_POST was run successfully and re-processed the 48-hr old prepbufr_pre-qc and prepbufr files so that previously restricted aircraft reports are now fully available in the nr prepbufr files. For example,
On 20220518 the prepbufr_pre-qc and prepbufr files from 20220516 are re-processed so that new nr prepbufr files are generated that include all reports that were previously eliminated from the nr file because they were still restricted.

from obsproc.

Both JOBSPROC_GLOBAL_DUMP_POST and JOBSPROC_GLOBAL_PREP_POST are running in cron on wcoss2.
Files are scp'd to emcrzdm and populate this ftp directory:
ftp.emc.ncep.noaa.gov/users/smelchior/wcoss2/GDAS_NON_RESTRICTED_48HRS/

End users have been notified.

from obsproc.

This cron is running well on WCOSS2 cactus when that machine is configured as dev. Dogwood is the newly configured dev machine. I will assure the cron is functioning as intended from that platform, too.

from obsproc.

@ShelleyMelchior-NOAA , will this need to go in the upcoming obsproc+ release ?

from obsproc.

Let's save this for the release following the early August release.

from obsproc.

Bump ... internal discussions have moved this target to early 2023.

from obsproc.

obsproc feature branch: feature/48hr-old-aircraft-nr-33 and prepobs feature branch: feature/48hr-old-aircraft-nr-33 have both been merged w/ latest obsproc and prepobs operational code. Both have been installed on WCOSS2 and incorporated in an end-to-end test of global dump, dump_post, prep, and prep_post. Test runs successfully ran to completion w/ expected results. Following the current WCOSS2 maintenance period this test process will be enabled in cron. Code will be prepared for PR. This will happen in early February 2023.

The current state of the code is such that the re-processed *.nr files (*aircar*.bufr_d.nr, *aircft*.bufr_d.nr, *prepbufr*nr) overwrite those that were generated 48 hours earlier. This is the design as coded by original author, Dennis Keyser. May want to check w/ NCO to assure that this is OK.

Also still need to revisit how to get these files onto nomads 48 hours later.

from obsproc.