cempd / smoke Goto Github PK
View Code? Open in Web Editor NEWCreate emissions inputs for multiple air quality modeling systems with unmatched speed and flexibility
Home Page: https://www.cmascenter.org/smoke/
Create emissions inputs for multiple air quality modeling systems with unmatched speed and flexibility
Home Page: https://www.cmascenter.org/smoke/
Prevent overwriting of existing helper files between CONUS and non-CONUS runs and possibly allow a concatenation of CONUS and non-CONUS helper files
lib/initem.f writes out various information at the start of each program. Currently the online documentation link is an old URL that doesn't exist anymore (http://www.cep.unc.edu/empd/products/smoke). Update to https://cmascenter.org/smoke/
Under a particular set of circumstances: when 1) using the new speciation-in-SMOKE option, and 2) also using the relatively new multiple subsector output option (e.g. gas/diesel) at the same time, and 3) when at least one of the subsectors is empty.
It looks like when there is an empty subsector, SMOKE mixes up the different species and puts the value for one species under another species in the subsector files. This only happens when we are doing speciation-in-SMOKE, so this hasn’t come up in our recent runs like 2016v2 platform where we are splitting gas and diesel but are not doing speciation in SMOKE.
In the code "met4moves.f" at line 869, a variable named "FUELIDX" is initialized which is never used. It causes no error in the compile, but when it comes to the run, it causes a segmentation fault and therefore the code crashes.
Removing this line causes the code to run smoothly (I have not checked the results yet).
Would you please take a look at this to see what's the purpose of this variable, and whether it may have any negative impact on the outputs?
well, another gfortran specific issue...
/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/../Linux2_x86_64gfort; gfortran -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/ioapi/fixed_src -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/inc -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/Linux2_x86_64gfort -ffixed-line-length-132 -fno-backslash -O3 -ffast-math -funroll-loops -m64 -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1 -c /home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/emutil/gentpro.f
/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/emutil/gentpro.f:1663.25:
& ((',',PROF_MON(S,NP)),NP = 1,12 )
1
Error: Expected a right parenthesis in expression at (1)
/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/emutil/gentpro.f:1695.34:
& ( (',', PROF_DAY( S,NP ) ), NP = 1,31 )
1
Error: Expected a right parenthesis in expression at (1)
Makefile:348: recipe for target 'gentpro.o' failed
make: *** [gentpro.o] Error 1
Setting the src_id baed on the resolution and it will need to be 12_1, 9_1, or 3_1 depending on the grid size per the approach in table A-2
Another configuration that causes a Smkreport segmentation fault with SMOKE 4.8.1, even with the UNC-compiled version that fixed some of our other issues. These new faults do not happen with Smkreport from 4.8.
To replicate the segmentation fault, start with the public 2016v2 emissions platform, 2016fj_16j case, cmv_c1c2 sector, onetime job (Annual_cmv_c1c2_onetime_12US1_2016fj_16j.csh). Replace the COSTCY, GSPROTMP_A, GSREFTMP_A, and REPCONFIG_INV with the files from the attached inputs. This setup is giving us a segmentation fault referencing the agsnbins.f program.
For a second test, replace EMISINV_A, EMISINV_B, and EMISINV_C with the inventories in the attached inventories. This setup gives a different seg fault, referencing the rdstcy.f program
User cannot run SMOKE for biogenic emissions for more than 5 days using EDGAR data for a domain in Paraguay.
More info here:
https://forum.cmascenter.org/t/smoke-run-halts-on-temporal-process/2540
NORMBEIS4 can BALD6 for both BELD6 and BIOMASS and this should be avoided.
We need one or two additional checks here or maybe some form of these checks:
• Check units of the BELD6 (percent) and BIOMASS files (g/m**2); if they aren’t set correctly then exit in error with some sort of units in error message
• If any of the grid cell values of BELD6 are greater than 100, then write error and exit program in error
Defining of resolution. For nonroad all sources are designated as 12 km sources. It will need to differentiate between 12/9/3
As part of the SMOKE v4.9 release, chapters specific to the old Assigns file and run scripts were removed from the SMOKE User's Guide (v4.9 guide). This how-to guidance needs to be updated based on the Emissions Modeling Platform scripts and made available to users, possibly on the SMOKE GitHub site or through the CMAS wiki.
old chapters from the SMOKE v4.8.1 User's Guide
SMOKE Directory Structure
How to Use SMOKE
Source Code and Include Files
Downloading, Installing, and Compiling SMOKE
The CMAS wiki has a page detailing the 2011v6.3 package:
https://www.airqualitymodeling.org/index.php/SMOKE-NEI-Platform6.3
Each platform provides similar information in a text file:
2016v1
2016v2
2017
Currently, the STKFLW property of any "stack groups" file is incorrect. The value in STKFLW is consistently the stack flow as if stack diameter and velocity were equal to those of the last stack in the file.
The error can be seen on line 1039 of elevpoint.f.[1] On that line, GRPFL is being set using DMVAL and VEVAL. Note that GRPDM and GRPVE do not use DMVAL or VEVAL, which is why they are unaffected. DMVAL was last set on line 705[2] within another loop over sources. As a result, DMVAL is the last diameter of any give set of sources. The same is true for VEVAL[3], which is set one line later than DMVAL.
The fix is to either reset DMVAL and VEVAL before use on line 1039. I have not tested this fix, so this is just my hypothesis.
[1]
Line 1039 in 84eaa07
Line 705 in 84eaa07
Line 706 in 84eaa07
UTM needs to be the same for all sources within the same facility for ptegu and ptnoipm sectors
Gentpro from SMOKE v4.* doesn’t work for ag because of the way SMOKE v4.* formats FIPS codes with extra leading zeroes. So, we use Gentpro from SMOKE v3.7 when creating new met-based profiles for ag. (I’m not sure if this is on UNC’s SMOKE update list; I remember bringing it up with BH when we first encountered it.) For RWC, either version of SMOKE can be used.
The unit for HFLUX in inline emissions file is incorrectly designated as "moles/s" instead of "BTU/hr"
Apparently there are still some issues with the CEMS incorporation and James isn’t getting matches between data going into SMOKE and data coming out of SMOKE. Attached are a few issues from a recent case. James also said the by boiler Smkreport option doesn’t seem to be working properly anymore. This situation is when we derived pseudo CEMS. James thinks a good long-term approach is to move to FF10-hourly but there were a couple of enhancements that would be helpful for that as well.
The most confusing example of this is that smkreport still outputs “Plant ID, Char 1, Char 2, Char 3” instead of “Facility ID, Unit ID, Rel Pt ID, Process ID”.
Well,
getpid is Intel Fortran intrincis function, I am looking for Gfortran counterpart ...
cd /home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/../Linux2_x86_64gfort; gfortran -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/ioapi/fixed_src -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/inc -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/Linux2_x86_64gfort -ffixed-line-length-132 -fno-backslash -O3 -ffast-math -funroll-loops -m64 -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1 -o smkinven adjustinv.o asgnar2pt.o asgnnhapx.o chklstfl.o fixstk.o formlist.o genmedsout.o genpdout.o gethdr.o getpdinfo.o initinfo.o openinvin.o openinvout.o openpdout.o procar2pt.o procinven.o procinvsrcs.o rdcempd.o rdcemsum.o rddataff10ar.o rddataff10mb.o rddataff10pt.o rddatamedspt.o rddatantiar.o rddatantifr.o rddatantimb.o rddatantinp.o rddatantipt.o rdemspd.o rdff10pd.o rdgrdapi.o rdgrdncf.o rdinvdata.o rdinvsrcs.o rdlooppd.o rdmedsinfo.o rdmedspd.o rdorlfr.o rdsrcff10ar.o rdsrcff10mb.o rdsrcff10pt.o rdsrcmedspt.o rdsrcntiar.o rdsrcntifr.o rdsrcntimb.o rdsrcntinp.o rdsrcntipt.o setnonhap.o smkinven.o srcmem.o wrepinven.o wrinvchr.o wrinvemis.o wrinvpol.o wrpdemis.o wrptref.o libemmod.a /home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/../Linux2_x86_64gfort/libsmoke.a /home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/../Linux2_x86_64gfort/libfileset.a -L/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/../Linux2_x86_64gfort -lfileset -lsmoke -lemmod -lfileset -lsmoke -L/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/Linux2_x86_64gfort -lioapi -lnetcdff -lnetcdf -fopenmp -dynamic -L/usr/lib64 -lm -lpthread -lc
rdinvsrcs.o: In function `rdinvsrcs_':
rdinvsrcs.f:(.text+0x3e9): undefined reference to `getpid_'
collect2: error: ld returned 1 exit status
Makefile:717: recipe for target 'smkinven' failed
make: *** [smkinven] Error 1
Update Smkinven and possibly Temporal to process the new FF10 CEMS hourly inventory files
Turning off the grid cell subsetting for 4 km run groups when working with a nonCONUS 9 km or 3 km domain. The tool should not try to split a 3 km grid cell into nine 4 km sources
smk_run.csh currently looks for .debug in the relevant script directory if DEBUGMODE is set. With the new build process, these executables don't exist.
To build IOAPI and relevant libraries:
Previously I have compiled SMOKE 4.8.1 with the following directions for intel compilers and most options disabled:
https://github.com/USEPA/CMAQ/blob/main/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_build_library_intel.md
I had also chosen the medium memory IOAPI Makefile.
Recently trying to work with EQUATES data, I followed this guide that is for gcc, with openmpi and many libraries (HDF,zlib,etc) enabled.
https://github.com/lizadams/EQUATES_BENCHMARK/blob/main/Tutorials/CMAQ_UG_tutorial_build_library_gcc_support_nc4.md
still relies on:
4. Change branches to 20200828 for a tagged stable version
git checkout -b 20200828
but not compiled with medium memory option.
m3tools programs confirmed to work this way with EQUATES SMOKE output (12US1_cmaq_cb6_WR413_MYR_2016.nc4).
There should never be zeroes for velocity or diameter, temperature or any other release param. Zeroes occurred due to rounding and computing flow to velocity. We are getting 0 temperature (Kelvin) it is due to records that are 0 or below 0 Farhenheit. This is occurring in SMOKE report. Not yet fixed. Temperature isn’t working, but others are. Velocity was just a rounding thing so it is fixed.
I suggest adding the topic air-quality
in the About section.
Correct the emission total difference between the inventory and helper files for the nonroad FF10
The nonroad.pl file will need to be changed to allow for diurnal profiles by month. It looks like it currently defaults every nonroad diurnal profile to profile 26. We essentially need lines 120->126 of nonroad.pl removed
Updating SMOKE system to handle both single and double precision lat/lon coordinates for a proper processing
FACILITY_ID, UNIT_ID, REL_POINT_ID, and PROCESS_ID are all 20 characters instead of 15
Update Lat/Lon in point helper location files need to be exactly the same as what is in the FF10 and the UTM should be based on that (UNC: 1/12/2017): Done (CSRA)
Well, upon compilation with gfortran 4.9.2 I got this error:
cd /home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/../Linux2_x86_64gfort; gfortran -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/ioapi/fixed_src -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/inc -I/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/Linux2_x86_64gfort -ffixed-line-length-132 -fno-backslash -O3 -ffast-math -funroll-loops -m64 -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1 -c /home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/biog/normbeis361.f
/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/biog/normbeis361.f:163.22:
INTEGER :: IS_AG, IS_TAG
1
Error: Return type mismatch of function 'is_ag' at (1) (INTEGER(4)/LOGICAL(4))
/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/biog/normbeis361.f:693.36:
IF (IS_AG( M, MODIS12 , MODIS14, NLCD81,
1
Error: IF clause at (1) requires a scalar LOGICAL expression
/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/biog/normbeis361.f:163.30:
INTEGER :: IS_AG, IS_TAG
1
Error: Return type mismatch of function 'is_tag' at (1) (INTEGER(4)/LOGICAL(4))
/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/biog/normbeis361.f:719.40:
IF( IS_TAG (M,MODIS14) ) THEN
1
Error: IF clause at (1) requires a scalar LOGICAL expression
Makefile:348: recipe for target 'normbeis361.o' failed
make: *** [normbeis361.o] Error 1
James has worked on this and waiting for his final test
Small change for the rwc.pl script is line 296 – change the 1 to 0. It was grabbing the 2nd and 3rd characters of the 5 character fips instead of the 1st and 2nd.
Movesmrg was probably coded to apply the adjustment to HONO/NO/NO2 (and total NOX) but not HONO_INV/NO_INV/NO2_INV. When running with the speciation option, model-ready HONO/NO/NO2 are derived from HONO_INV/NO_INV/NO2_INV. (The EF tables contain both HONO_INV/NO_INV/NO2_INV, which have mass units, and HONO/NO/NO2, which have moles units. With the speciation option, we need to use the mass unit variables rather than the moles unit variables because SMOKE will convert to moles when it “speciates”.)
Hi, Is the executable also available for Linux Portland Group Fortran users?
Update Smkreport related code to generate UNIT ID level report for point sources.
Now that we have the code update in SMOKE to support multiple ORIS units for one EIS unit it would be helpful to add a feature to smkreport where emissions can be summarized by ORIS boiler. Currently it only goes to the ORIS facility level. The ORIS boiler level would allow for quick comparisons to the CEMsum values to confirm that the CEM values were incorporated correctly.
Temporal cross-referencing of SCC (00000000000) with county-specific does not working properly
Currently the scripts put out NONRD12 regardless of domain, but it will need to produce the run groups names in table A-1 of the nonpoint/onroad design document according to group and domain
Hello,
I am trying to compile fresh SMOKE-4.7. It requires ioapi library, which I downloaded and compiled in a separate folder.
Now I am trying to find out how to couple this library to avoid the error below:
[email protected]:~/work/air-pollution-modelling/SMOKE/installed/subsys/smoke/src/.m
Makeinclude:44: /home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/ioapi/Makeinclude.: No such file or directory
make: *** No rule to make target '/home/milias/work/air-pollution-modelling/SMOKE/installed/subsys/ioapi/ioapi/Makeinclude.'. Stop.
When running Smkreport with the AERMOD NONPOINT instruction, Smkreport will output the grid cell coordinates in lat-lon. These coordinates aren't matching what would be expected based on the Lambert coordinates and spheroid.
James Beidler will provide examples of correct inputs and outputs.
Here's the section of code that does the coordinate conversions:
Lines 815 to 849 in ffa9e3c
According to the IOAPI documentation, the XY2XY method should take the environment variable IOAPI_ISPH into account: https://www.cmascenter.org/ioapi/documentation/all_versions/html/XY2XY.html
I suggest creating a simple Fortran program that calls XY2XY to see what the output values are, using James' example data.
The behavior has been documented with certain repconfig files and the program either crashes or core dumps. A scenario where the report scrambles the header info with @ symbols may be related and needs to be checked.
CEMs for winter season may have a single value that causes smkinven to put multiple of the seasonal emissions in each hour of the day that contains the active hour.
In rdemspd.f, the LFIP variable is declared as an integer and then initialized as a zero-length string. gfortran throws an error, which seems appropriate (see below).
Makefile:351: recipe for target 'rdemspd.o' failed
/content/subsys/smoke/src/smkinven/rdemspd.f:304:15:
LFIP = ''
1
Error: Can't convert CHARACTER(1) to INTEGER(4) at (1)
make: *** [rdemspd.o] Error 1
In rdorlfr.f, the same problem occurs. In rdff10pd.f, LFIP is a character so no problem there or in rdmedspd.f. I have no idea what value ifort gives LFIP when set to '', but perhaps you could just initialize with that integer value.
Hello,
to prevent GNU gfortran bugs, it would be good to couple this project to travis-ci automatic testing.
Enabling of “fac_source_type" for point (not airports) emission files jn SMOKE modeling system
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.