I managed to kill the very old pyArt version Scott installed on my machine a long time ago. So I am reinstalling.

FYI, I have anaconda installed and am working of many of their distributions. The reason for this was simply that NCAR suggest this for the eventual PySolo software. I wanted to give you feedback on my install process and I'll address a couple of things below that I hope are helpful in numeric fashion. Basically as I encounter them or remember.

So you know some package versions:

nickguy[//anaconda]: conda list numpy

packages in environment at //anaconda:

numpy 1.7.1 py27_0
nickguy[//anaconda]: conda list scipy

packages in environment at //anaconda:

scipy 0.12.0 np17py27_0
nickguy[//anaconda]: conda list matplotlib

packages in environment at //anaconda:

matplotlib 1.3.0 np17py27_0
nickguy[//anaconda]: conda list netCDF4

packages in environment at //anaconda:

netcdf4 1.0.5 np17py27_0

1. I had to install baseman using > conda basemap install
   This resulted in matplotlib going down to version 1.2.1
   So I had to manually update matplotlib to v 1.3.0 using conda manager.
2. You might want to note that pyproj is also a needed dependency. I searched and found at:
   http://code.google.com/p/pyproj/

It installed with no issues.

This is mostly as a placeholder to reduce duplicated work.
Py-Art needs the ability to write UF files. I have most of a PR ready that implements this, just gotta iron out a few things and ensure RSL compatibility.

As pointed out by @kirknorth

When creating a Grid with the function grid_from_radar gates which are masked in non-reflectivity fields can contribute to the interpolation if the corresponding gate is not masked in the reflectivity field.

A work around for this issue is to map each field to a grid individually and set the refl_field parameter to the field being mapped. Any filtering of gates based on the actual reflectivity field would need to be performed prior to this.

The following script shows this bug and the workaround

import numpy as np
import matplotlib.pyplot as plt
import pyart

# create a radar with 2 fields
# a reflectivity field of all zeros and a velocity field with
# a masked center.
radar = pyart.testing.make_empty_ppi_radar(100, 360, 1)
radar.add_field('reflectivity', {'data': np.zeros((360, 100))})
vdata = np.ma.zeros((360, 100))
vdata[:, :60] = 999.0
vdata[:, :60] = np.ma.masked
radar.add_field('velocity', {'data': vdata})


# create a grid
grid = pyart.map.grid_from_radars(
    (radar, ),
    grid_shape=(21, 21, 2),
    grid_limits=((-1000, 1000), (-1000, 1000), (0, 1.)),
    qrf_func=lambda x, y, z: 20.)

# plot, notice that 999 appear in the grid
print grid.fields['velocity']['data'][0, 11, 11]
plt.imshow(grid.fields['velocity']['data'][0])
plt.show()


# work-around, grid with filtering on the velocity field
grid = pyart.map.grid_from_radars(
    (radar, ),
    refl_field='velocity',
    fields=['velocity'],
    grid_shape=(21, 21, 2),
    grid_limits=((-1000, 1000), (-1000, 1000), (0, 1.)),
    qrf_func=lambda x, y, z: 20.)

# center is masked out as it should be
print grid.fields['velocity']['data'][0, 11, 11]
plt.clf()
plt.imshow(grid.fields['velocity']['data'][0])
plt.show()

Output:

999.0
--

Images:

The root cause are lines 446-452 in the file grid_mapper.py. The copy to field_data or field_data_objs does not take into account for the mask. Adding masked to these objects and using masked aware functions during the interpolation should fix this.

Currently PyArt doesn't really open IRIS Raw files correctly. It reads in a few variables, but ignores the majority of the 44(including zdr). I'm mostly sticking this here as a placeholder to remind myself that when I get time I need to fix this.

Additionally several of the variables are named wrong. ZT should be total power instead of reflectivity for instance.
I added a default variable type and if I get time tonight I'll shoehorn in it loading any not defined with that. Afterwards however we should add the proper definitions for them,.

The speed of the sobel convolution can be considerably improved by using the scipy.ndimage.filter.convolve1d function as follows:

sobel = 2. * np.arange(window_len)/(window_len-1.0) - 1.0
sobel = sobel/(abs(sobel).sum())
sobel = sobel[::-1]
kdp = scipy.ndimage.filters.convolve1d(proc_ph['data'], sobel, axis=1) / ((len(sobel)/3.0) * 2.0 * (radar.range['data'][1] - radar.range['data'][0]) / 1000.0))

I've noticed that the lat and lon variables in a Grid object are no longer the latitude and longitude corresponding to the x and y axes, i.e. len(x) == len(lon) and len(y) == len(lat). The original grids @scollis created were like this. Instead it is now the latitude and longitude at the origin. This is fine, but for wind retrieval efforts we need to know the location of the radar in the grid, and this was easily done before by something like j = abs(grid_lats - radar_lat).argmin() to get the y index of the radar in the grid.

So I'd like to recommend that we include both the full latitude and longitude coordinates of the grid axes like we were before, as well as the latitude and longitude of the radar itself in the Grid object.

Add a rays_per_sweep read only property to the Radar object which determined the number of rays in each sweep. This can be calculated as:

radar.sweep_end_ray_index['data'] - radar.sweep_start_ray_index['data'] + 1

NEXRAD files can contain volumes where the gate spacing and gate locations are not constant between sweeps. When these file are read, radar.range is set to the gate spacing for the sweep with the largest number of gates. These results in some sweeps having an incorrect range. An example file which displays this property is KMLB20101118_014806_V03 which is available from the NCDC.

File exhibiting this behavior should at the least detected and a warning/error raised. If possible find a method for returning all or some of the sweeps with the correct gate spacing (multiple radar objects?).

Thanks to Timoth Lang for reporting this bug.

I have a version of PyArt reading .CHL files. The code is ugly at this point(I was parsing through the C file to figure out the format and so the code was written as I figured out each part). It's in my fork of pyart under the chl_reader branch if anyone wants to give comments on it in the meantime.

I'll try to get it cleaned up over next week or three and submit an actual pull request(There is a laundry list of things to make it palatable), but it's there if anyone is impatient and wants to clean it up in the meantime. It at least gets the data into a py-art radar object that works with display.

You can find an example CHL file at
https://www.dropbox.com/s/s6pczejem8kajhn/CHL20120705_230123

If nothing else you might be amazed by just how many different radar parameters get saved into these files.

I have noticed the following inconsistencies when using some of the Py-ART algorithms. As an example, consider reading a C-SAPR file with radar = read_mdv()

1. All fields in the radar object are dictionaries and each field has a _FillValue key, and the data arrays are all NumPy masked arrays with fill_value = -9999.0
2. Running vr_corr = dealias_fourdd() outputs the dictionary vr_corr which is missing a _FillValue key, but its data is still a NumPy masked array with fill_value = -9999.0
3. Running phidp_corr, kdp_corr = phase_proc_lp() outputs the two dictionaries phidp_corr and kdp_corr. The data in these dictionaries are NumPy arrays, not NumPy masked arrays. However each dictionary does have a _FillValue key which is set to -9999.0
4. Running spec_at, ze_corr = calculate_attenuation() outputs the two dictionaries spec_at and ze_corr. The spec_at dictionary doesn't have a _FillValue key, its data is a NumPy array not a masked array, its data has NaN in it, and there are no -9999.0 values in it. The ze_corr dictionary does have a _FillValue key set to -9999.0, its data is a NumPy masked array with fill_value = -9999.0, but its data also has NaN in it.

Hello everyone,

I am pretty new to python and have run into an error when trying to plot ppi's by first initializing a figure with matplotlib. The error I receive is:

Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/matplotlib/backends/backend_qt4.py", line 261, in idle_draw
self.draw()
File "/usr/local/lib/python2.6/dist-packages/matplotlib/backends/backend_qt4agg.py", line 131, in draw
self.update()
RuntimeError: underlying C/C++ object has been deleted

I only get this error when trying to run remotely on a linux machine. The code runs perfectly on windows.

After looking into this problem for awhile, I found that this has something to do with me running the script using Spyder. When I try to run the code remotely using Spyder, the Agg backend of the matplotlib environment.
See: http://matplotlib.org/faq/howto_faq.html#howto-webapp

Thus, I have tried using the fix suggested in the link above but couldn't get it to work. I kept receiving a message saying the backend was already set even if it was the first thing I did.

Then, I tried using what is suggested here: https://groups.google.com/forum/#!topic/spyderlib/uFfi1rqhMKw
However, is this a viable solution when trying to plot a ppi? Can pyart utilize API's to avoid the above problem?

Any help on this issue would be greatly appreciated! Thanks!

Below is the code I am trying to run:

Load relevant modules

import pyart
from matplotlib import pyplot as plt
import numpy as np

def ppiplotter(filename, data_dir, save_dir):

# Concatinate togather filename and data directory to make full path.
data_dir = data_dir + '/'
fqfn = data_dir + filename

save_dir = save_dir + '/'

# Create radar object. If fails, print command read.
try:
    radar = pyart.io.read_cfradial(fqfn)
except:
    print "Could not read file", filename

 # Make radar display
try:    
    display = pyart.graph.RadarDisplay(radar)
except:
    print "Error in 'display = pyart.graph.RadarDisplay(radar)' for", filename

# Specify fields, colormap ranges, colormap, title location, colorbar labels    
fields = ['DBZ_F', 'VEL_F', 'WIDTH_F', 'ZDR_F']
ranges = [(-10,65), (-30.0, 30.0), (0., 8.0), (-2.0, 6.0)]
cmaps = ['spectral', 'spring', 'cool' , 'jet']
title_flags = [True, False, False, False]
colorbar_labels = ['DBZ F (dBZ)', 'VEL F (m/s)', 'WIDTH F (m/s)', 'ZDR F (dB)']

for sweepnum in range(0, np.size(radar.sweep_number['data'])):
    # Initiate plot and specify size
    fig = plt.figure(figsize = [14, 12])     

    #try:
    for i in range(len(fields)):
        field = fields[i]
        cmap = cmaps[i]
        title_flag = title_flags[i]
        colorbar_label = colorbar_labels[i]
        vmin, vmax = ranges[i]
        plt.subplot(2,2,i+1, adjustable='box', aspect=1.0)
        fig.subplots_adjust(hspace=.5)
        display.plot_ppi(field, sweepnum, vmin = vmin, vmax = vmax, title_flag = title_flag, cmap = cmap, axislabels = (None, "N-S distance (km)"), colorbar_label = colorbar_label)
    #except IndexError:
       # print 'IndexError: index out of bounds. Unexpected number in sweep in plot_radar for', filename
    #except:
       # print 'Unexpected error in surplotter.py for', filename
    #else:

    # Close plot and save.
    plt.close()
    save_name = "%s%s.%d.png" % (save_dir, filename, sweepnum)
    fig.savefig(save_name)

    print ("%s %d" % ('Sweep number', sweepnum))

The function pyart.io.radar.join_radar is broken and does not work as intended. The function also needs to be documented and added to the pyart.io namespace.

Attempting to use read_sigmet and it seems to fail on this file:
https://www.dropbox.com/s/zbt6vkwe8brexs6/np1130502192951.RAWSN3M

It is a birdbath scan generated with the IRIS software. RSL reads it in without issue.

The tb is:

KeyError Traceback (most recent call last)
in ()
1 filename = '/space/Data/ifloods/np1130502192951.RAWSN3M'
----> 2 radar = pyart.io.read_sigmet(filename)

/usr/lib64/python2.7/site-packages/pyart/io/sigmet.pyc in read_sigmet(filename, field_names, field_metadata, sigmet_field_names, time_ordered, debug)
68
69 # open the file, read data
---> 70 sigmetfile = SigmetFile(filename, debug=debug)
71 sigmet_data, sigmet_metadata = sigmetfile.read_data()
72 ingest_config = sigmetfile.ingest_header['ingest_configuration']

/usr/lib64/python2.7/site-packages/pyart/io/_sigmetfile.so in pyart.io._sigmetfile.SigmetFile.init (pyart/io/_sigmetfile.c:2814)()

KeyError: 62

Add a reader for the ODIM_H5 format. Daniel Michelson can provide sample files if needed.

As suggested by @kirknorth this is a feature request to add a parameter to the dealias_fourdd function that will replace gates flagged as "bad" by the FourDD algorithm with the original velocities. Kirk suggested "keep_original" as the parameter name.

Suggestion to add a get_sweep and put_sweep method to the Radar object which would extract or store a given sweep for a field. This could also take care of masking or removing locations where the antenna_transition indicate that the radar was in transition. Downstream functions (plotting, correcting) should then be updated to use this functionality.

• get_sweep (implemented via the extract_sweeps)
• put_sweep (implemented via join_radars)

Hi, I'm trying to convert the SIGMET data to netCDF and have not had success. I work with two SIGMET radars having different versions of IRIS software. In both cases the time attribute 'units' does not match time in time_coverage_start variable and sweep variable 'fixed_angle' has incorrect type: float64.
The file .nc is generated, but when opening or using the ferret or ncview the field variables do not open properly.
I'm using attached files to tests.
XXX140311182159.RAWZ2D2 is a file of s-band radar with IRIS version 8.12.3
IAC140311172529.RAWZEEJ is a file of x-band radar with IRIS version 8.12.1.1

https://drive.google.com/folderview?id=0B9evx7LuFEiSNl9DM2VKN3Z2eFU&usp=sharing

I'm not sure if I'm missing something with the process of correcting for attenuation in rain, but should I be expecting the corrected reflectivity field to have any gates with lower reflectivity than the input reflectivity field? Because I certainly do, and pretty much for every gate.

FYI these results are from the C-SAPR MDV file 092825.mdv.

The first image is the two-panel comparison of raw versus corrected. Just with your eye you should be able to see that the corrected field is lower in most regions.

The second image is the difference field, in this case Z_corr - Z_raw. Note the negative color scale limits. The mean of this difference field for the first elevation (0.5 deg) was about -5 dBZ.

These results were produced with the following,

phidp_corr, kdp_corr = phase_proc_lp(radar, -2.0, self_const=60000.0, low_z=10.0,
                                     high_z=53.0, min_phidp=0.01, min_ncp=0.5,
                                     min_rhv=0.8, fzl=3900.0, overide_sys_phase=False,
                                     nowrap=291, LP_solver='cylp_mp', window_len=35,
                                     proc=6, debug=False, really_verbose=False)

spec_at, ze_corr = calculate_attenuation(radar, -2.0, doc=15, fzl=3900.0, rhv_min=0.8, 
                                         ncp_min=0.5, a_coef=0.06, beta=0.8)

I don't think the radius of influence function in grid_from_radars, both the default when qrf_func = None or a user-defined one with qrf_func = f(x,y,z) is working properly. The radius of influence should be relative to the radar in question, i.e. it should be small close to the radar and grow bigger the further you get from the radar in order to capture the discrete sampling of the radar. However this doesn't appear to be the case.

In this example case I want to map C-SAPR data to a 100 x 100 x 10 km domain centered at the SGP CF, which has coordinates 36.605 N and 97.485 W, which is what I set the origin parameter to in grid_from_radars. Plotting the output radius of influence field shows this field is actually centered at the CF, not the radar (black crosshairs). This caught my attention because I noticed the detail in the C-SAPR reflectivity field I was gridding was quite high near the CF, and got smoother as you moved away from the CF. But the detail should be maximized closer to the radar, not the origin which you picked for your domain.

Seems to be an issue with the calendar fix

TypeError Traceback (most recent call last)
in ()
----> 1 myradar=pyart.io.read_netcdf('/data/tmp/'+outfile1)

/Users/scollis/Library/Python/2.7/lib/python/site-packages/pyart/io/netcdf.pyc in read_netcdf(filename)
133 return Radar(nsweeps, nrays, ngates, scan_type, naz, nele, _range,
134 azimuth, elevation, tu, cal, time, fields, sweep_info,
--> 135 sweep_mode, sweep_number, location, inst_params, metadata)
136
137

/Users/scollis/Library/Python/2.7/lib/python/site-packages/pyart/io/radar.pyc in init(self, time, _range, fields, metadata, scan_type, latitude, longitude, altitude, sweep_number, sweep_mode, fixed_angle, sweep_start_ray_index, sweep_end_ray_index, azimuth, elevation, altitude_agl, target_scan_rate, scan_rate, antenna_transition, instrument_parameters, radar_calibration)
119 ):
120
--> 121 if 'calendar' not in time:
122 time['calendar'] = 'gregorian'
123 self.time = time

TypeError: argument of type 'int' is not iterable

This enhancement came from an discussion with Kalle Nordling.

Being able to select out a one or more sweeps from a Radar object which would be used to build a new Radar object (to be used for gridding, etc) would be very helpful. Along these lines other methods for joining or splitting Radar objects would he useful.

The pyart.correct.dealias_fourdd function fails when the number of rays per sweep is not uniform. It should be determined if this is a requirement of the FourDD algorithm and if so a check should be added to the function and an informative message provided.

I'm not 100% sure and haven't had time to dig into the C code yet, but I think the read_rsl code leaves a reference to the file handle open. I have some code that was looping through 4000+ radar files, and after a couple hundred of them, python crashes with an error about too many file handles open.
I may be wrong but this is here as a placeholder until I can figure out what is going wrong with it.

Hey @jjhelmus,
I played with building maps configurable by json setup files.

Do you think this belongs in examples or as part of the pyart.graph module?

Not quite done yet but take a look
http://nbviewer.ipython.org/github/scollis/notebooks/blob/master/json_map.ipynb

Traceback (most recent call last):
File "./process_and_save_xsapr_2.py", line 128, in
pyart.io.write_netcdf(ofilename, myradar, format='NETCDF4')
File "/home/sc8/.local/lib/python2.6/site-packages/pyart/io/netcdf.py", line 200, in write_netcdf
dataset.setncatts(radar.metadata)
File "netCDF4.pyx", line 1719, in netCDF4.Dataset.getattr (netCDF4.c:18117)
File "netCDF4.pyx", line 1664, in netCDF4.Dataset.getncattr (netCDF4.c:17436)
File "netCDF4.pyx", line 895, in netCDF4._get_att (netCDF4.c:10492)
AttributeError: NetCDF: Attribute not found

So, as far as I can tell XSAPR-SW fails more than SE.. I wonder if it has to do with fill values getting through (ie -9999s, different to RSL's fill value... Would be good to document the climatology of when it fails...

The map_to_grid function does not provide a method for specifying the starting altitude of a grid. The default is to set it to the altitude of the first radar which can then be adjusted by setting the limits of the z grid_limits. This is far from ideal.

Noted NEXRAD is available on OpenDAP via a thredds server..
http://nbviewer.ipython.org/urls/raw.github.com/scollis/tds-python-workshop/master/radar_level2.ipynb

A feature request for the future will be an ingest to a radar object from this opendap object via netCDF4

Bug might be a bit too harsh, but it should be noted that when using netCDF4.Dataset to read in data arrays, the array will not be a numpy.ma.core.MaskedArray when none of the values in the array are equal to the fill value, even if the variable has a _FillValue key (or equivalent) . Instead it will be read as a numpy.ndarray. This can cause issues when scripts depend on accessing the mask attribute of NumPy masked arrays.

Should we make sure there is consistency in the 'data' keys of Radar and Grid objects? Could use the isinstace() built-in to check the type, and change accordingly.

Would it be possible to add an exclude or include kwarg (preferably exclude) to some of the IO modules, like read_mdv() and read_sigmet()? What I mean by this is if the user wants to exclude the differential phase field from their radar object when reading X-SAPR data, they could do something like radar = read_sigmet(fname, exclude=['diff_reflectivity']). The opposite would be true if it's an include kwarg, i.e. only read in the specified fields. I've seen the exclude kwarg in the netCDF4 module, and found it handy when wanting to save on memory consumption.

@scollis, are these standards still relevant for VAPs?

When attempting to use the rsl.read_rsl function, the file listed below causes the following errors(The error is reproducible just using rsl.read_rsl) . This is the file from the ARM archives if you wish to check it on your system.

./two_panel_plot.py XSE110425020046.RAWUBT3
plotting XSE110425020046.RAWUBT3 to test.png

gzip: stdout: Broken pipe

gzip: stdout: Broken pipe
[1] 26095 abort ./two_panel_plot.py XSE110425020046.RAWUBT3

The class collections.OrderedDict is, from my understanding, available from Python 2.7.x, but the install says Python 2.6.x is fine in terms of dependencies. Since I'm running Python 2.6.6, I get an ImportError on OrderedDict, namely from sigmet.py.

gzip: stdout: Broken pipe
Traceback (most recent call last):
File "./process_and_save_xsapr_2.py", line 69, in
myradar = pyart.io.read_rsl(filename)
File "/home/sc8/.local/lib/python2.6/site-packages/pyart/io/rsl.py", line 211, in read_rsl
fielddict['_FillValue'] = _FillValue
NameError: global name '_FillValue' is not defined

Somewhat of an amateur question but when my system administrator installed CyLP, it was done such that the module is imported like import CyLP. However in phase_proc.py it is imported using all lower case letters, so I get an ImportError... How can I fix this?

Origin in grid objects should be 1D arrays not zero D.. This allows moving platforms..

This enhancement was suggested by Kalle Nordling

Currently the gridding routine in Py-ART required a reflectivity field in all of the radar objects which is used to filter the gates to include in the interpolation based of masking and optionally value. It would be nice if gridding could be possible without a reflectivity field filter which include all gates.

Suggested implementation would be to use a refl_filter_flag parameter to the map_to_grid function.

Processing C-band data from 24-May-2011 revealed a new artifact I had not seen before when performing phase processing and/or attenuation correction. I'll provide code snippets of how I created this later. In the meantime check out the image below. The thin slice in the corrected reflectivity is the artifact I'm referring too, and those values are the missing value -9999.0. I was using min_ncp = 0.3 and min_rhv = 0.7, and using the default values (min_ncp=0.5 and min_rhv=0.8) produces more of these artifacts.

I'm not sure if this is something weird with my data, but when plotting RHI's I get a plot that is backward from what people normally use for rhi's. The normal convention is the radar is at 0, distance goes out to positive numbers towards the right. I don't remember seeing this issue before so it may just be the data.
If this is indeed the format pyart uses, would it be possible to get an optional argument to switch to the normal layout. I'm attaching a plot. I checked and teh elevations are correct(0, to 40'ish). Code to reproduce:

import pyart
radar =pyart.io.read_sigmet(filename)
radar.elevation['data'][radar.elevation['data']>80]=0 #Fix some Iris weirdness
display = pyart.graph.RadarDisplay(radar)
display.plot_rhi('reflectivity',3,vmin=0,vmax=60)

You can find the data file at https://www.dropbox.com/s/t0nw10izn2br9en/np1130504050139.RAW7WPK

If rsl-read_rsl() gets a nonexisting file, it manages to segfault python and python on my machine(OS X ML Py 2.7.4, newest from home-brew).

radar_object = rsl.read_rsl('XSE110425020046.RAWUBT3')
XSE110425020046.RAWUBT3: No such file or directory
Unknown input file type. File <XSE110425020046.RAWUBT3> is not recognized by RSL.
[1] 26052 segmentation fault ipython

I'm having issues when trying to display grid objects using the GridMapDisplay class. Here's some code and the output PNG of it,

fig = plt.figure(figsize=(8,6))
ax = fig.add_subplot(111)
display = GridMapDisplay(grid)
display.plot_basemap(lat_lines=None, lon_lines=None, resolution='l',
                     auto_range=True, ax=ax)
display.plot_grid('corrected_reflectivity_horizontal', 0, vmin=-8, vmax=64)
display.plot_colorbar(orientation='horizontal', mappable=display.mappables[0], cax=ax,
                      label='(dBZ)')
fig.savefig('graph_grid_test.png', format='png', dpi=100)

G'day team,
As many of you know DoE ARM has authorized work on CMAC 2.0 outlined in ECO 01077
(if you do not know what this is, don't worry!)

The synopsis is this: Before each algorithm made up its own mind where it needed to run based on things like NCP, Z, RHV etc.. We are now moving to ID-ing each gate and having algorithms in pyart.correct use this ID instead. I will be adding a module to pyart.retrieve which will be one of the first things run in an application chain.. Of course this will be a feature add, not a reconfig all modules will still have the ability (default?) to override the gate_id field based conditional application.

So my question to the Py-ART userbase is this:
what is needed in gate_id?
Proposal:
This will be a regular field dictionary in radar.fields
Key will always be 'gate_id'
shape of (nrays, ngates)
type of int
units: unitless
translation (to be stored under 'comment'):
0: Non-significant, noise floor
1: Clutter
2: Multiple trip
3: melting layer
4: hail
5: returns above the melting layer (ice, graupel, hail)
6: liquid precipitation

As I see it the above categories should be sufficient to allow conditional application of:
Dealiasing
LP phase processing
KDP calculation
attenuation correction
RHV correction
rainfall rate retrieval

please give feedback as we will be encouraging all code submitted that does conditional application to look for a gate_id field first before accepting a PR and we do not want to go changing the categories..

Or any "out of the box" thinking welcome.. note: I have been non-prescriptive about how this is calculated.. this is a question for another time.

Under install instructions on the master github.io page https://github.com/ARM-DOE/pyart/blob/master/INSTALL.TXT

The images below show some poor velocity unfolding behaviour. @scollis, I'm not sure if this was the same problem you were running into with the X-bands. Note how this is for the C-band.

From my quick investigation, there does appear to be some relation between areas of beam blockage and/or extinction and poor velocity correction. I'm also showing the uncorrected velocity field for reference.

In terms of the C-SAPR files that I processed for the May 20th, 2011 case, this issue was quite prevalent. In the 69 files that I processed between 0500 and 1300 UTC, about 23% had these artifacts.

So I have a few different file formats I was going to add to pyart. They are mostly based around netcdf, but each with it's own 'flavor'. I was thinking of just adding an aux_readers submodule to the io directory to put these in as they are probably less common and usually fairly radar specific. In particular I was looking to add the D3R netcdf versions(multiple), casa netcdf, and CHILL X-Band netcdf versions.

Would this be the preferred way, do you have a different preferred way, or would you rather this specific of reader not be included?

@jjhelmus and @scollis, I have a new mapping routine on a test branch. It makes use of SciPy's cKDTree object, but unlike the current implementation, it does not explicitly loop over the entire analysis grid, and therefore it is quite fast. For example, I can map every field (15 total) found in a CSAPR surveillance file with ~ 6 million gates onto a Cartesian domain with ~ 2 million grid points in ~ 1 minute using a Cressman distance-dependent weighting technique. This routine also has the ability to do a nearest neighbor technique, which for the same inputs mentioned previously, takes ~ 30 seconds.

Another enhancement is that MaskedArrays are handled quite nicely and every radar field's masked data is handled independently, therefore a refl_filter_flag is not required to remove potentially bad gates (assuming input data is good). What this means is fields like normalized coherent power or cross correlation ratio can be mapped everywhere, as they should be (in my opinion). See below for a couple examples.

Normalized coherent power

Reflectivity factor

Is anyone aware of an inability of netCDF4 to properly write large files (e.g. writing a large radar object) when format = 'NETCDF4'? Basically I was able to write an X-band file when format = 'NETCDF3_CLASSIC', and read this file after. However, when format = 'NETCDF4', calling pyart.io.write_cfradial finished, but I was getting a segmentation fault when trying to read it after. This file was 200-300 MB in size.

Currently the LP method uses Z to keep KDP high when in regions of High Z. At X-Band where we can get phase shift on backscatter that is of greater width than the (5pnt) filter this can lead to the phase taking the "High Path" At the moment in the "B Matrix" we only a constraint for Z > an amount to force higher dPhidp/dr we also need to force a lower dPhidp/dr in low Z (say Z lower than 25dBz)

I have just found a very subtle issue with our display module. When calling pcolormesh you can with the edges of the patches so when we call with the same length azimuth array (PPI) or elevation array (RHI) it misses the last ray.. no biggie for a 360 PPI as this just ends up with a cheese wedge at the last ray..
when you do a 10 elevation synthetic RHI (under development) this looses that last elevation and is very noticeable.. This is going to take a bit of a rewrite and experiment with the radardisplay module.. Documentation to follow later..

I noticed that grid_from_radars function treats velocity field as same as reflectivity field. However, it possibly tends to be incorrect because velocity is different from reflectivity, is vector not scale. The way to calculate true velocity using two Doppler radars is described in book of "Doppler radar and weather observation", Chapter 9. Generally speaking, if we have a grid observed by two radars, we can use following equation to calculate velocity vector (u,v), where \theta is elevation and \phi azimuth and w_p is vertical speed (I usually assign it to zero if I cannot obtain them).