Implement xray objects in place of numpy arrays about aospy HOT 4 CLOSED

I have recently discovered the xray package, which provides N-dimensional arrays (like numpy) but with named dimensions (like netCDF4 or pandas. It looks stable, well supported, and well documented. So I have decided to implement xray rather than create my own datatype(s) for this purpose.

from aospy.

I have begun to experiment with implementing xray support on a new branch [python3+xray]. At the moment it is fairly disjoint from the current develop branch (in that I have also migrated the code to python 3); there are also a lot of additions, many of which likely require some review and refactoring. That said it is functioning for my purposes at the moment.

I preserve all the code that works using the python-netCDF4 library to read in files, and numpy arrays for all computations. To account for this, I require the user to specify within a Run object how they would like things to be handled for that run with a new keyword argument (they can choose from either the read_mode='netcdf4' or read_mode='xray' tracks). There are basically just a number of if-statements within the code that depend on this argument, which keep the tracks separate; I feel like there must be a cleaner way, but for now it is quick and easy. With these modifications I can write calculation functions which take in DataArrays as arguments and return DataArrays as results. These results can then be saved directly as netCDF files rather than pickle files.

I apologize for taking a big leap here, but I wanted to be able to write calculation functions using DataArrays rather than numpy arrays from the start as I begin to develop my own library of functions (and not have to port them at some later time). I'm guessing that for the master version of aospy, we'll want to think about these issues a bit more carefully, but hopefully this is a decent start.

from aospy.

No apology necessary -- by all means, keep it coming. Getting both Python 3 support and xray implemented is huge. Let's meet in person to discuss the changes; want to make sure I understand it all. (Will email separately about that.)

I am inclined towards ultimately using xray exclusively, at least internally. Perhaps there is a clean way -- i.e. using decorators, isinstance, and/or try/except clauses -- to automatically detect which datatype a particular user-specified function is expecting: if it is numpy arrays, then generate an xray object containing that array. (Not actually sure that would work -- haven't looked at the code in a while)

But you're right that, in the near term, we need to still support the existing functions. How most elegantly to support both, we should give some thought.

from aospy.

As of the most recent commit (3ebb63f), aospy uses xray almost exclusively within itself, accomplished by incorporating the code contributed by @spencerkclark.

The issue of converting functions that are used to compute physical quantities from handling numpy to xray objects is a separate one.

from aospy.