Giter Club home page Giter Club logo

cdms's Introduction

cdat

⚠️ WARNING: Maintenance-only mode until around the end of 2023.
The CDAT library is now in maintenance-only mode, with plans for deprecation and cease of support around the end of calendar year 2023. Until this time, the dependencies for specific CDAT packages (cdms2, cdat_info, cdutil, cdtime, genutil, libcdms) will be monitored to ensure they build and install in Conda environments. We currently support Python versions 3.7, 3.8, 3.9, and 3.10. Unfortunately, feature requests and bug fixes will no longer be addressed.
If you are interested in an alternative solution, please check out the xarray and xCDAT - Xarray Extended With Climate Data Analysis Tools projects.

build status stable version platforms DOI

Anaconda-Server Badge Anaconda-Server Badge

CDAT builds on the following key technologies:

  1. Python and its ecosystem (e.g. NumPy, Matplotlib);
  2. Jupyter Notebooks and iPython;
  3. A toolset developed at LLNL for the analysis, visualization, and management of large-scale distributed climate data;
  4. VTK, the Visualization Toolkit, which is open source software for manipulating and displaying scientific data.

These combined tools, along with others such as the R open-source statistical analysis and plotting software and custom packages (e.g. DV3D), form CDAT and provide a synergistic approach to climate modeling, allowing researchers to advance scientific visualization of large-scale climate data sets. The CDAT framework couples powerful software infrastructures through two primary means:

  1. Tightly coupled integration of the CDAT Core with the VTK infrastructure to provide high-performance, parallel-streaming data analysis and visualization of massive climate-data sets (other tighly coupled tools include VCS, DV3D, and ESMF/ESMP);
  2. Loosely coupled integration to provide the flexibility of using tools quickly in the infrastructure such as ViSUS or R for data analysis and visualization as well as to apply customized data analysis applications within an integrated environment.

Within both paradigms, CDAT will provide data-provenance capture and mechanisms to support data analysis.

CDAT is licensed under the [BSD-3][bds3] license.

We'd love to get contributions from you! Please take a look at the Contribution Documents to see how to get your changes merged in.

cdms's People

Contributors

aashish24 avatar chaosphere2112 avatar danlipsa avatar davis278 avatar dnadeau4 avatar doutriaux1 avatar downiec avatar durack1 avatar eric-wieser avatar forsyth2 avatar jasonb5 avatar leung25 avatar linamuryanto avatar mcenerney1 avatar muryanto1 avatar painter1 avatar remram44 avatar stefraynaud avatar stephenpascoe avatar tomvothecoder avatar xylar avatar zshaheen avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

macweather zshaheen dnadeau4 scottwittenburg james-crean mauzey1 forsyth2 xylar xunius jasonb5

cdms's Issues

File open fails when lon and lat are variables (not dimensions)

The attached file is unreadable in the current version of cdms2 (v2.8). I believe the issue is that the variables (not dimensions) lon and lat included in the file are assumed to be standard dimensions (which they aren't), and these assumptions lead to a read failure.

cdmsReadFail.zip

The file looks like:

ncdump -h 161122_RobertPincus_multiple_input4MIPs_radiation_RFMIP_UColorado-RFMIP-20161122_none.nc 
netcdf \161122_RobertPincus_multiple_input4MIPs_radiation_RFMIP_UColorado-RFMIP-20161122_none {
dimensions:
	expt = 18 ;
	level = 61 ;
	layer = 60 ;
	site = 100 ;
variables:
	float lon(site) ;
		lon:long_name = "ERA-Interim longitude" ;
		lon:units = "degree_north" ;
		lon:standard_name = "longitude" ;
	float lat(site) ;
		lat:long_name = "ERA-Interim latitude" ;
		lat:units = "degree_east" ;
		lat:standard_name = "latitude" ;
	float time(site) ;
		time:long_name = "ERA-Interim fractional day of the year 2014" ;
		time:units = "days since 2014-1-1 0:0:0" ;
		time:standard_name = "time" ;
		time:calendar = "gregorian" ;
	float sst(site) ;
		sst:title = "sea surface temperature" ;
		sst:units = "K" ;
		sst:long_name = "ERA-Interim sea surface temperature (= \"missing_value\" over land)" ;
		sst:standard_name = "sea_surface_temperature" ;
		sst:missing_value = -9.99f ;
		sst:FillValue = -9.99f ;
		sst:coordinates = "lon lat time" ;
...

And the read failure looks like:

In [1]: infile = '161122_RobertPincus_multiple_input4MIPs_radiation_RFMIP_UColorado-RFMIP-20161122_none.nc'
In [2]: import cdms2 as cdm
In [3]: f = cdm.open(infile)
---------------------------------------------------------------------------
CDMSError                                 Traceback (most recent call last)
<ipython-input-3-da3bd29babf2> in <module>()
----> 1 f = cdm.open(infile)
/uvcdat280/lib/python2.7/site-packages/cdms2/dataset.pyc in openDataset(uri, mode, template, dods, dpath, hostObj)
    357 
    358             # The file exists
--> 359             file1 = CdmsFile(path, "r")
    360             if libcf is not None:
    361                 if hasattr(file1, libcf.CF_FILETYPE):
/uvcdat280/lib/python2.7/site-packages/cdms2/dataset.pyc in __init__(self, path, mode, hostObj, mpiBarrier)
   1137                 # Get grid information for the variable. gridkey has the form
   1138                 # (latname,lonname,order,maskname, abstract_class).
-> 1139                 gridkey, lat, lon = var.generateGridkey(self._convention_, self.variables)
   1140 
   1141                 # If the variable is gridded, lookup the grid. If no such grid exists,
/uvcdat280/lib/python2.7/site-packages/cdms2/avariable.pyc in generateGridkey(self, convention, vardict)
    239         lon, nlon = convention.getVarLonId(self, vardict)
    240         if (lat is not None) and (lat is lon):
--> 241             raise CDMSError, "Axis %s is both a latitude and longitude axis! Check standard_name and/or axis attributes."%lat.id
    242 
    243         # Check for 2D grid
CDMSError: Axis lon is both a latitude and longitude axis! Check standard_name and/or axis attributes.

numpy.NewAxis broken

From @doutriaux1 on May 16, 2016 21:45

the follwoing used to work

f=cdms2.open("/usr/local/uvcdat/2.4.1/share/uvcdat/sample_data/clt.nc")
>>> s=f("clt",slice(0,1))
>>> s.shape
(1, 46, 72)
>>> s=f("clt",slice(0,1),squeeze=1)
>>> s.shape
(46, 72)
>>> import cdutil
>>> zm = cdutil.averager(s,axis="x")
>>> zm.shape
(46,)
>>> s2 = s -zm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda2/envs/uvcdat-2.4.1/lib/python2.7/site-packages/cdms2/avariable.py", line 1436, in __sub__
    return MV.subtract(self, other)
  File "/usr/local/anaconda2/envs/uvcdat-2.4.1/lib/python2.7/site-packages/cdms2/MV2.py", line 228, in __call__
    maresult = self.mafunc(ta,tb)
  File "/usr/local/anaconda2/envs/uvcdat-2.4.1/lib/python2.7/site-packages/numpy/ma/core.py", line 935, in __call__
    m = umath.logical_or(getmaskarray(a), mb)
ValueError: operands could not be broadcast together with shapes (46,72) (46,) 
>>> s2 = s -zm[:,numpy.newaxis]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda2/envs/uvcdat-2.4.1/lib/python2.7/site-packages/cdms2/avariable.py", line 1401, in __getitem__
    speclist = self._process_specs(key, {})
  File "/usr/local/anaconda2/envs/uvcdat-2.4.1/lib/python2.7/site-packages/cdms2/avariable.py", line 1169, in _process_specs
    raise CDMSError, 'Sorry, you cannot use NewAxis in this context ' + str(specs)
cdms2.error.CDMSError: Sorry, you cannot use NewAxis in this context (slice(None, None, None), None)
>>> s2 = s - zm[:,numpy.newaxis]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda2/envs/uvcdat-2.4.1/lib/python2.7/site-packages/cdms2/avariable.py", line 1401, in __getitem__
    speclist = self._process_specs(key, {})
  File "/usr/local/anaconda2/envs/uvcdat-2.4.1/lib/python2.7/site-packages/cdms2/avariable.py", line 1169, in _process_specs
    raise CDMSError, 'Sorry, you cannot use NewAxis in this context ' + str(specs)
cdms2.error.CDMSError: Sorry, you cannot use NewAxis in this context (slice(None, None, None), None)

Copied from original issue: CDAT/cdat#1974

cdms2.getGrid()/setGrid() doesn't work 

(Pdb) print s.getGrid()
<CurveGrid, id: grid_37, shape: (384, 320)>
(Pdb) heatContent.setGrid(s.getGrid())
*** CDMSError: grid does not match axes for variable variable_524300
(Pdb) heatContent.shape
(60, 384, 320)
(Pdb) print s.shape
(60, 384, 320)
(Pdb) print s.getGrid().shape
(384, 320)
(Pdb) print cdm.createGenericGrid(s.getLatitude(),s.getLongitude(),order='yx')
*** ValueError: could not broadcast input array from shape (384,320) into shape (384)
(Pdb) print cdm.createGenericGrid(s.getLatitude(),s.getLongitude(),order='xy')
*** ValueError: could not broadcast input array from shape (384,320) into shape (384)
(Pdb) print s.getGrid().clone()
<CurveGrid, id: grid_37, shape: (384, 320)>
(Pdb) heatContent.setGrid(s.getGrid().clone())
*** CDMSError: grid does not match axes for variable variable_524300

Migrated from: CDAT/cdat#1707

mean broken on MV array

From @doutriaux1 on August 15, 2016 15:32

In [1]: import MV2

In [2]: a=MV2.ones((13,14))

In [3]: a.mean()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-dc887f7a7bfb> in <module>()
----> 1 a.mean()

/export/doutriaux1/anaconda2/envs/blah/lib/python2.7/site-packages/numpy/ma/core.pyc in mean(self, axis, dtype, out)
   4779         """
   4780         if self._mask is nomask:
-> 4781             result = super(MaskedArray, self).mean(axis=axis, dtype=dtype)
   4782         else:
   4783             dsum = self.sum(axis=axis, dtype=dtype)

/export/doutriaux1/anaconda2/envs/blah/lib/python2.7/site-packages/numpy/core/_methods.pyc in _mean(a, axis, dtype, out, keepdims)
     67     if isinstance(ret, mu.ndarray):
     68         ret = um.true_divide(
---> 69                 ret, rcount, out=ret, casting='unsafe', subok=False)
     70     elif hasattr(ret, 'dtype'):
     71         ret = ret.dtype.type(ret / rcount)

/export/doutriaux1/anaconda2/envs/blah/lib/python2.7/site-packages/numpy/ma/core.pyc in __array_wrap__(self, obj, context)
   2897                 return masked
   2898             else:
-> 2899                 result._mask = m
   2900                 result._sharedmask = False
   2901         #....

AttributeError: 'numpy.ndarray' object has no attribute '_mask'

Copied from original issue: CDAT/cdat#2088

forecast test fails

see comment lines in test_forecast_io.py

line 69 try to assert time axis is time but not anymore

Consider creating MPAS convention for generic grids

MPAS files have a Conventions attribute with the value 'MPAS'; we could use this to generate a new Convention object in CDMS that uses the appropriate variables/axes in the file to construct the correct grid for the variables.

This is the algorithm I used to construct the generic grids for the data; it creates meshfill-able data.
I'll post a link to a file to use as a test, if it's decided that we should support this.

import cdms2
import numpy

f = cdms2.open("average.nc")
cell_x = f("xCell")
cell_y = f("yCell")

vert_ids = f("indexToVertexID")
cell_verts = f("verticesOnCell")
vert_x = f("xVertex")
vert_y = f("yVertex")

# This can be improved dramatically by someone better at numpy than me
def extract_cell_bounds(cells, vertex_coord):
    cell_bounds = numpy.zeros(cells.shape)
    for i in range(len(cells)):
        cell_bounds[i] = numpy.take(vertex_coord, cells[i] - 1)
    return cell_bounds

xbounds = extract_cell_bounds(cell_verts, vert_x)
ybounds = extract_cell_bounds(cell_verts, vert_y)

virt_axis = f.getAxis("nCells")
yaxis = cdms2.auxcoord.TransientAuxAxis1D(cell_y, axes=(virt_axis,), bounds=ybounds, id="latitude")
xaxis = cdms2.auxcoord.TransientAuxAxis1D(cell_x, axes=(virt_axis,), bounds=xbounds, id="longitude")
generic_grid = cdms2.gengrid.TransientGenericGrid(yaxis, xaxis, id="mpas_grid")

Obtain new CMIP6 grids and add to data test suite

It occurred to me that there are likely a bunch of new (weird) grids in the pipeline for CMIP6 (particularly the ocean data) and consequently it would be a good idea to get access to this data and add these grids to the current UV-CDAT devel test suite, and additionally feed these new specs/files through to the ESMF folks so that their software updates can deal with the new formats.

If this did work, then it would mean that UV-CDAT 2.x/3.x would automagically be able to happily regrid and read all the as yet unreleased data formats.. And we'd certainly be ahead of the game when the CMIP6 data starts appearing on ESGF rather than playing catchup as was done with CMIP5..

@doutriaux1 @aashish24 I'm happy for you folks to assign this to me, and some future milestone.. Maybe UV-CDAT 2.5/3.x?

@williams13 - not sure what discussions you've been having with Balaji about the gridspec stuff, but I think we know enough now to start thinking about this..

Migrated from: CDAT/cdat#819

XQuartz version and numpy?

  1. VCS not opening xwindow for Mac terminal when XQuartz 2.7.9 or higher version.
  2. Loading variable returns error message which I've not seen before.
>>> import vcs, cdms2
>>> f=cdms2.open('clt.nc')
>>> d=f('clt')

/export_backup/lee1043/anaconda2/lib/python2.7/site-packages/numpy/ma/core.py:2761: DeprecationWarning: Non-string object detected for the array ordering. Please pass in 'C', 'F', 'A', or 'K' instead
order=order, subok=True, ndmin=ndmin)

>>> x=vcs.init()
>>> x.plot(d)

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
X Error of failed request: BadValue (integer parameter out of range for operation)
Major opcode of failed request: 149 (GLX)
Minor opcode of failed request: 3 (X_GLXCreateContext)
Value in failed request: 0x0
Serial number of failed request: 42
Current serial number in output stream: 43

Migrated from: CDAT/cdat#2132

cdms2.open fails on an xml file with "invalid" attributes

The following error output from ipython should explain the error enough. The .xml input file was generated using cdscan - same version as the uvcdat:

In [2]: fh = cdm.open('ORAS4_1958-2014.xml')
error near line 17: invalid attribute value positive=unknown of element axis, must be one of ('up', 'down')
---------------------------------------------------------------------------                                
AssertionError                            Traceback (most recent call last)                                
<ipython-input-2-c0e2c24bde85> in <module>()                                                               
----> 1 fh = cdm.open('ORAS4_1958-2014.xml')                                                               

/usr/local/uvcdat/2016-03-09/lib/python2.7/site-packages/cdms2/dataset.pyc in openDataset(uri, mode, template, dods, dpath, hostObj)
...

/usr/local/uvcdat/2016-03-09/lib/python2.7/site-packages/cdms2/cdmsNode.pyc in validate(self, idtable)
    359                 if type(atttype) is TupleType:
    360                     attval=self.getExternalAttr(attname)
--> 361                     assert attval in atttype, 'Invalid attribute %s=%s must be in %s'%(attname,attval,`atttype`)
    362                 elif atttype==CDML.Idref:
    363                     attval=self.getExternalAttr(attname)

AssertionError: Invalid attribute positive=unknown must be in ('up', 'down')

Migrated from: CDAT/cdat#1942

remove libcf?

I don't think anybody uses or trust the libcf regrid implementation, and it just adds a useless dependency to cdms2 (and it also adds a dependency to uuid)

CDMS2 fails during init for netCDF variable of type unsigned byte

If I attempt to open a dataset produced by cdscan from a group of netCDF files containing a variable of type unsigned byte, the cdms2.open() function fails with the exception "TypeError: 'Not implemented for this type'". The unsigned byte type is interpreted as character type and the values are converted to single-character strings. The exception is thrown at line 136 in the file avariable.py when numpy.isnan() is called with the value of the missing_value attribute for the variable, which has been converted into a string, as an argument. CDL for the variable in question is included below. This variable is not being used by the calling code, but it is processed by the cdms2.open() function because it is present in the files. The problem is not seen when opening an individual netCDF file from group (as opposed to opening the cdscan XML output file).

byte qa_of_seaice_conc_cdr(time=1, ygrid=448, xgrid=304);
  :_FillValue = 0B; // byte
  :valid_range = 1S, 255S; // short
  :_Unsigned = "true";
  :long_name = "Passive Microwave Daily Northern Hemisphere Sea Ice Concentration QC flags";
  :standard_name = "sea_ice_area_fraction status_flag";
  :coordinates = "longitude latitude time";
  :flag_masks = 1B, 2B, 4B, 8B, 32B, -128B; // byte
  :flag_meanings = "BT_source_for_CDR NT_source_for_CDR no_ice_allowed_per_climatology grid_cell_near_to_coast concentration_below_fifty_percent melt_start_detected";
  :missing_value = 0B; // byte
  :datum = "+ellps=urn:ogc:def:crs:EPSG::4326";
  :grid_mapping = "projection";

var.squeeze() truncates axis information

I don't think the behavior below is what a user expects - certainly not me:

so: (1, 27, 180, 360)
['time', 'LEVEL', 'LATITUDE', 'LONGITUDE']
so.squeeze()
so: (27, 180, 360)
['axis_0', 'axis_1', 'axis_2']

Migrated from: CDAT/cdat#1738

Revisit xmls - cdscan optimization and investigation of psql as alternative

As discussed earlier by @doutriaux1 @williams13 @painter1 there is a need to revisit cdscan as it's started to become difficult to maintain.

As noted in #124 additional to cdscan the psql package by Charlie O'Connor (https://github.com/UV-CDAT/uvcdat/tree/master/Packages/psql) may also be an approach to revisit when rethinking xmls.

@doutriaux1 @williams13 @painter1 - I think it's better to have a dedicated issue here, rather than tacking comments onto a bugfix issue for cdscan.

@doutriaux1 is 2.4 a likely milestone?

Migrated from: CDAT/cdat#1059

Building/testing modifies source tree

The following files are created in the source tree when building/testing UVCDAT:

installation/install.py
testing/cdms2/junk.xml
testing/dv3d/.baseline_path

I'm not sure what the purpose of the first two are, but the third contains a path to the baseline directory for the dv3d tests. I had added a mechanism to properly pass that path into dv3d via an argument[1], but it appears @ThomasMaxwell reverted my change at some point. This should be cleaned up -- Source clutter with build-dependent info will cause issues when multiple builds reference the same source tree.

[1] CDAT/cdat@65d545a#diff-d3e8161439180f671ad7a70585da5cc3

Migrated from: CDAT/cdat#1364

Meshfill plot with Mercator projection issue

This use case currently fails. Test case provided in 9d6bed6

Run ctest -V -R vcs_test_animate_projected_meshfill_mercator

Error Reported:

File "<uvcdat>/install/lib/python2.7/site-packages/cdms2/coord.py", line 289, in subSlice
newbounds = bounds[specs]   # bounds can be a numarray or DatasetVariable
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

Migrated from: CDAT/cdat#1671

cdms2: Error on file that works fine with other netcdf readers.

The netcdf file in g2_BIOPAR_SWI-TS_201001310000_C0243_ASCAT_V1.0.0.zip can not be opened using cdms2, but it works fine with netcdf4-python, panoply or gdal.

I first stumbled onto this error when using the cf-checker and have already reported the problem to @RosalynHatcher .

See this code snippet for an example and below the conda environment I used for testing this.

Python 2.7.11 |Continuum Analytics, Inc.| (default, Dec  6 2015, 18:08:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import netCDF4 as nc4
>>> import cdms2
>>> f = cdms2.open("g2_BIOPAR_SWI-TS_201001310000_C0243_ASCAT_V1.0.0.nc", "r")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cpa/.pyenv/versions/miniconda-3.16.0/envs/cdms-test/lib/python2.7/site-packages/cdms2/dataset.py", line 309, in openDataset
    file1 = CdmsFile(path,"r")
  File "/home/cpa/.pyenv/versions/miniconda-3.16.0/envs/cdms-test/lib/python2.7/site-packages/cdms2/dataset.py", line 1001, in __init__
    self.attributes[att]=self._file_.__dict__[att]
  File "/home/cpa/.pyenv/versions/miniconda-3.16.0/envs/cdms-test/lib/python2.7/site-packages/cdms2/cdmsobj.py", line 452, in _listatts
    for nm,val in self.__dict__.items():
ValueError: data type must provide an itemsize
>>> ds = nc4.Dataset("g2_BIOPAR_SWI-TS_201001310000_C0243_ASCAT_V1.0.0.nc")
>>> ds
<type 'netCDF4._netCDF4.Dataset'>
root group (NETCDF4 data model, file format UNDEFINED):
    producer: tuwien
    product_version: V1.0.0
    long_name: Soil Water Index
    processing_mode: Consolidated
    references: http://land.copernicus.eu/global/products/swi
    id: g2_BIOPAR_SWI-TS_201001310000_C0243_ASCAT_V1.0.0.nc
    geospatial_lat_max: 2.487
    copyright: Copyright Copernicus Global Land Service, 2015. Produced by Technische Universität Wien (TU Wien), Austria. Distributed by VITO NV, Belgium.
    title: Copernicus Global Land Service Soil Water Index Time Series (SWI_TS)
    platform: METOP
    archive_facility: VITO
    sensor: ASCAT
    processing_level: L3
    orbit_type: LEO
    featureType: timeSeries
    geospatial_lat_min: 0.0
    time_coverage_start: 2010-01-01T00:00:00
    institution: TU Wien
    geospatial_lon_max: -47.7229
    geospatial_lon_min: -49.9967
    parent_identifier: urn:cgls:tiles:swits_v1_12.5km
    time_coverage_end: 2010-01-31T00:00:00
    Conventions: CF-1.6
    source: ASCAT on board MetOP
    date_created: 2016-06-22 16:38:22
    identifier: urn:cgls:tiles:swits_v1_12.5km:SWI-TS_201001310000_C0243_ASCAT_V1.0.0
    dataset_name: SWI time series for cell 0243
    history: 2016-06-22 -- Product creation by swi_ts processor version 0.6.3.post.dev6.pre
    dimensions(sizes): locations(196), time(31)
    variables(dimensions): float64 lon(locations), float64 lat(locations), float64 alt(locations), int64 location_id(locations), <type 'str'> location_description(locations), float64 time(time), uint8 SWI_005(locations,time), uint8 SWI_010(locations,time), uint8 SWI_020(locations,time), uint8 SWI_001(locations,time), uint8 SWI_015(locations,time), uint8 QFLAG_040(locations,time), uint8 SWI_100(locations,time), uint8 QFLAG_010(locations,time), uint8 SWI_040(locations,time), uint8 QFLAG_015(locations,time), uint8 QFLAG_100(locations,time), uint8 SWI_060(locations,time), uint8 QFLAG_020(locations,time), uint8 QFLAG_005(locations,time), uint8 SSF(locations,time), uint8 QFLAG_001(locations,time), uint8 QFLAG_060(locations,time)
    groups: 

>>> ds.variables['SWI_060'][:]
masked_array(data =
 [[-- -- -- ..., -- -- --]
 [-- -- -- ..., -- -- --]
 [-- -- -- ..., -- -- --]
 ..., 
 [-- -- -- ..., -- -- --]
 [-- -- -- ..., -- -- --]
 [-- -- -- ..., -- -- --]],
             mask =
 [[ True  True  True ...,  True  True  True]
 [ True  True  True ...,  True  True  True]
 [ True  True  True ...,  True  True  True]
 ..., 
 [ True  True  True ...,  True  True  True]
 [ True  True  True ...,  True  True  True]
 [ True  True  True ...,  True  True  True]],
       fill_value = 255)

conda

cdat-info                 0.0.0                     <pip>
cdat_info                 2.4.1                    UVCDAT    uvcdat
cdms                      1.0                       <pip>
cdms2                     2.4.1                    UVCDAT    uvcdat
cdtime                    2.4.1                    UVCDAT    uvcdat
clapack                   3.2.1                    UVCDAT    uvcdat
curl                      7.49.0                        0    defaults
distarray                 2.4.1                    UVCDAT    uvcdat
esmf                      ESMF_6_3_0rp1_ESMP_01          UVCDAT    uvcdat
g2clib                    1.4.0b                   UVCDAT    uvcdat
hdf5                      1.8.16                        0    defaults
jasper                    1.900.1                  UVCDAT    uvcdat
jbig                      2.1                           0    defaults
jpeg                      8d                            0    defaults
lapack                    3.4.2                    UVCDAT    uvcdat
libcdms                   2.4.1                    UVCDAT    uvcdat
libcf                     1.0.beta11               UVCDAT    uvcdat
libgfortran               3.0.0                         1    defaults
libnetcdf                 4.3.3.1                  UVCDAT    uvcdat
libpng                    1.6.22                        0    defaults
libtiff                   4.0.6                         2    defaults
ma                        1.0                       <pip>
mv                        1.0                       <pip>
mv2                       1.0                       <pip>
netcdf4                   1.2.4                     <pip>
numpy                     1.9.2                    py27_2    defaults
openblas                  0.2.14                        4    defaults
openssl                   1.0.2h                        1    defaults
ossuuid                   1.6.2                    UVCDAT    uvcdat
pip                       8.1.2                    py27_0    defaults
python                    2.7.11                        0    defaults
readline                  6.2                           2    defaults
regrid                    1.0                       <pip>
regrid2                   2.4.1                    UVCDAT    uvcdat
setuptools                23.0.0                   py27_0    defaults
sqlite                    3.13.0                        0    defaults
tk                        8.5.18                        0    defaults
wheel                     0.29.0                   py27_0    defaults
xz                        5.2.2                         0    defaults
zlib                      1.2.8                         3    defaults

Migrated from: CDAT/cdat#2034

Subsetting cdms2 variables using arbitrary indices

It would be great to be able to subset a cdms2 transientVariable using arbitrary indices - similar to the numpy.take function but applicable to multiple dimensions concurrently, so the 14 or so lines below could be condensed into something much more compact:

climInfile = '/work/durack1/Shared/obs_data/WOA13/141027_woa/woa13_decav_t00_01.nc'
cF = cdm.open(climInfile)
climDepths = cF.getAxis('depth').getData()
anomDepths = uF.getAxis('level').getData()
commonLevels = np.intersect1d(climDepths,anomDepths)
clim = cF('t_an') ; #,depth=commonLevels) - it would be great to just load the indices I want!
inds = []
for x,val in enumerate(commonLevels):
    inds.append(np.where(climDepths==commonLevels[x])[0][0])
climTrim = np.take(clim,inds,axis=1)
climTrim = cdm.createVariable(climTrim,id='climTrim',missing_value=1e20)
climTrim.setAxis(0,clim.getAxis(0))
climTrim.setAxis(1,uF.getAxis('level'))
climTrim.setAxis(2,uF.getAxis('latitude'))
climTrim.setAxis(3,uF.getAxis('longitude'))
del(clim,inds,x,val,commonLevels,anomDepths,climDepths,climInfile) ; gc.collect()
cF.close()

In the case above, uF contains a subset of the depths found in the cF file which is loaded above.

Ideally it would be great if this worked not only in index space, but also in value/data space so for e.g. in the above target levels could be specified rather than the indices captured in the variable inds.

So for an example what I want is:

clim.shape
(1, 102, 180, 360)
climTrim = clim[:,[0,5,10,20,50,100],:,:] ; an example using index values,
or using the variable above
climTrim = clim[:,inds,:,:] ; a list (or equivalent type) variable that contains the indexes,
or alternatively the axis values (rather than index values):
depths = [0,5,10,20,50,100,values=True] in whatever native units the coordinate is stored

@doutriaux1 can you add the cdms2 and enhancement labels please - and a milestone linked to existing issue CDAT/cdat#885 and PR CDAT/cdat#915

Migrated from: CDAT/cdat#1288

openDataset() speedup

On a mis-configured machine such as rhea.ccs.ornl.gov where a file open operation is expensive, one unnecessary bottleneck is in openDataset(), in cdms2 dataset.py.

This function opens a file as 'file1', checks an attribute, then opens it again as 'file' and closes 'file1'. In some cases 'file' has been opened differently from 'file1', so this is the only option. But in most normal use, the file is opened the same way both times. So if a file open be expensive, you can save some time by checking the prospective arguments to the second call of CdmsFile (if there is one), and skipping it if they are the same as on the first call.

I have made and tested this change in a private copy of UV-CDAT. The performance improvement I have seen isn't as bit as for issues #1609, #1610, #1611. But it's worth the trouble.

Migrated from: CDAT/cdat#1612

Problems loading grib2 variables in cdms (working in GrADS)

Dear UV-CDAT Developers,

Greetings!

When I am trying to access ensemble dimension (which has embedded within variable as another dimension / axis / coordinates) from grib2 file, cdms2 just suppressed to single ensemble dimension (i.e. no ensemble dimension at all) .

But when I am accessing the same ctl file in grads, I am able to set / select which ensemble need to be plotted. In grads, we can do "set E 5" to select 5th ensemble.

Attached zip file which contains sample grib2, ctl, idx files ensemble-example-grib2.zip

$ wgrib2 -v example.grib2 shows the ensembles.

If you point out me, which module do I need to look out, then I may give a try to fix it.

Thanks.

Migrated from: CDAT/cdat#1823

Update cdms2 to deal with all common datatypes (uint8, uint16, char, str etc)

It appears that UV-CDAT 2.0 beta doesn't play nicely with the unsigned integer data types, and it would appear this is a hangover from numeric data types a long time ago:

Taking a look in https://github.com/UV-CDAT/cdms/blob/master/Lib/cdmsNode.py#L23-65 do not include these unsigned types:

# Data types

CdChar = CDML.CdChar
CdByte = CDML.CdByte
CdShort = CDML.CdShort
CdInt = CDML.CdInt
CdLong = CDML.CdLong
CdInt64 = CDML.CdInt64
CdFloat = CDML.CdFloat
CdDouble = CDML.CdDouble
CdString = CDML.CdString
CdFromObject = CDML.CdFromObject
CdAny = CDML.CdAny
CdDatatypes = [CdChar,CdByte,CdShort,CdInt,CdLong,CdInt64,CdFloat,CdDouble,CdString]

CdScalar = CDML.CdScalar
CdArray = CDML.CdArray

NumericToCdType = {numpy.sctype2char(numpy.float32):CdFloat,
                   numpy.sctype2char(numpy.float):CdDouble,
                   numpy.sctype2char(numpy.int16):CdShort,
                   numpy.sctype2char(numpy.int32):CdInt,
                   numpy.sctype2char(numpy.int):CdLong,
                   numpy.sctype2char(numpy.int64):CdInt64,
                   numpy.sctype2char(numpy.intc):CdLong,
                   numpy.sctype2char(numpy.int8):CdByte,
                   'c':CdChar,
                   'B':'B',
                   'H':'H',
                   'L':'L',
                   'q':CdInt64,
                   'Q':'Q',
                   'S':'S'
                   }

CdToNumericType = {CdChar:'c',
                   CdByte:numpy.int8,
                   CdShort:numpy.int16,
                   CdInt:numpy.int32,
                   CdLong:numpy.int,
                   CdInt64:numpy.int64,
                   CdFloat:numpy.float32,
                   CdDouble:numpy.float}

It would be great to include these, as netcdf 4 also includes these types as NC_ubyte (uint8; unsigned byte), NC_ushort (uint16; unsigned short), NC_uint (uint32; unsigned int), NC_uint64 (uint64; unsigned long long)

Migrated from: CDAT/cdat#537

Can't add missing value attribute to netcdf file variable

This code fails when writing a simple variable to a file:

import cdms2, MV2
f = cdms2.open(__file__[:-2]+'.nc', 'w')
f.write(MV2.ones(1))
f.close()

Output:

Traceback (most recent call last):
  File "bug_write_netcdf.py", line 6, in <module>
    f.write(MV2.ones(1))
  File "/home1/caparmor/sraynaud/soft/uvcdat/lib/python2.7/site-packages/cdms2/dataset.py", line 1728, in write
    id=varid, extend=extend, fill_value=fill_value, index=index)
  File "/home1/caparmor/sraynaud/soft/uvcdat/lib/python2.7/site-packages/cdms2/dataset.py", line 1661, in createVariableCopy
    setattr(newvar, attname, attval)
  File "/home1/caparmor/sraynaud/soft/uvcdat/lib/python2.7/site-packages/cdms2/fvariable.py", line 137, in __setattr__
    setattr(self._obj_, name, value)
ValueError: Invalid data-type for array

where the attribute name is "missing_value".

It fails only on a particular installation of uvcdat close to 2.2.0, that already worked in the past.
Among the things that changed since it worked is a combined update of matplotlib and numpy using pip -U. Maybe a solution would be an appropriate downgrade of numpy without breaking dependent packages.

Is numpy the problem and what is the maximal version of numpy this version of uvcdat/cdunif can support?

Note that I can't update my version of UVCDAT since the installation fails due to the problematic CRYPTOGRAPHY package whose installation fails for unkown reasons (already reported here).

Migrated from: CDAT/cdat#1853

ESMF regrid documentation

@pletzer, I was a happy user of the ESMF/cdms documentation that used to reside at https://ice.txcorp.com/trac/modave/wiki/esmfLinear, however when I went to check this link today it appears that the documentation and examples that used to reside here are now no longer available.

Is there a more persistent repository of this information somewhere - it would be great to pull this into the cdms documentation site if it exists..

cannot slice string variable from NetCDF file

I have a NetCDF file (ACME output) with a string-valued variable. With cdms2 I can read one character at a time, but no more. Thus I can make a FileVariable out of it but not a Transient Variable. For example:

>>> time_written = f['time_written']
>>> time_written.shape
(1, 8)
>>> time_written[0,1]
'1'
>>> time_written[0,2]
':'
>>> time_written[0,:2]
Traceback (most recent call last):
File "", line 1, in
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/variable.py", line 82, in getitem
return AbstractVariable.getitem(self, key)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/avariable.py", line 1399, in getitem
return self.getSlice(numericSqueeze=1, raw=0, isitem=1, _speclist)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/avariable.py", line 656, in getSlice
result = self.subSlice(_specs, **keys)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/avariable.py", line 609, in subSlice
id = self.id)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/tvariable.py", line 270, in new
fill_value = numpy.array(fill_value).astype(dtype),
ValueError: could not convert string to float: N/A

>>> f('time_written')
Traceback (most recent call last):
File "", line 1, in
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/cudsinterface.py", line 45, in call
return v(_args, *_kwargs)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/avariable.py", line 155, in call
grid=grid)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/selectors.py", line 195, in unmodified_select
raw=raw)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/avariable.py", line 759, in subRegion
return self.subSlice(_slicelist, *_d)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/avariable.py", line 609, in subSlice
id = self.id)
File "/Users/painter1/bin/uvcdat-2015.03.25/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cdms2/tvariable.py", line 270, in new
fill_value = numpy.array(fill_value).astype(dtype),
ValueError: could not convert string to float: N/A

Migrated from: CDAT/cdat#1333

Suppress numpy warnings?

I've just started seeing these warnings:

import cdms2 as cdm
import MV2 as MV
# Set nc classic as outputs
cdm.setNetcdfShuffleFlag(0)
cdm.setNetcdfDeflateFlag(0)
cdm.setNetcdfDeflateLevelFlag(0)

os.chdir('/work/durack1/Shared/131118_PaperPlots_SaltyVariability/151001_NOEMP02_IPSL-CM6')
domains = ['glo','atl','pac','ind']
for i in domains:
    var = 'zosal' + i
    infile = ''.join(['NOEMP02_1850_1899_1Y_',var,'.nc'])
    print '** Processing: ',infile
    f_h = cdm.open(infile)
    var1850to1899 = f_h(var) ; f_h.close()
    infile = replace(infile,'1850_1899','1900_1949')
    f_h = cdm.open(infile)
    var1900to1949 = f_h(var) ; f_h.close()
    vars()[i] = MV.concatenate([var1850to1899,var1900to1949])
    del(var1850to1899,var1900to1949,var,infile) ; gc.collect()
del(i,domains) ; gc.collect()

>>> glo_1850to1899.shape
(50, 31, 149, 1)
>>> glo_1900to1949.shape
(50, 31, 149, 1)
>>> glo = MV.concatenate([glo_1850to1899,glo_1900to1949])
/usr/local/uvcdat/2.4.0rc1/lib/python2.7/site-packages/cdms2/axis.py:2266: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if None in boundsArray:

It will likely disappear in planned numpy/cdms updates #885 (and has already been noted before #576), but wondering whether this should be cleaned up before 2.4.0?

Migrated from: CDAT/cdat#1580

AbstractAxis.isTime() speedup

On a mis-configured machine such as rhea.ccs.ornl.gov where a file open operation is expensive, one unnecessary bottleneck is in AbstractAxis.isTime(), in cdms2 axis.py.

Every time isTime() is called on an axis without a :axis attribute, it will try to analyze the units. Each call of available_units() involves opening a file. (Later on I'll enter this as another issue.)

An easy fix is to save results of previous isTime() calls, as a dictionary keyed by the axis id. I have already made this change and tested it in a private copy of UV-CDAT.

Migrated from: CDAT/cdat#1609

Cannot regrid generic grids 

import cdms2, vcs

f = cdms2.open(vcs.sample_data + "/sampleGenGrid3.nc")
f2 = cdms2.open(vcs.sample_data + "/clt.nc")
sample = f("sample")
clt = f("clt")
sample.regrid(clt.getGrid())

raises:

/Users/fries2/anaconda/envs/cdat_widgets/lib/python2.7/site-packages/cdms2/avariable.pyc in regrid(self, togrid, missing, order, mask, **keywords)
   1128                             dstGridMask = None,
   1129                             dstGridAreas = None,
-> 1130                             **keywords)
   1131             # now interpolate
   1132             return ro(self, **keywords)

/Users/fries2/anaconda/envs/cdat_widgets/lib/python2.7/site-packages/cdms2/mvCdmsRegrid.pyc in __init__(self, srcGrid, dstGrid, dtype, regridMethod, regridTool, srcGridMask, srcGridAreas, dstGridMask, dstGridAreas, **args)
    337         self.dstGrid = dstGrid
    338 
--> 339         srcCoords = _getCoordList(srcGrid)
    340         dstCoords = _getCoordList(dstGrid)
    341 

/Users/fries2/anaconda/envs/cdat_widgets/lib/python2.7/site-packages/cdms2/mvCdmsRegrid.pyc in _getCoordList(grid)
    216 
    217         # have axes, need to convert to curvilinear grid
--> 218         cgrid = grid.toCurveGrid()
    219 
    220         lats = cgrid.getLatitude()

AttributeError: 'TransientGenericGrid' object has no attribute 'toCurveGrid'

This is because the TransientGenericGrid object has no function to convert to a TransientCurveGrid. I tried switching toCurveGrid() to toGenericGrid(), but that leads to a lot of weird errors.

cdtime and big hours 

import cdtime
val = 239261039241284857122685270850462023680.0
cdtime.reltime(val, "hours since 1").tocomp()

fails as follow:

CDMS error: Error on time conversion: invalid hour = 239261039241284857122685270850462023680.000000

Traceback (most recent call last):
  File "/git/uvcdat/testing/cdms2/test_long_hour.py", line 5, in <module>
    cdtime.reltime(val, "hours since 1").tocomp()
Cdtime error: Invalid component time

Migrated from: CDAT/cdat#781

scalar FileVariable as data for createVariable()

Suppose you have a scalar FileVariable fv, i.e. fv.shape==(). You want to create a scalar TransientVariable from it, so you call cdms2.createVariable( fv ). It throws an exception! The reason is that TransientVariable.init() calls data.subSlice() as almost the first thing it does, and assumes that the result is a TransientVariable with a getDomain() method. It isn't, subSlice() on a scalar FileVariable is a numpy scalar (numpy.int32 in my case) and hence has no such method.

My workaround is to supply as data fv.subSlice() rather than fv. But this is a bug - either in TransientVariable.init() or in FileVariable.subSlice(), your choice (or my choice if I'm the one to fix it).

Migrated from: CDAT/cdat#1325

cdms2.open() should support a "gridfile" keyword

Hi,

some ocean models like NEMO don't save their 2D axes in the same file as outputs.
It would be useful to support this feature when reading such model outputs.
One way is probably to provide a keyword like "gridfile" to specify this grid file.
Coordinates variables will be searched for using the grid file descriptor,
which will defaults to the data file descriptor when the grid file is not set.

Migrated from: CDAT/cdat#535

Error reading a variable with cdms2

I get the following error when trying to slice a variable originating on an opendap server:

 $ python -c "import cdms2; cdms2.open('http://esgcet.llnl.gov/dap/ipcc4/20c3m/ncar_ccsm3_0/pcmdi.ipcc4.ncar_ccsm3_0.20c3m.run6.atm.mo.xml').variables['ua'][0, 0, :, :]"
ncvargets: ncid 65536: NetCDF: Variable has no data in DAP request
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<prefix>/cdms2/variable.py", line 82, in __getitem__
    return AbstractVariable.__getitem__(self, key)
  File "<prefix>/cdms2/avariable.py", line 1399, in __getitem__
    return self.getSlice(numericSqueeze=1, raw=0, isitem=1, *speclist)
  File "<prefix>/cdms2/avariable.py", line 656, in getSlice
    result = self.subSlice(*specs, **keys)
  File "<prefix>/cdms2/avariable.py", line 567, in subSlice
    axis = self.getAxis(i).subaxis(slice.start, slice.stop, slice.step, wrap=(numericSqueeze==0))
  File "<prefix>/cdms2/axis.py", line 1256, in subaxis
    fullBounds = self.getBounds()
  File "<prefix>/cdms2/axis.py", line 1931, in getBounds
    boundsArray = self.getExplicitBounds()
  File "<prefix>/cdms2/axis.py", line 1949, in getExplicitBounds
    boundsArray = numpy.ma.filled(boundsVar)
  File "/Library/Python/2.7/site-packages/numpy-override/numpy/ma/core.py", line 582, in filled
    return np.array(a)
  File "<prefix>/cdms2/avariable.py", line 138, in __array__
    return numpy.ma.filled(self.getValue(squeeze=0))
  File "<prefix>/cdms2/fvariable.py", line 161, in getValue
    return self.getSlice(Ellipsis, squeeze=squeeze)
  File "<prefix>/cdms2/avariable.py", line 656, in getSlice
    result = self.subSlice(*specs, **keys)
  File "<prefix>/cdms2/avariable.py", line 549, in subSlice
    d = self.expertSlice (slicelist)
  File "<prefix>/cdms2/fvariable.py", line 84, in expertSlice
    result = apply(self._obj_.getitem,slist)
IOError: Unknown error

However accessing the components one at a time works:

$ python -c "import cdms2; print cdms2.open('http://esgcet.llnl.gov/dap/ipcc4/20c3m/ncar_ccsm3_0/pcmdi.ipcc4.ncar_ccsm3_0.20c3m.run6.atm.mo.xml').variables['ua'][1, 10, 0, 0]"
-1.70405423641

netCDF4 python also reads it fine:

$ python -c "from netCDF4 import Dataset; print Dataset('http://esgcet.llnl.gov/dap/ipcc4/20c3m/ncar_ccsm3_0/pcmdi.ipcc4.ncar_ccsm3_0.20c3m.run6.atm.mo.xml').variables['ua'][1, 10, :, :]"
[[-1.70405424 -1.72662032 -1.74810171 ..., -1.62960672 -1.65557122  -1.68037927]
 [-1.28498518 -1.27757108 -1.26952016 ..., -1.30228746 -1.2974571   -1.29165149]
 [-0.92818403 -0.89008546 -0.85163492 ..., -1.038167   -1.00244784  -0.96571213]
 ..., 
 [ 5.33871937  5.37260056  5.40242577 ...,  5.21467352  5.259552     5.30097008]
 [ 4.2086339   4.22407246  4.23648071 ...,  4.14485407  4.16894293   4.19023132]
 [ 2.89284873  2.89466238  2.89463282 ...,  2.87655139  2.88376951   2.88921022]]

Migrated from: CDAT/cdat#1250

Warning from numpy1.9.0 in ESMP_API.py and avariable.py

[durack1@oceanonly 140115_PeterGleckler]$ /usr/local/uvcdat/2014-07-21/bin/python
Python 2.7.7 (default, Jul 21 2014, 15:49:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import numpy
numpy.version.version
'1.9.0b1'

/usr/local/uvcdat/2014-07-21/lib/python2.7/site-packages/ESMP/src/ESMP_API.py:1241: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if (srcMaskValues != None):

and same file another line:

/usr/local/uvcdat/2014-08-22/lib/python2.7/site-packages/ESMP/src/ESMP_API.py:1248: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if (dstMaskValues != None):

Migrated from: CDAT/cdat#576

cdsm2 createGenericGrid need to be generic

From @doutriaux1 on August 9, 2016 16:18

f2=  cdms2.open(fnmcurv)
In [4]: s=f2("sample")
In [5]: g=s.getGrid()
In [6]: lat = g.getLatitude()
In [7]: lon=g.getLongitude()
In [9]: g2 = cdms2.createGenericGrid(lat,lon)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-da081e424d79> in <module>()
----> 1 g2 = cdms2.createGenericGrid(lat,lon)

Copied from original issue: CDAT/cdat#2081

test_fillvalue.py failed

fill_value is not retain when writing a variable to a file.

======================================================================
FAIL: testWriteFillValue (__main__.TestFillValueWrite)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_fillvalue.py", line 60, in testWriteFillValue
    self.assertTrue(all([v == 1.33 for v in (m.fill_value, m._FillValue, m.missing_value)]))
AssertionError: False is not true

----------------------------------------------------------------------

cdscan xml from multiple files dropped time axis bounds --> cdutil.YEAR fails

This is on a Ubuntu 14.04.3 system with uvcdat 2.1.0.
I see the same failure on a vm with Ubuntu 13.10 with uvcdat 1.5.0-rc5.

When I used cdscan for two .nc files, the resulting xml no longer contains time axis bounds which makes the cdutil.YEAR function fail:

ev_mean = cdutil.YEAR(ev)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/uvcdat/2.1.0/lib/python2.7/site-packages/cdutil/times.py", line 1286, in get
s.append(TimeSlicer.get(self,slab,season,criteriaarg,statusbar=statusbar,sum=sum))
File "/usr/local/uvcdat/2.1.0/lib/python2.7/site-packages/cdutil/times.py", line 383, in get
slices,bounds,norm=self.slicer(tim,slicerarg)
File "/usr/local/uvcdat/2.1.0/lib/python2.7/site-packages/cdutil/times.py", line 631, in monthBasedSlicer
b0=cdtime.reltime(bnds[i][0],units)
TypeError: 'NoneType' object has no attribute 'getitem'

I am able to use cdutil.YEAR on each of the .nc files individually (and confirmed that they have bounds on their time axes).

I include output below showing the xml failure and the success on the two .nc files.
I also show the var info for the two individual .nc files.

------------ example code ----------------------------

bhayes@dge-calc:.../CESM1-CAM5/r1i1p1$ cdscan -x ev.xml evspsbl_Amon*
Finding common directory ...
Common directory:
Scanning files ...
evspsbl_Amon_CESM1-CAM5_rcp26_r1i1p1_200601-210012.nc
Setting reference time units to days since 2006-01-01 00:00:00
evspsbl_Amon_CESM1-CAM5_rcp26_r1i1p1_210101-230012.nc
Setting reference time units to days since 2006-01-01 00:00:00
ev.xml written

bhayes@dge-calc:.../CESM1-CAM5/r1i1p1$ cdat
Python 2.7.8 (default, Jan 14 2015, 10:22:34)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import cdms2 as cdms
import cdutil
f = cdms.open('ev.xml')
f.variables.keys()
['time_bnds', 'lat_bnds', 'evspsbl', 'lon_bnds']
ev = f('evspsbl')
ev.shape
(3540, 192, 288)
ev_mean = cdutil.YEAR(ev)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/uvcdat/2.1.0/lib/python2.7/site-packages/cdutil/times.py", line 1286, in get
s.append(TimeSlicer.get(self,slab,season,criteriaarg,statusbar=statusbar,sum=sum))
File "/usr/local/uvcdat/2.1.0/lib/python2.7/site-packages/cdutil/times.py", line 383, in get
slices,bounds,norm=self.slicer(tim,slicerarg)
File "/usr/local/uvcdat/2.1.0/lib/python2.7/site-packages/cdutil/times.py", line 631, in monthBasedSlicer
b0=cdtime.reltime(bnds[i][0],units)
TypeError: 'NoneType' object has no attribute 'getitem'

ta = ev.getAxis(0)
ta.shape
(3540,)
tab = ta.getBounds()
type(tab)
<type 'NoneType'>

----------- mean for file 1 ---------

f1 = cdms.open('evspsbl_Amon_CESM1-CAM5_rcp26_r1i1p1_200601-210012.nc')
ev1 = f1('evspsbl')
ev1_mean = cdutil.YEAR(ev1)
ev1_mean.shape
(95, 192, 288)

f1.close()
ta1 = ev1.getAxis(0)
tab1 = ta1.getBounds()
type(tab1)
<type 'numpy.ndarray'>
tab1.shape
(1140, 2)

----------------- mean for file 2 -----

f2 = cdms.open('evspsbl_Amon_CESM1-CAM5_rcp26_r1i1p1_210101-230012.nc')
ev2 = f2('evspsbl')
ev2.shape
(2400, 192, 288)
ev2_means = cdutil.YEAR(ev2)
ev2_means.shape
(200, 192, 288)
ta2 = ev2.getAxis(0)
tab2 = ta2.getBounds()
type(tab2)
<type 'numpy.ndarray'>
tab2.shape
(2400, 2)

-------------- ev1 info ------------------

ev1.info()
*** Description of Slab evspsbl ***
id: evspsbl
shape: (1140, 192, 288)
filename:
missing_value: 1e+20
comments:
grid_name:
grid_type: generic
time_statistic:
long_name: Evaporation
units: kg m-2 s-1
comment: QFLX no change, CMIP5_table_comment: at surface; flux of water into the atmosphere due to conversion of both liquid and solid phases to vapor (from underlying surface and vegetation)
_FillValue: [ 1.00000002e+20]
autoApiInfo: <AutoAPI.AutoAPI.Info instance at 0x7efcc240ef38>
tileIndex: None
original_name: QFLX
associated_files: baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CESM1-CAM5_rcp26_r0i0p0.nc areacella: areacella_fx_CESM1-CAM5_rcp26_r0i0p0.nc
standard_name: water_evaporation_flux
cell_methods: time: mean (interval: 30 days)
cell_measures: area: areacella
original_units: kg/m2/s
history: 2012-06-08T23:37:19Z altered by CMOR: Converted units from 'kg/m2/s' to 'kg m-2 s-1'. 2012-06-08T23:37:19Z altered by CMOR: Reordered dimensions, original order: lat lon time. 2012-06-08T23:37:19Z altered by CMOR: replaced missing value flag (-1e+32) with standard missing value (1e+20).
Grid has Python id 0x7efcc241b050.
Gridtype: generic
Grid shape: (192, 288)
Order: yx
** Dimension 1 **
id: time
Designated a time axis.
units: days since 2006-01-01 00:00:00
Length: 1140
First: 15.5
Last: 34659.5
Other axis attributes:
long_name: time
standard_name: time
calendar: noleap
axis: T
Python id: 0x7efcc241b890
** Dimension 2 **
id: lat
Designated a latitude axis.
units: degrees_north
Length: 192
First: -90.0
Last: 90.0
Other axis attributes:
long_name: latitude
standard_name: latitude
axis: Y
Python id: 0x7efcc241b910
** Dimension 3 **
id: lon
Designated a longitude axis.
units: degrees_east
Length: 288
First: 0.0
Last: 358.75
Other axis attributes:
modulo: 360.0
long_name: longitude
standard_name: longitude
axis: X
topology: circular
Python id: 0x7efcc241b990
*** End of description for evspsbl ***

---------- ev2 info ---------------------

...
ev2.info()
*** Description of Slab evspsbl ***
id: evspsbl
shape: (2400, 192, 288)
filename:
missing_value: 1e+20
comments:
grid_name:
grid_type: generic
time_statistic:
long_name: Evaporation
units: kg m-2 s-1
comment: QFLX no change, CMIP5_table_comment: at surface; flux of water into the atmosphere due to conversion of both liquid and solid phases to vapor (from underlying surface and vegetation)
_FillValue: [ 1.00000002e+20]
autoApiInfo: <AutoAPI.AutoAPI.Info instance at 0x7efcc24236c8>
tileIndex: None
original_name: QFLX
associated_files: baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CESM1-CAM5_rcp26_r0i0p0.nc areacella: areacella_fx_CESM1-CAM5_rcp26_r0i0p0.nc
standard_name: water_evaporation_flux
cell_methods: time: mean (interval: 30 days)
cell_measures: area: areacella
original_units: kg/m2/s
history: 2012-06-15T22:28:22Z altered by CMOR: Converted units from 'kg/m2/s' to 'kg m-2 s-1'. 2012-06-15T22:28:22Z altered by CMOR: Reordered dimensions, original order: lat lon time. 2012-06-15T22:28:22Z altered by CMOR: replaced missing value flag (-1e+32) with standard missing value (1e+20).
Grid has Python id 0x7efcc241bf50.
Gridtype: generic
Grid shape: (192, 288)
Order: yx
** Dimension 1 **
id: time
Designated a time axis.
units: days since 2100-01-01 00:00:00
Length: 2400
First: 380.5
Last: 73349.5
Other axis attributes:
long_name: time
standard_name: time
calendar: noleap
axis: T
Python id: 0x7efcc241bf10
** Dimension 2 **
id: lat
Designated a latitude axis.
units: degrees_north
Length: 192
First: -90.0
Last: 90.0
Other axis attributes:
long_name: latitude
standard_name: latitude
axis: Y
Python id: 0x7efcc241b850
** Dimension 3 **
id: lon
Designated a longitude axis.
units: degrees_east
Length: 288
First: 0.0
Last: 358.75
Other axis attributes:
modulo: 360.0
long_name: longitude
standard_name: longitude
axis: X
topology: circular
Python id: 0x7efcc2437150
*** End of description for evspsbl ***

Migrated from: CDAT/cdat#1646

cdutil.ANNUALCYCLE.departures lopping off the final month

LWP_forCharles.nc.zip

@doutriaux1
There seems to be an issue with cdutil.ANNUALCYCLE.departures. It is lopping off the final month, like so:

In [1]: import cdms2 as cdms

In [2]: f=cdms.open('LWP_forCharles.nc')

In [5]: LWP_grd.shape
Out[5]: (252, 45, 180)

In [9]: anomLWP_grd = cdutil.ANNUALCYCLE.departures(LWP_grd)

In [10]: anomLWP_grd.shape
Out[10]: (251, 45, 180)

In [11]: LWP_grd.getTime()
Out[11]:
id: axis_229493
Designated a time axis.
units: months since 1988-01-01
Length: 252
First: 0.0
Last: 251.0
Other axis attributes:
calendar: gregorian
axis: T
realtopology: linear
Python id: 0x7fa604766890

In [12]: anomLWP_grd.getTime()
Out[12]:
id: time
Designated a time axis.
units: months since 1988-01-01
Length: 251
First: 0.0
Last: 250.0
Other axis attributes:
calendar: gregorian
axis: T
Python id: 0x7fa60431ee90

Note how the first dimension is shortened by 1.

Migrated from: CDAT/cdat#2014

Catch cdscan errors without triggering runtime failures

The following problems cause runtime failures in cdscan rather than gracefully being caught in an error and quitting..

[durack1@oceanonly _logs]$ more 150828_120003_make_cmip5_xml-oceanonly-threads40-PID30552.log | grep "PROBLEM 4"
** 0000001 150828_235226 042462.37s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp26/mon/land/Lmon/r1i1p1/v20120416/mrro **
** 0000002 150828_235226 042535.49s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp26/mon/land/Lmon/r1i1p1/v20120719/mrro **
** 0000003 150828_235226 042544.44s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp26/mon/land/Lmon/r1i1p1/v20121012/mrro **
** 0000004 150829_005141 045984.87s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp45/mon/land/Lmon/r1i1p1/v20120416/mrro **
** 0000005 150829_005141 046003.45s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp45/mon/land/Lmon/r1i1p1/v20120626/mrro **
** 0000006 150829_005141 046047.42s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp45/mon/land/Lmon/r1i1p1/v20121012/mrro **
** 0000007 150829_015437 049842.48s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp60/mon/land/Lmon/r1i1p1/v20120416/mrro **
** 0000008 150829_015437 049848.99s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp60/mon/land/Lmon/r1i1p1/v20120626/mrro **
** 0000009 150829_015437 049861.26s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp60/mon/land/Lmon/r1i1p1/v20121012/mrro **
** 0000010 150829_030739 054018.56s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp85/mon/land/Lmon/r1i1p1/v20120416/mrro **
** 0000011 150829_030739 054033.80s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp85/mon/land/Lmon/r1i1p1/v20120626/mrro **
** 0000012 150829_030739 054104.69s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp85/mon/land/Lmon/r1i1p1/v20121012/mrro **
** 0000013 150830_011805 133894.04s PROBLEM 4 (no outfile) indexing /cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs **
** 0000014 150830_011805 133897.95s PROBLEM 4 (no outfile) indexing /cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130326/hurs **
** 0000015 150830_012636 134436.62s PROBLEM 4 (no outfile) indexing /cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/ts **
** 0000016 150830_012636 134785.00s PROBLEM 4 (no outfile) indexing /cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/ta **
** 0000017 150830_101001 166191.95s PROBLEM 4 (no outfile) indexing /cmip5_css02/scratch/cmip5/output1/NASA-GISS/GISS-E2-H/historicalExt/mon/atmos/Amon/r1i1p1/v20120119/vas **
** 0000018 150831_211730 292641.93s DATA PROBLEM 4 (no outfile) indexing /cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1 **
** 0000019 150831_221729 296236.05s DATA PROBLEM 4 (no outfile) indexing /cmip5_css02/data/cmip5/output1/INPE/HadGEM2-ES/historical/mon/atmos/Amon/r5i1p1/tasmax/1 **

[durack1@oceanonly _logs]$ source /usr/local/uvcdat/2015-08-25/bin/setup_runtime.csh
Successfully updated your environment to use UVCDAT
(changes are valid for this session/terminal only)
Version: 2.2.0-304-geb6c9a3
Location: /usr/local/uvcdat/2015-08-25
[durack1@oceanonly _logs]$ which cdscan
/usr/local/uvcdat/2015-08-25/bin/cdscan

[durack1@oceanonly _logs]$ cdscan -x ~/cdscan_test.xml /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp26/mon/land/Lmon/r1i1p1/v20120416/mrro/*.nc
Finding common directory ...
Common directory: /cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp26/mon/land/Lmon/r1i1p1/v20120416/mrro/
Scanning files ...
/cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp26/mon/land/Lmon/r1i1p1/v20120416/mrro/mrro_Lmon_CCSM4_rcp26_r1i1p1_200501-210012.nc
Setting reference time units to days since 2005-01-01 00:00:00
/cmip5_css02/scratch/cmip5/output1/NCAR/CCSM4/rcp26/mon/land/Lmon/r1i1p1/v20120416/mrro/mrro_Lmon_CCSM4_rcp26_r1i1p1_200601-210012.nc
Setting reference time units to days since 2005-01-01 00:00:00
Traceback (most recent call last):
  File "/usr/local/uvcdat/2015-08-25/bin/cdscan", line 1680, in <module>
    main(sys.argv)
  File "/usr/local/uvcdat/2015-08-25/bin/cdscan", line 1539, in main
    raise RuntimeError, "Variable '%s' is duplicated, and is a function of lat or lon: files %s, %s"%illegalvars[0]
RuntimeError: Variable 'mrro' is duplicated, and is a function of lat or lon: files mrro_Lmon_CCSM4_rcp26_r1i1p1_200501-210012.nc, mrro_Lmon_CCSM4_rcp26_r1i1p1_200601-210012.nc

[durack1@oceanonly _logs]$ cdscan -x ~/cdscan_test.xml /cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/*.nc
Finding common directory ...
Common directory: /cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/
Scanning files ...
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_085001-094912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_095001-104912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_105001-114912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_115001-124912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_125001-134912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_135001-144912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_145001-154912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_155001-164912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_165001-174912.nc
Setting reference time units to days since 850-01-01 00:00:00
/cmip5_css01/scratch/cmip5/output1/LASG-IAP/FGOALS-s2/past1000/mon/atmos/Amon/r1i1p1/v20130315/hurs/hurs_Amon_FGOALS-s2_past1000_r1i1p1_175001-185012.nc
Setting reference time units to days since 850-01-01 00:00:00
Traceback (most recent call last):
  File "/usr/local/uvcdat/2015-08-25/bin/cdscan", line 1680, in <module>
    main(sys.argv)
  File "/usr/local/uvcdat/2015-08-25/bin/cdscan", line 1539, in main
    raise RuntimeError, "Variable '%s' is duplicated, and is a function of lat or lon: files %s, %s"%illegalvars[0]
RuntimeError: Variable 'hurs' is duplicated, and is a function of lat or lon: files hurs_Amon_FGOALS-s2_past1000_r1i1p1_145001-154912.nc, hurs_Amon_FGOALS-s2_past1000_r1i1p1_155001-164912.nc

[durack1@oceanonly _logs]$ cdscan -x ~/cdscan_test.xml /cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/*.nc
Finding common directory ...
Common directory: /cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/
Scanning files ...
/cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/zg_Amon_HadGEM2-AO_piControl_r1i1p1_000101-010012.nc
Setting reference time units to days since 0001-01-01
/cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/zg_Amon_HadGEM2-AO_piControl_r1i1p1_010101-020012.nc
Setting reference time units to days since 0001-01-01
/cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/zg_Amon_HadGEM2-AO_piControl_r1i1p1_020101-030012.nc
Setting reference time units to days since 0001-01-01
/cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/zg_Amon_HadGEM2-AO_piControl_r1i1p1_030101-040012.nc
Setting reference time units to days since 0001-01-01
/cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/zg_Amon_HadGEM2-AO_piControl_r1i1p1_040101-050012.nc
Setting reference time units to days since 0001-01-01
/cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/zg_Amon_HadGEM2-AO_piControl_r1i1p1_050101-060012.nc
Setting reference time units to days since 0001-01-01
/cmip5_css02/data/cmip5/output1/NIMR-KMA/HadGEM2-AO/piControl/mon/atmos/Amon/r1i1p1/zg/1/zg_Amon_HadGEM2-AO_piControl_r1i1p1_060101-070012.nc
Setting reference time units to days since 0001-01-01
Traceback (most recent call last):
  File "/usr/local/uvcdat/2015-08-25/bin/cdscan", line 1680, in <module>
    main(sys.argv)
  File "/usr/local/uvcdat/2015-08-25/bin/cdscan", line 1539, in main
    raise RuntimeError, "Variable '%s' is duplicated, and is a function of lat or lon: files %s, %s"%illegalvars[0]
RuntimeError: Variable 'zg' is duplicated, and is a function of lat or lon: files zg_Amon_HadGEM2-AO_piControl_r1i1p1_020101-030012.nc, zg_Amon_HadGEM2-AO_piControl_r1i1p1_030101-040012.nc

Migrated from: CDAT/cdat#1512

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.