python 3.7 version of e3sm-unified about e3sm-unified HOT 9 CLOSED

@jhkennedy, here's what we're up against, I'm afraid. A lot of CDAT packages (those from the CDAT channel) aren't built with python 3.7 yet. I think both processflow and zstash are strictly python 2.7 right now.

from e3sm-unified.

@jgfouca didn't CIME use six.py to help support both python 2 and python 3? Maybe that could help here.

from e3sm-unified.

@rljacob, thanks for the tip. I think we have a pretty good handle on the process for making packages work with both. @jhkennedy and I have both done this with LIVVKit and MPAS-Analysis, respectively. I think @zshaheen and @chengzhuzhang have also done this with e3sm_diags.

The remaining work falls into two categories:

• A lot of CDAT packages (on the cdat channel) were built with python 3.6 but not 3.7. With some help from @doutriaux1 or someone else on the CDAT team, this shouldn't be too hard to address
• processflow and zstash were written in python 2.7. It's just a matter of @sterlingbaldwin and @zshaheen putting in the time to make them python-3 compatible, which is partly a question of priorities. @jhkennedy and I are happy to help with that (as I think everyone involved knows).

from e3sm-unified.

As a fallback, all of the packages we need from the cdat channel do support python 3.6. So if we get to a point where processflow and zstash are ready to go but cdat stuff isn't, we could do the first py3 version with 3.6.

from e3sm-unified.

It sounds like the various cdat packages will have a v8.1 release early next week, including python 3.7 support! So that will take care of most of this.

from e3sm-unified.

Q: what does it mean to "support" python 3.6 vs. 3.7? Are python minor versions so different that code is not forward portable? Like going from 2 to 3?

from e3sm-unified.

@rljacob in the conda context, it means there is a compatible build for that specific version of python. For example, e3sm_nex has a build matrix that includes py27, py36, win, osx, and linux, which you can see reflected in the file names here: https://anaconda.org/cdat/e3sm_nex/files

So supporting python 3.7 just requires someone to build a 3.7 version (or building a noarch package that works on any python, if possible).

The major benefit of conda-forge is that the build matrix is handled automatically by CI tools, so if it's possible to get a package on conda-forge, that should be what's done.

(@xylar I think that mostly covers it, did I miss anything? )

from e3sm-unified.

And python minor versions should be backwards compatible, so if it works in 3.(x) it should work in 3.(x+1).

See PEP 387

from e3sm-unified.

Thanks @jhkennedy, that covers it and I was in the process of writing essentially the same thing.

As @jhkennedy said, it's not necessarily about changing the code (I didn't have to do anything in MPAS-Analysis to support 3.6 and 3.7 once I got set up for python 3.5) but each package has to either be specifically built for each python major and minor version or it has to say that it works with all supported versions (on conda-forge, this is currently 2.7, 3.6, 3.7) without any differences in build.

from e3sm-unified.