Error calling NCO command with tempfile helper about pynco HOT 5 CLOSED

Thanks, @jhamman . There are several failing tests that I have fixed, plus a few more cleanups (PEP8, unused variables, etc.) that I've worked in. I will submit a PR soon. I appreciate your help.

from pynco.

I think this usage should be returning a path to a temporary. I'm not actively working on this package but would be happy to help others fix any current problems.

from pynco.

Thanks, @jhamman . I will do my best.

From what I can see the command is being passed into the shell and executed as is without the tempfile helper, eg /miniconda3/envs/climate/bin/ncpdq -a "-lat,lon,time" in.nc

It appears that there was no temporary file being created for use as output arguments to the NCO commands, i.e. no actual tempfile helper wrapper is in place. Or maybe I'm missing something obvious?

I have added some code for this on my local branch and it seems to work as expected now, in that it's now creating the temporary file and using this temp file's path as the value for the --output argument. The trouble at this point is that the command never seems to execute, it just shows up in the Ubuntu task manager with sleeping as its status property (same with the associated sh process). Can anyone advise?

from pynco.

@monocongo - just took a brief look at the referenced commit above. Looks mostly good to me. Please open a PR and we can get it cleaned up enough to be merged.

from pynco.

BTW while testing this with a Python2.7 virtual environment I am seeing some cases where the NCO executable path is being set from the OS environment's PATH, and in that case it doesn't find the NCO executables installed in the virtual environment but rather from the base OS location instead. This is problematic if NCO is installed into the OS but ncdump is not, i.e. when the NCO path is set it uses the location where ncks is located in the OS, such as /usr/bin, and when the test for ncdump -h is run it fails since ncdump isn't located under /usr/bin (file not found error). Maybe there's something amiss with how I have my PATH being set when running these tests under a 2.7 (Anaconda) virtual environment? I've tried this again using the previous virtual environment I was using before and seeing the same behavior. I'm not sure why this was working well before, i.e. it was locating the ncks in the virtual environment before the OS location and using the virtual environment's path to ncks as the NCO path, and since this matched with where ncdump was installed in the venv all was good. This behavior has changed since creating a new Python2.7 virtualenv using conda, and I assume that this has goobered up some PATH settings that were causing the venv path to be used, which I assume is what we want? I think the underlying issue for the test at play (test_operator_prints_out()) is that it tests ncdump using the path to ncks, and the two could be installed in separate locations since ncdump isn't part of NCO per se. Or am I missing something obvious?

TL;DR: The 2.7 support is a bit hosed, while trying to fix that I have run into issues with finding NCO path under the new 2.7 conda environment I've created for this. One test involving ncdump is failing since the path to ncdump isn't always the same as the path to ncks, especially in cases when NCO and/or ncdump is installed only within a virtual environment and not on the base OS.

from pynco.