foroozandehgroup / nmrpoise Goto Github PK

It's only a container for a function. Just use the function!

I have a nagging feeling that we don't actually need numpy & everything can be done purely using lists...
This would mean that we don't actually need to use Py3 at all... and everything can be built into TS directly. It seems ambitious, but I don't see why not.

It's a waste of time, and gives 2 ugly popups

e.g. parse_poptpy_log() which I just wrote -- convert it to pandas dataframes

Shouldn't try to be too smart; just tell the user that they need the packages and ask them to install it themselves.

Yeah, this needs to be written.

e.g. for PSYCHE flip angle optimisation

probably something like get_real_spec(left=float, right=float, expno=int) would work.

xatol is only a single number, not a vector. So it is impossible to implement different tolerances for different parameters.

This could probably be built into a decorator (the same one which counts number of experiments ran). Not sure if this is a good idea because the cost function has noise - so if we get a poor value the first time we acquire it, this will always persist - in the worst case scenario we could end up stuck in a loop. Maybe code it to re-acquire the spectrum after a certain number of times?

e.g. mdsmax.m

If the installation is done manually, there's a chance that the pypopt/routines and pypopt/cost_functions folders are not there; the Python scripts should check for their existence and make them first if necessary.

Trying to open a file object f = open("/path/to/nonexistent_dir/file", "wb") directly will throw an IOError.

pip install . doesn't allow one to import poptpy, but pip install -e . does.

Something gone wrong in setup.py...

e.g. user specifies bounds for parameter 1, then clicks Next to go to parameter 2, but wants to go back to edit parameter 1 again.

Occasionally TopSpin hangs during subprocess.check_call(), suspect that this may be related, although cannot quite tell https://stackoverflow.com/questions/5998947/nio-error-when-parsing-xml-file

This bug cannot be reliably reproduced (yet)...

For pypopt-be.py and pypopt-makecf.py, to be precise.

CNSTxx? There are 64 of them...

There isn't one. We might need one.

Just --maxfev probably...

--setup flag I suppose

The user shouldn't have to edit the source code depending on the intended usage...!

It's not obvious what the best way to do this is. We could make a new folder inside poise_backend, along with routines and cost_functions (big downside: TopSpin cannot detect this folder automatically). Or we could install the AU programmes into the TopSpin directory. The name of the AU programme would need to be tied to the Routine. I'm already running into issues where I forget to change the AU programme between experiments...!

Should at least print all routine parameters including lb, ub, and tols.

Although probably only after we have a final acronym

In case the user (read: me) accidentally runs pypopt_makecf in TopSpin instead of edpy pypopt_makecf.

Most easily done by checking NS vs status NS. There doesn't seem to be a sensible use case for letting the optimisation continue if the acquisition doesn't finish.

This is available in the *nix instaler but not on the Windows one.

Can we do it with a check to, say, the name of the process (which we may be able to assign(?) when using subprocess.Popen)?

Some error checking is needed for the file management commands (cp, mkdir, ...) for example, most people shouldn't have issues with permissions, but what happens if (e.g.) $TOPSPINDIR is not set correctly and then the installer tries to copy the Python scripts to a non-existent directory. We need to capture the error code

For experiments that utilise WaveMaker shapes, they need to be generated with XCMD("wvm -q") before every acquisition. However, if we run this on a pulse programme that doesn't need WaveMaker shapes, it will throw an error before every acquisition, which has to be manually closed by the user:

So we need a way of running that line IF... otherwise don't...

Instead of giving the user an error whenever multiple TopSpin directories are present, why not just.... install to.... ALL of them?????

Routines should be serialised with json extension.

.... poptpy does not quite roll off the tongue

the final popup with the optima blocks subsequent actions

Installing from wheels (which pip prefers) doesn't actually install into TopSpin. However, installing from sdist, viz. pip install nmrpoise --no-binary :all: works just fine. I think I should just not upload wheels to PyPI.

But I am also thinking, maybe even if it doesn't install into TopSpin, we should provide a setuptools entry point that runs topspin_install.py.

If set to true then just do RE_IEXPNO() or whatever it was.

pypopt is already a thing... https://pypi.org/project/pypopt/

Probably a Python wrapper that checks platform.system() and calls the appropriate script.

When defining a cost function, the function has to be defined with the name cost_function(), because of this line:

https://github.com/yongrenjie/poptpy/blob/ac4943380537ffcda899db6796e4bec695ee9ecb/poptpy/poptpy_be.py#L52

It's probably a good idea to keep the function naming consistent, but we can catch accidental typos by checking if ld only has one key and if so, using that function as the cost function, viz.

ld = {}
exec(open("b.py").read(), globals(), ld)

try:
    cost_function = ld["cost_function"]
except KeyError as e:
    if len(ld) == 1:
        cost_function = ld[list(ld)[0]]
    else:
        raise e

print(cost_function())

e.g. poptpy list to show a dialog box with all routines available, also use argv[2] to select the optimiser

The tolerances are the tricky part here -- since rho_end has to be one value, we need to scale every parameter such that the tolerance is the same.

That entails adding some flexibility into many of the other functions such that lb and ub aren't hardcoded to be 0 and 1 for everything.

Ideas:

• total intensity (sqrt(real^2+imag^2), or just the real) - can use for HSQC
• 1D NOE (using spoffs2)
• Similarity function (how to specify expno/procno? we don't have this capability yet)
• var(atan(re/absim)) (this is still lacking justification)
• selective pulse (for selective excitation eg S/DPFGSE)
• selective pulse (for Zangger-Sterk)
• other pulse shapes, to demonstrate WaveMaker use

we must prevent cases where any sort of data overwriting is possible...!

For example, this:

****** Py-BOBYQA Results ******
Exit flag = -1
Error (bad input): gap between lower and upper must be at least 2*rhobeg

nfev overestimates the time taken because it includes function evaluations outside the ranges, which don't actually lead to spectra being acquired.

The most direct solution is to use a global variable, but there should be something better...

This doesn't actually stop the optimisation from finishing (as evidenced by the logs). However, it stops the optima from being stored in the dataset (bad), and also prevents the popup at the end notifying the user (doesn't matter quite so much).

From the stack trace it appears to be related to the errlog context manager which surrounds the main loop. There appears to be a problem with closing the ferr file. But honestly, the stack trace is incredibly useless, so I don't actually know what's causing it.

If the frontend is terminated (e.g. using TopSpin's kill command) while the backend is still running, the backend subprocess isn't terminated properly.

(1888: Still in Use)

Especially on Windows, this can cause issues with trying to open / delete / otherwise modify the poptpy.log file. This includes seemingly innocuous things such as renaming the EXPNO in TopSpin (which tries to rename the entire EXPNO folder).

This can be fixed with e.g. Stop-Process -name python from PowerShell, but there has to be a better way. I don't know how TopSpin implements the kill function, but we need some cleanup code to terminate the subprocess (perhaps a subprocess context manager - not sure if that exists in Py2.7).