Type Error: unsopported operand type(s) for -: 'function and function' about hyperactive HOT 12 CLOSED

Hello @mlittmanabbvie,

thank you for the information about this issue, but I think there is some code missing. I must be able to take your example, run it and reproduce the error.

Only part of your code is in a code-block. Please make sure to put the code into a code-block like this:
```python

```
This makes it much easier for me to help you.

from hyperactive.

Fixed the code block, it was right below the actual block. Thank you for your help in advance!

from hyperactive.

Thank you very much!
There is still the problem, that I cannot run the code. I need a full example, that I can run. Without a complete script it is almost impossible for me to figure out the issue.

I need a code block that I can just copy into a *.py file and execute (to get the same error as you).

from hyperactive.

Updated code block, you can run the block directly now

from hyperactive.

Very good! :-) I will look into this issue later today.

from hyperactive.

I looked into this issue. I am certain, that it occurs because of a problem of the transformation between the hyperactive warm start into gfo warm start. I am surprised i missed this test case! I will fix this tomorrow (and write multiple tests) and release it in v3.3.3.

Thank you very much for pointing me to this problem! :-)

from hyperactive.

Thank you for looking into this! By the way, it also fails for strings, it says unsupported for type string and string. My guess would be that the line pos = np.abs(value[n] - np.array(space_dim)).argmin() is essentially assuming the value to be a numeric type which is why you then can't subtract a function from a function or a string from a string.

from hyperactive.

Your guess was right! ;-) It is a part of the code that converts the hyperactive parameter into gfo parameter. I already fixed it by checking the types in the search space beforehand and using the fast method for numeric search space dimensions:

pos = np.abs(value[n] - np.array(space_dim)).argmin()

If the type is not numeric but string or other python objects another method is used to convert the parameter.

This works fine for most cases! But there is an issue if the search space contains class instances and the number of jobs is > 1. For some reason the class instances change their position in memory, so that the comparison during the hyper -> gfo-conversion fails. I think this could be a problem with pickle.

The solution for all this would be something I am already planing for Hyperactive v4:
The search space should only contain numbers, strings or functions. If you want to put a class into the search space you can wrap it into a function:

class test_class:
  def __init__(self):
    pass

def class_wrapper1():
  return test_class

I think this should get around this problem.

I will continue (and probably finish) this tomorrow. I want to make sure that this is very stable and thoroughly tested.

from hyperactive.

After some more testing I discovered a problem with classes wrapped in functions in the search space when using multiprocessing (n_jobs>1 or multiple .add_search()). The package dill cannot pickle those objects. A solution could be to use cloudpickle, which is included in joblib for parallel computing. Joblib can already be used in Hyperactive see joblib tests (test_joblib_0 and follows). I will test if joblib (+ cloudpickle) solves the problem of classes in the search space.

@mlittmanabbvie If you need a solution fast you can clone the master, go into the dir and run make reinstall. This patches the warm start error (if you do not use multiprocessing). Otherwise you can wait for the official solution in v3.3.3.

from hyperactive.

Thank you so much for all of your help with this! I have another question, but I will open a new ticket for it

from hyperactive.

I released v3.3.3 of Hyperactive yesterday. It contains the fix to solve this issue, multiple new tests and more fixes and stability improvements for some related problems that might occur.

It is recommended to only use numbers, strings or functions in the search space. If you want to put an array, dataframe, class, ... into the search space you should wrap it into a function beforehand and put the wrapper-function into the search space. This is important for services like the warm-start (initialization) or the memory-warm-start, because Hyperactive needs an identifier for the parameter in the search space. Numbers and strings are good identifiers, because you can compare them with == or is, but this can be much more complicated with classes, lists or other objects.
This is why functions as wrappers are a better solution: Hyperactive recognizes the function by getting its name (a string) via .__name__. This is a proper identifier to compare functions with each other. For now Hyperactive throws a warning if the search space does not contain numbers, strings or functions, but in future versions (v4) Hyperactive will raise a ValueError.

There was also a problem pickling classes, objects and objects that are not top level. To solve this problem I added support for the pathos library. It uses cloudpickle to give the user more flexibility for pickling objects during parallel computing.

from hyperactive.

Is there an example of how the pathos library gets used? I just tried and I am getting the error "AttributeError: Can't pickle local object 'optimize..func_min'" when passing a local function into the optimizer. Will open new ticket for it.

from hyperactive.