Giter Club home page Giter Club logo

Comments (12)

SimonBlanke avatar SimonBlanke commented on May 12, 2024 1

I fixed this error. But you have update Hyperactive (v3.0.6) and Gradient-Free-Optimizers (v0.3.1). I also added your script as a test, so that this never happens again. Let me know if this works for you now, so that we can close this issue.

And thanks for your help in finding this bug! :-)

from hyperactive.

cryptocoinserver avatar cryptocoinserver commented on May 12, 2024

I think it's related to a warm start. It happens if the optimization results (CSV) only have scores with np.nan.

from hyperactive.

SimonBlanke avatar SimonBlanke commented on May 12, 2024

Hello @cryptocoinserver,

could you provide the code to reproduce this error?

from hyperactive.

cryptocoinserver avatar cryptocoinserver commented on May 12, 2024
  def run(self):

    hyper = hyperactive.Hyperactive(distribution="multiprocessing",
                                    verbosity=["progress_bar", "print_results", "print_times"])

    self.search_space = self.get_search_space()

    # Later use actual search space combinations to determin n_iter
    # keys, values = zip(*self.search_space.items())
    # combinations = [dict(zip(keys, v)) for v in itertools.product(*values)]
    # combinations_count = len(combinations)

    mem = None

    if jh.file_exists(self.path):
      with open(self.path, "r") as f:
        mem = pd.read_csv(f, sep=";", na_values='nan')
      if not mem.empty:
        if not click.confirm('Previous optimization results for {} exists. Continue?'.format(self.study_name),
                             default=True):
          mem = None

    if self.optimizer == "RandomSearchOptimizer":
      optimizer = hyperactive.RandomSearchOptimizer()
    elif self.optimizer == "RandomRestartHillClimbingOptimizer":
      optimizer = hyperactive.RandomRestartHillClimbingOptimizer(
        epsilon=0.1,
        distribution="laplace",
        n_neighbours=4,
        rand_rest_p=0.1,
        n_iter_restart=20,
      )
    elif self.optimizer == "RandomAnnealingOptimizer":
      optimizer = hyperactive.RandomAnnealingOptimizer(
        epsilon=0.1,
        distribution="laplace",
        n_neighbours=4,
        rand_rest_p=0.1,
        annealing_rate=0.999,
        start_temp=0.8,
      )
    elif self.optimizer == "HillClimbingOptimizer":
      optimizer = hyperactive.HillClimbingOptimizer(
        epsilon=0.1, distribution="laplace", n_neighbours=4, rand_rest_p=0.1
      )
    elif self.optimizer == "RepulsingHillClimbingOptimizer":
      optimizer = hyperactive.RepulsingHillClimbingOptimizer(
        epsilon=0.1,
        distribution="laplace",
        n_neighbours=4,
        repulsion_factor=5,
        rand_rest_p=0.1,
      )
    elif self.optimizer == "SimulatedAnnealingOptimizer":
      optimizer = hyperactive.SimulatedAnnealingOptimizer(
        epsilon=0.1,
        distribution="laplace",
        n_neighbours=4,
        rand_rest_p=0.1,
        p_accept=0.15,
        norm_factor="adaptive",
        annealing_rate=0.999,
        start_temp=0.8,
      )
    elif self.optimizer == "ParallelTemperingOptimizer":
      optimizer = hyperactive.ParallelTemperingOptimizer(n_iter_swap=5, rand_rest_p=0.05)
    elif self.optimizer == "ParticleSwarmOptimizer":
      optimizer = hyperactive.ParticleSwarmOptimizer(
        inertia=0.4,
        cognitive_weight=0.7,
        social_weight=0.7,
        temp_weight=0.3,
        rand_rest_p=0.05,
      )
    elif self.optimizer == "EvolutionStrategyOptimizer":
      optimizer = hyperactive.EvolutionStrategyOptimizer(
        mutation_rate=0.5, crossover_rate=0.5, rand_rest_p=0.05
      )
    else:
      raise ValueError('Entered optimizer which is {} is not known.'.format(
        self.optimizer
      ))

    if mem is None or mem.empty:
      # init empty pandas dataframe
      search_data = pd.DataFrame(columns=list(self.search_space.keys()) + ["score", "daily_balance"])
      with open(self.path, "w") as f:
        search_data.to_csv(f, sep=";", index=False, na_rep='nan')

      hyper.add_search(self.objective_function, self.search_space, optimizer=optimizer,
                       n_iter=self.iterations,
                       n_jobs=self.cpu_cores)
    else:
      mem.drop('daily_balance', 1, inplace=True)
      hyper.add_search(self.objective_function, self.search_space, optimizer=optimizer, memory_warm_start=mem,
                       n_iter=self.iterations,
                       n_jobs=self.cpu_cores)
    hyper.run()

https://github.com/jesse-ai/jesse/blob/hyperactive/jesse/modes/optimize_hyperactive_mode/init.py

from hyperactive.

cryptocoinserver avatar cryptocoinserver commented on May 12, 2024

I did more tests. It only happens running very few iterations on something that's not working in the sense of scores being np.nan.
(What I did during development right now). In production this probably wouldn't be a error. Only if someone stopped a optimization very early.

from hyperactive.

SimonBlanke avatar SimonBlanke commented on May 12, 2024

Hello @cryptocoinserver,

it is very difficult to reproduce the error with the code snipped above. I need a script I can run that produces the error otherwise it is just a guessing game for me. Can you provide a small and complete script?

from hyperactive.

cryptocoinserver avatar cryptocoinserver commented on May 12, 2024
import random
import numpy as np
import pandas as pd

from hyperactive import Hyperactive

# set a path to save the dataframe
path = "./search_data.csv"
search_space = {
    "n_neighbors": list(range(1, 50)),
}

# get para names from search space + the score
para_names = list(search_space.keys()) + ["score"]

# init empty pandas dataframe
search_data = pd.DataFrame(columns=para_names)
search_data.to_csv(path, index=False)


def objective_function(para):
    #score = random.choice([1.2, 2.3, np.nan])
    score = np.nan

    # you can access the entire dictionary from "para"
    parameter_dict = para.para_dict

    # save the score in the copy of the dictionary
    parameter_dict["score"] = score

    # append parameter dictionary to pandas dataframe
    search_data = pd.read_csv(path, na_values='nan')
    search_data_new = pd.DataFrame(parameter_dict, columns=para_names, index=[0])
    search_data = search_data.append(search_data_new)
    search_data.to_csv(path, index=False, na_rep='nan')

    return score


hyper0 = Hyperactive()
hyper0.add_search(objective_function, search_space, n_iter=50)
hyper0.run()


search_data_0 = pd.read_csv(path, na_values='nan')
"""
the second run should be much faster than before, 
because Hyperactive already knows most parameters/scores
"""
hyper1 = Hyperactive()
hyper1.add_search(
    objective_function, search_space, n_iter=50, memory_warm_start=search_data_0
)
hyper1.run()

from hyperactive.

cryptocoinserver avatar cryptocoinserver commented on May 12, 2024

This should help. This proofs it. If we use the commented random.choice it works. If we only return np.nan we get the error.
Thank you for your time.

from hyperactive.

cryptocoinserver avatar cryptocoinserver commented on May 12, 2024

Wow very fast. Thank you.
Sadly now I got:

============================== EXCEPTION TRACEBACK:
  File "/usr/local/bin/jesse", line 33, in <module>
    sys.exit(load_entry_point('jesse', 'console_scripts', 'jesse')())
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/home/src/jesse/jesse/__init__.py", line 379, in optimize_hyperactive
    optimize_mode_hyperactive(start_date, finish_date, optimal_total, cpu, optimizer, iterations)
  File "/home/home/src/jesse/jesse/modes/optimize_hyperactive_mode/__init__.py", line 309, in optimize_mode_hyperactive
    optimizer.run()
  File "/home/home/src/jesse/jesse/modes/optimize_hyperactive_mode/__init__.py", line 268, in run
    hyper.run()
  File "/usr/local/lib/python3.9/site-packages/hyperactive/hyperactive.py", line 201, in run
    self.results_list = run_search(self.process_infos, self.distribution)
  File "/usr/local/lib/python3.9/site-packages/hyperactive/run_search.py", line 42, in run_search
    results_list = single_process(_process_, process_infos)
  File "/usr/local/lib/python3.9/site-packages/hyperactive/distribution.py", line 10, in single_process
    results = [process_func(**search_processes_infos[0])]
  File "/usr/local/lib/python3.9/site-packages/hyperactive/process.py", line 37, in _process_
    optimizer.print_info(
  File "/usr/local/lib/python3.9/site-packages/hyperactive/optimizers.py", line 128, in print_info
    self.optimizer.print_info(*args)
  File "/usr/local/lib/python3.9/site-packages/gradient_free_optimizers/search.py", line 95, in print_info
    print_info(*args)
  File "/usr/local/lib/python3.9/site-packages/gradient_free_optimizers/print_info.py", line 103, in print_info
    _print_results(objective_function, score_best, para_best)
  File "/usr/local/lib/python3.9/site-packages/gradient_free_optimizers/print_info.py", line 66, in _print_results
    para_names = list(para_best.keys())
=========================================================================

 Uncaught Exception: AttributeError: 'NoneType' object has no attribute 'keys'

from hyperactive.

SimonBlanke avatar SimonBlanke commented on May 12, 2024

You have to update Gradient-Free-Optimizers to v0.3.1 like described above.

from hyperactive.

cryptocoinserver avatar cryptocoinserver commented on May 12, 2024

Oh indeed. Sorry, my bad. I was too fast 😄. 0.3.0.1 != 0.3.1
Now it works.

from hyperactive.

cryptocoinserver avatar cryptocoinserver commented on May 12, 2024

Thank you for the fast support.

from hyperactive.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.