zuliani99 / automl-benchmark Goto Github PK

I don't know how but this two does not save the pipelines in the result_class

clines the repository , build the docker image ( i hade to add the README.md file to it for it to work) , and when it ran i got:

RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xe

build docker image and run

my machine:

• OS: MaxOS
• Browser chrome
• Version latest

After clicking the button the application does not open the modal for showing the pipelines of all algorithms for the specific dataset

Dataframe tested: 10.csv

Describe the bug
Error when trying to comapre two past openml benchmarks

Depends on the measure of the Kaggle competition. The score tables have to adapt and insert or not the leader's score.
Analise also the effect of doing a Kaggle benchmark on multiple competition

Referr to this issue

Make attention to line 165 of utils.py:

if (
  collections.Counter(cls) == collections.Counter(dfs_class)
  and collections.Counter(reg) == collections.Counter(dfs_reg)
  and collections.Counter(pip) != collections.Counter(options_end)
)

This allow comparison only if there are the same classification and regression dataframe and if the end lifetime is different, note that now the end lifetime is the actual time consumed for computation. Which is also rounded to 3 significant digits.

So decide in a short time what logic to implement in this part of the code