Comments (15)
juliasilge
commented on September 27, 2024
I would double check that you have the correct package versions, especially the development version of finetune. You can use devtools::install_dev_deps() if you can cloned the TMwR repo locally. This is what I get with the current versions:
library(tidymodels)
#> Registered S3 method overwritten by 'tune':
#> method from
#> required_pkgs.model_spec parsnip
library(workflowsets)
library(rules)
#>
#> Attaching package: 'rules'
#> The following object is masked from 'package:dials':
#>
#> max_rules
library(baguette)
data(concrete, package = "modeldata")
concrete <-
concrete %>%
group_by(cement, blast_furnace_slag, fly_ash, water, superplasticizer,
coarse_aggregate, fine_aggregate, age) %>%
summarize(compressive_strength = mean(compressive_strength),
.groups = "drop")
set.seed(1501)
concrete_split <- initial_split(concrete, strata = compressive_strength)
concrete_train <- training(concrete_split)
concrete_test <- testing(concrete_split)
set.seed(1502)
concrete_folds <-
vfold_cv(concrete_train, strata = compressive_strength, repeats = 5)
normalized_rec <-
recipe(compressive_strength ~ ., data = concrete_train) %>%
step_normalize(all_predictors())
poly_recipe <-
normalized_rec %>%
step_poly(all_predictors()) %>%
step_interact(~ all_predictors():all_predictors())
linear_reg_spec <-
linear_reg(penalty = tune(), mixture = tune()) %>%
set_engine("glmnet")
nnet_spec <-
mlp(hidden_units = tune(), penalty = tune(), epochs = tune()) %>%
set_engine("nnet", MaxNWts = 2600) %>%
set_mode("regression")
mars_spec <-
mars(prod_degree = tune()) %>% #<- use GCV to choose terms
set_engine("earth") %>%
set_mode("regression")
svm_r_spec <-
svm_rbf(cost = tune(), rbf_sigma = tune()) %>%
set_engine("kernlab") %>%
set_mode("regression")
svm_p_spec <-
svm_poly(cost = tune(), degree = tune()) %>%
set_engine("kernlab") %>%
set_mode("regression")
knn_spec <-
nearest_neighbor(neighbors = tune(), dist_power = tune(), weight_func = tune()) %>%
set_engine("kknn") %>%
set_mode("regression")
cart_spec <-
decision_tree(cost_complexity = tune(), min_n = tune()) %>%
set_engine("rpart") %>%
set_mode("regression")
bag_cart_spec <-
bag_tree() %>%
set_engine("rpart", times = 50L) %>%
set_mode("regression")
rf_spec <-
rand_forest(mtry = tune(), min_n = tune(), trees = 1000) %>%
set_engine("ranger") %>%
set_mode("regression")
xgb_spec <-
boost_tree(tree_depth = tune(), learn_rate = tune(), loss_reduction = tune(),
min_n = tune(), sample_size = tune(), trees = tune()) %>%
set_engine("xgboost") %>%
set_mode("regression")
cubist_spec <-
cubist_rules(committees = tune(), neighbors = tune()) %>%
set_engine("Cubist")
nnet_param <-
nnet_spec %>%
parameters() %>%
update(hidden_units = hidden_units(c(1, 27)))
normalized <-
workflow_set(
preproc = list(normalized = normalized_rec),
models = list(SVM_radial = svm_r_spec, SVM_poly = svm_p_spec,
KNN = knn_spec, neural_network = nnet_spec)
)
normalized <-
normalized %>%
option_add(param = nnet_param, id = "normalized_neural_network")
model_vars <-
workflow_variables(outcomes = compressive_strength,
predictors = everything())
no_pre_proc <-
workflow_set(
preproc = list(simple = model_vars),
models = list(MARS = mars_spec, CART = cart_spec, CART_bagged = bag_cart_spec,
RF = rf_spec, boosting = xgb_spec, Cubist = cubist_spec)
)
with_features <-
workflow_set(
preproc = list(full_quad = poly_recipe),
models = list(linear_reg = linear_reg_spec, KNN = knn_spec)
)
all_workflows <-
bind_rows(no_pre_proc, normalized, with_features) %>%
# Make the workflow ID's a little more simple:
mutate(wflow_id = gsub("(simple_)|(normalized_)", "", wflow_id))
library(finetune)
race_ctrl <-
control_race(
save_pred = TRUE,
parallel_over = "everything",
save_workflow = TRUE
)
doParallel::registerDoParallel()
race_results <-
all_workflows %>%
workflow_map(
"tune_race_anova",
seed = 1503,
resamples = concrete_folds,
grid = 25,
control = race_ctrl
)
#> i Creating pre-processing data to finalize unknown parameter: mtry
#> Warning: The `...` are not used in this function but one or more objects were
#> passed: 'param'
autoplot(
race_results,
rank_metric = "rmse",
metric = "rmse",
select_best = TRUE
)
Created on 2021-06-18 by the reprex package (v2.0.0)
🎯 Can you run the tune_race_anova() example successfully?
from finetune.
Mayalaroz
commented on September 27, 2024
Hello Julia,
Sorry for my late answer. When I ran this command: devtools::install_dev_deps(), I had this error: Error: Could not find package root. Is your working directory inside a package?. It's the first I used it. I don't know how it works. When I ran the example:
system.time({
set.seed(2)
svm_wflow %>% tune_grid(resamples = rs, grid = svm_grid)
})
It took more time than the expected run time showed in the example. I think the cloning did not go well. How should I define the directory?
from finetune.
juliasilge
commented on September 27, 2024
Ah @Mayalaroz I would trying using these directions from tidyverse dev day for how to clone a repo locally and install the dependencies.
For the tune_race_anova() example, did it successfully run? Did you get results out that look right? It is very possible that your computer will take more time than what is shown if it is less powerful, but you can use that to problem solve whether the problem is in the version of finetune you have installed or something else.
You might consider stepping through the reprex I posted above and making sure each step looks correct and does not error.
from finetune.
Mayalaroz
commented on September 27, 2024
Hello Julia,
Thanks for answering me. I followed the instructions from you link on how to clone the repo. After running the command devtools::check(), the package finetune can't be installed:
Have you an idea where it comes from?
from finetune.
juliasilge
commented on September 27, 2024
Looks like a problem with the version of ellipsis you have; do install.packages("ellipsis") and then try again.
from finetune.
Mayalaroz
commented on September 27, 2024
Yes, I noticed the version of ellipsis is not up to date. Once it's updated, I tried again. It seems to work well except the "testthat.R"
E Running 'testthat.R' [486s] (8m 6.2s)
Running the tests in 'tests/testthat.R' failed.
Last 13 lines of output:
19. \-finetune:::get_outcome_names(object, resamples)
20. +-tune::outcome_names(x)
21. \-tune:::outcome_names.workflow(x)
22. \-tune::outcome_names(preprocessor)
-- Error (test-sa-overall.R:50:3): variable interface --------------------------
Error: object 'res' not found
Backtrace:
x
1. \-testthat::expect_equal(...) test-sa-overall.R:50:2
2. \-testthat::quasi_label(enquo(object), label, arg = "object")
3. \-rlang::eval_bare(expr, quo_get_env(quo))
[ FAIL 2 | WARN 0 | SKIP 0 | PASS 248 ]
Error: Test failures
Execution halted
√ checking for non-standard things in the check directory
√ checking for detritus in the temp directory
See
'C:/Users/XXXX/AppData/Local/Temp/5/Rtmpsfsg2v/finetune.Rcheck/00check.log'
for details.
-- R CMD check results ----------------------------------------------------------------------------------------------------------- finetune 0.0.1.9000 ----
Duration: 13m 27s
> checking tests ...
See below...
-- Test failures ---------------------------------------------------------------------------------------------------------------------------- testthat ----
> library(testthat)
> library(finetune)
Loading required package: tune
>
> test_check("finetune")
Attaching package: 'rsample'
The following object is masked from 'package:testthat':
matches
Attaching package: 'dplyr'
The following object is masked from 'package:testthat':
matches
The following objects are masked from 'package:stats':
filter, lag
The following objects are masked from 'package:base':
intersect, setdiff, setequal, union
Loading required package: Matrix
For binary classification, the first factor level is assumed to be the event.
Use the argument `event_level = "second"` to alter this as needed.
i Resamples are analyzed in a random order.
i Fold3, Repeat1: 2 eliminated; 2 candidates remain.
i Fold3, Repeat2: 0 eliminated; 2 candidates remain.
i Fold2, Repeat2: 0 eliminated; 2 candidates remain.
Attaching package: 'discrim'
The following object is masked from 'package:dials':
smoothness
> Generating a set of 1 initial parameter results
v Initialization complete
Optimizing roc_auc
Initial best: 0.89418
1 <3 new best roc_auc=0.89975 (+/-0.01128)
2 ( ) accept suboptimal roc_auc=0.89708 (+/-0.01191)
3 + better suboptimal roc_auc=0.89811 (+/-0.01136)
> Generating a set of 1 initial parameter results
v Initialization complete
Optimizing roc_auc
Initial best: 0.87370
1 <3 new best roc_auc=0.87775 (+/-0.01362)
2 ( ) accept suboptimal roc_auc=0.87503 (+/-0.01356)
3 + better suboptimal roc_auc=0.87611 (+/-0.0138)
> Generating a set of 1 initial parameter results
v Initialization complete
Optimizing roc_auc
Initial best: 0.87133
1 <3 new best roc_auc=0.88489 (+/-0.01664)
2 ( ) accept suboptimal roc_auc=0.88328 (+/-0.01707)
3 ( ) accept suboptimal roc_auc=0.88175 (+/-0.01578)
> Generating a set of 1 initial parameter results
v Initialization complete
Optimizing roc_auc
Initial best: 0.87133
1 <3 new best roc_auc=0.88489 (+/-0.01664)
2 ( ) accept suboptimal roc_auc=0.88328 (+/-0.01707)
3 ( ) accept suboptimal roc_auc=0.88175 (+/-0.01578)
> Generating a set of 1 initial parameter results
v Initialization complete
Optimizing roc_auc
Initial best: 0.71495
1 ( ) accept suboptimal roc_auc=0.69963 (+/-0.008871)
2 ( ) accept suboptimal roc_auc=0.69876 (+/-0.007773)
i Resamples are analyzed in a random order.
i Fold3, Repeat1: 1 eliminated; 4 candidates remain.
i Fold3, Repeat2: 0 eliminated; 4 candidates remain.
i Fold2, Repeat2: 0 eliminated; 4 candidates remain.
== Failed tests ================================================================
-- Failure (test-sa-overall.R:41:3): variable interface ------------------------
`{ ... }` threw an unexpected error.
Message: no applicable method for 'outcome_names' applied to an object of class "workflow_variables"
Class: simpleError/error/condition
Backtrace:
x
1. +-testthat::expect_silent(...) test-sa-overall.R:41:2
2. | \-testthat:::quasi_capture(enquo(object), NULL, evaluate_promise)
3. | +-testthat:::.capture(...)
4. | | +-withr::with_output_sink(...)
5. | | | \-base::force(code)
6. | | +-base::withCallingHandlers(...)
7. | | \-base::withVisible(code)
8. | \-rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
9. +-testthat::expect_error(...)
10. | \-testthat:::expect_condition_matching(...)
11. | \-testthat:::quasi_capture(...)
12. | +-testthat:::.capture(...)
13. | | \-base::withCallingHandlers(...)
14. | \-rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
15. +-var_wflow %>% tune_sim_anneal(cell_folds, iter = 2, control = control_sim_anneal(verbose = FALSE)) test-sa-overall.R:44:6
16. +-finetune::tune_sim_anneal(., cell_folds, iter = 2, control = control_sim_anneal(verbose = FALSE))
17. \-finetune:::tune_sim_anneal.workflow(...)
18. \-finetune:::tune_sim_anneal_workflow(...)
19. \-finetune:::get_outcome_names(object, resamples)
20. +-tune::outcome_names(x)
21. \-tune:::outcome_names.workflow(x)
22. \-tune::outcome_names(preprocessor)
-- Error (test-sa-overall.R:50:3): variable interface --------------------------
Error: object 'res' not found
Backtrace:
x
1. \-testthat::expect_equal(...) test-sa-overall.R:50:2
2. \-testthat::quasi_label(enquo(object), label, arg = "object")
3. \-rlang::eval_bare(expr, quo_get_env(quo))
[ FAIL 2 | WARN 0 | SKIP 0 | PASS 248 ]
Error: Test failures
Execution halted
1 error x | 0 warnings √ | 0 notes √
Is it a problem?
from finetune.
juliasilge
commented on September 27, 2024
Hmmm, I did not notice before that you were building finetune from source. I apologize for putting you on a path that may be more difficult than necessary!
The goal of all of this is to make sure you have installed the correct versions of packages, because the original code you asked about runs on many platforms for us correctly when the current versions of packages are installed. Can you use reprex to run the following code with session_info = TRUE? The goal of a reprex is to make it easier for us to recreate your problem so that we can understand it and/or fix it.
Your output should look about the same as mine, with the version information:
library(tidymodels)
#> Registered S3 method overwritten by 'tune':
#> method from
#> required_pkgs.model_spec parsnip
library(finetune)Created on 2021-06-25 by the reprex package (v2.0.0)
Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.0.5 (2021-03-31)
#> os macOS Big Sur 10.16
#> system x86_64, darwin17.0
#> ui X11
#> language (EN)
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz America/Denver
#> date 2021-06-25
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date lib source
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.0)
#> backports 1.2.1 2020-12-09 [1] CRAN (R 4.0.2)
#> broom * 0.7.8 2021-06-24 [1] CRAN (R 4.0.5)
#> class 7.3-19 2021-05-03 [1] CRAN (R 4.0.2)
#> cli 2.5.0 2021-04-26 [1] CRAN (R 4.0.2)
#> codetools 0.2-18 2020-11-04 [1] CRAN (R 4.0.5)
#> colorspace 2.0-1 2021-05-04 [1] CRAN (R 4.0.2)
#> crayon 1.4.1 2021-02-08 [1] CRAN (R 4.0.2)
#> DBI 1.1.1 2021-01-15 [1] CRAN (R 4.0.2)
#> dials * 0.0.9 2020-09-16 [1] CRAN (R 4.0.2)
#> DiceDesign 1.9 2021-02-13 [1] CRAN (R 4.0.2)
#> digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.2)
#> dplyr * 1.0.7 2021-06-18 [1] CRAN (R 4.0.2)
#> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.0.2)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0)
#> fansi 0.5.0 2021-05-25 [1] CRAN (R 4.0.2)
#> finetune * 0.0.1.9000 2021-06-17 [1] Github (tidymodels/finetune@717dfea)
#> foreach 1.5.1 2020-10-15 [1] CRAN (R 4.0.2)
#> fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.2)
#> furrr 0.2.3 2021-06-25 [1] CRAN (R 4.0.5)
#> future 1.21.0 2020-12-10 [1] CRAN (R 4.0.2)
#> generics 0.1.0 2020-10-31 [1] CRAN (R 4.0.2)
#> ggplot2 * 3.3.5 2021-06-25 [1] CRAN (R 4.0.5)
#> globals 0.14.0 2020-11-22 [1] CRAN (R 4.0.2)
#> glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.2)
#> gower 0.2.2 2020-06-23 [1] CRAN (R 4.0.0)
#> GPfit 1.0-8 2019-02-08 [1] CRAN (R 4.0.0)
#> gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.0)
#> highr 0.9 2021-04-16 [1] CRAN (R 4.0.2)
#> htmltools 0.5.1.1 2021-01-22 [1] CRAN (R 4.0.2)
#> infer * 0.5.4 2021-01-13 [1] CRAN (R 4.0.3)
#> ipred 0.9-11 2021-03-12 [1] CRAN (R 4.0.2)
#> iterators 1.0.13 2020-10-15 [1] CRAN (R 4.0.2)
#> knitr 1.33 2021-04-24 [1] CRAN (R 4.0.2)
#> lattice 0.20-44 2021-05-02 [1] CRAN (R 4.0.2)
#> lava 1.6.9 2021-03-11 [1] CRAN (R 4.0.2)
#> lhs 1.1.1 2020-10-05 [1] CRAN (R 4.0.2)
#> lifecycle 1.0.0 2021-02-15 [1] CRAN (R 4.0.2)
#> listenv 0.8.0 2019-12-05 [1] CRAN (R 4.0.0)
#> lubridate 1.7.10 2021-02-26 [1] CRAN (R 4.0.2)
#> magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.0.2)
#> MASS 7.3-54 2021-05-03 [1] CRAN (R 4.0.2)
#> Matrix 1.3-4 2021-06-01 [1] CRAN (R 4.0.2)
#> modeldata * 0.1.0 2020-10-22 [1] CRAN (R 4.0.2)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.0)
#> nnet 7.3-16 2021-05-03 [1] CRAN (R 4.0.2)
#> parallelly 1.26.0 2021-06-09 [1] CRAN (R 4.0.2)
#> parsnip * 0.1.6 2021-05-27 [1] CRAN (R 4.0.5)
#> pillar 1.6.1 2021-05-16 [1] CRAN (R 4.0.2)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0)
#> plyr 1.8.6 2020-03-03 [1] CRAN (R 4.0.0)
#> pROC 1.17.0.1 2021-01-13 [1] CRAN (R 4.0.2)
#> prodlim 2019.11.13 2019-11-17 [1] CRAN (R 4.0.0)
#> purrr * 0.3.4 2020-04-17 [1] CRAN (R 4.0.0)
#> R6 2.5.0 2020-10-28 [1] CRAN (R 4.0.2)
#> Rcpp 1.0.6 2021-01-15 [1] CRAN (R 4.0.2)
#> recipes * 0.1.16 2021-04-16 [1] CRAN (R 4.0.2)
#> reprex 2.0.0 2021-04-02 [1] CRAN (R 4.0.2)
#> rlang 0.4.11 2021-04-30 [1] CRAN (R 4.0.2)
#> rmarkdown 2.9 2021-06-15 [1] CRAN (R 4.0.2)
#> rpart 4.1-15 2019-04-12 [1] CRAN (R 4.0.5)
#> rsample * 0.1.0 2021-05-08 [1] CRAN (R 4.0.2)
#> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.0.2)
#> scales * 1.1.1 2020-05-11 [1] CRAN (R 4.0.0)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.0)
#> stringi 1.6.2 2021-05-17 [1] CRAN (R 4.0.2)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.0.0)
#> styler 1.4.1 2021-03-30 [1] CRAN (R 4.0.2)
#> survival 3.2-11 2021-04-26 [1] CRAN (R 4.0.2)
#> tibble * 3.1.2 2021-05-16 [1] CRAN (R 4.0.2)
#> tidymodels * 0.1.3 2021-04-19 [1] CRAN (R 4.0.2)
#> tidyr * 1.1.3 2021-03-03 [1] CRAN (R 4.0.2)
#> tidyselect 1.1.1 2021-04-30 [1] CRAN (R 4.0.2)
#> timeDate 3043.102 2018-02-21 [1] CRAN (R 4.0.0)
#> tune * 0.1.5 2021-04-23 [1] CRAN (R 4.0.2)
#> utf8 1.2.1 2021-03-12 [1] CRAN (R 4.0.2)
#> vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.0.2)
#> withr 2.4.2 2021-04-18 [1] CRAN (R 4.0.2)
#> workflows * 0.2.2 2021-03-10 [1] CRAN (R 4.0.3)
#> workflowsets * 0.0.2 2021-04-16 [1] CRAN (R 4.0.2)
#> xfun 0.24 2021-06-15 [1] CRAN (R 4.0.2)
#> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0)
#> yardstick * 0.0.8 2021-03-28 [1] CRAN (R 4.0.2)
#>
#> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/libraryWhat we're looking for is that you have the same updated CRAN versions of tidymodels packages as me and especially the current GitHub version of finetune. Thanks! 🙌
from finetune.
Mayalaroz
commented on September 27, 2024
Hello Julia,
This is what I obtained:
library(tidymodels); library(finetune)
#> Registered S3 method overwritten by 'tune':
#> method from
#> required_pkgs.model_spec parsnipCreated on 2021-06-28 by the reprex package (v2.0.0)
Session info
sessioninfo::session_info()
#> - Session info ---------------------------------------------------------------
#> setting value
#> version R version 4.0.5 (2021-03-31)
#> os Windows Server x64
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_United States.1252
#> ctype English_United States.1252
#> tz Europe/Paris
#> date 2021-06-28
#>
#> - Packages -------------------------------------------------------------------
#> package * version date lib source
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.5)
#> backports 1.2.1 2020-12-09 [1] CRAN (R 4.0.3)
#> broom * 0.7.6 2021-04-05 [1] CRAN (R 4.0.5)
#> class 7.3-18 2021-01-24 [2] CRAN (R 4.0.5)
#> cli 2.5.0 2021-04-26 [1] CRAN (R 4.0.5)
#> codetools 0.2-18 2020-11-04 [2] CRAN (R 4.0.5)
#> colorspace 2.0-2 2021-06-24 [1] CRAN (R 4.0.5)
#> crayon 1.4.1 2021-02-08 [1] CRAN (R 4.0.5)
#> DBI 1.1.1 2021-01-15 [1] CRAN (R 4.0.5)
#> dials * 0.0.9 2020-09-16 [1] CRAN (R 4.0.5)
#> DiceDesign 1.9 2021-02-13 [1] CRAN (R 4.0.5)
#> digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.5)
#> dplyr * 1.0.7 2021-06-18 [1] CRAN (R 4.0.5)
#> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.0.5)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.5)
#> fansi 0.5.0 2021-05-25 [1] CRAN (R 4.0.5)
#> finetune * 0.0.1 2020-11-20 [1] CRAN (R 4.0.5)
#> foreach 1.5.1 2020-10-15 [1] CRAN (R 4.0.5)
#> fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.5)
#> furrr 0.2.2 2021-01-29 [1] CRAN (R 4.0.5)
#> future 1.21.0 2020-12-10 [1] CRAN (R 4.0.5)
#> generics 0.1.0 2020-10-31 [1] CRAN (R 4.0.5)
#> ggplot2 * 3.3.5 2021-06-25 [1] CRAN (R 4.0.5)
#> globals 0.14.0 2020-11-22 [1] CRAN (R 4.0.3)
#> glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.5)
#> gower 0.2.2 2020-06-23 [1] CRAN (R 4.0.3)
#> GPfit 1.0-8 2019-02-08 [1] CRAN (R 4.0.5)
#> gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.5)
#> highr 0.9 2021-04-16 [1] CRAN (R 4.0.5)
#> htmltools 0.5.1.1 2021-01-22 [1] CRAN (R 4.0.5)
#> infer * 0.5.4 2021-01-13 [1] CRAN (R 4.0.5)
#> ipred 0.9-11 2021-03-12 [1] CRAN (R 4.0.5)
#> iterators 1.0.13 2020-10-15 [1] CRAN (R 4.0.5)
#> knitr 1.33 2021-04-24 [1] CRAN (R 4.0.5)
#> lattice 0.20-41 2020-04-02 [2] CRAN (R 4.0.5)
#> lava 1.6.9 2021-03-11 [1] CRAN (R 4.0.5)
#> lhs 1.1.1 2020-10-05 [1] CRAN (R 4.0.5)
#> lifecycle 1.0.0 2021-02-15 [1] CRAN (R 4.0.5)
#> listenv 0.8.0 2019-12-05 [1] CRAN (R 4.0.5)
#> lubridate 1.7.10 2021-02-26 [1] CRAN (R 4.0.5)
#> magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.0.5)
#> MASS 7.3-53.1 2021-02-12 [2] CRAN (R 4.0.5)
#> Matrix 1.3-2 2021-01-06 [2] CRAN (R 4.0.5)
#> modeldata * 0.1.0 2020-10-22 [1] CRAN (R 4.0.5)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.5)
#> nnet 7.3-15 2021-01-24 [2] CRAN (R 4.0.5)
#> parallelly 1.26.0 2021-06-09 [1] CRAN (R 4.0.5)
#> parsnip * 0.1.6 2021-05-27 [1] CRAN (R 4.0.5)
#> pillar 1.6.1 2021-05-16 [1] CRAN (R 4.0.5)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.5)
#> plyr 1.8.6 2020-03-03 [1] CRAN (R 4.0.5)
#> pROC 1.17.0.1 2021-01-13 [1] CRAN (R 4.0.5)
#> prodlim 2019.11.13 2019-11-17 [1] CRAN (R 4.0.5)
#> purrr * 0.3.4 2020-04-17 [1] CRAN (R 4.0.5)
#> R6 2.5.0 2020-10-28 [1] CRAN (R 4.0.5)
#> Rcpp 1.0.6 2021-01-15 [1] CRAN (R 4.0.5)
#> recipes * 0.1.16 2021-04-16 [1] CRAN (R 4.0.5)
#> reprex 2.0.0 2021-04-02 [1] CRAN (R 4.0.5)
#> rlang 0.4.11 2021-04-30 [1] CRAN (R 4.0.5)
#> rmarkdown 2.9 2021-06-15 [1] CRAN (R 4.0.5)
#> rpart 4.1-15 2019-04-12 [2] CRAN (R 4.0.5)
#> rsample * 0.1.0 2021-05-08 [1] CRAN (R 4.0.5)
#> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.0.5)
#> scales * 1.1.1 2020-05-11 [1] CRAN (R 4.0.5)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.5)
#> stringi 1.6.2 2021-05-17 [1] CRAN (R 4.0.5)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.0.5)
#> styler 1.4.1 2021-03-30 [1] CRAN (R 4.0.5)
#> survival 3.2-10 2021-03-16 [2] CRAN (R 4.0.5)
#> tibble * 3.1.2 2021-05-16 [1] CRAN (R 4.0.5)
#> tidymodels * 0.1.3 2021-04-19 [1] CRAN (R 4.0.5)
#> tidyr * 1.1.3 2021-03-03 [1] CRAN (R 4.0.5)
#> tidyselect 1.1.1 2021-04-30 [1] CRAN (R 4.0.5)
#> timeDate 3043.102 2018-02-21 [1] CRAN (R 4.0.5)
#> tune * 0.1.5 2021-04-23 [1] CRAN (R 4.0.5)
#> utf8 1.2.1 2021-03-12 [1] CRAN (R 4.0.5)
#> vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.0.5)
#> withr 2.4.2 2021-04-18 [1] CRAN (R 4.0.5)
#> workflows * 0.2.2 2021-03-10 [1] CRAN (R 4.0.5)
#> workflowsets * 0.0.2 2021-04-16 [1] CRAN (R 4.0.5)
#> xfun 0.24 2021-06-15 [1] CRAN (R 4.0.5)
#> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.5)
#> yardstick * 0.0.8 2021-03-28 [1] CRAN (R 4.0.5)
#>
#> [1] C:/Users/qcadmin/Documents/R/win-library/4.0
#> [2] C:/Program Files/R/R-4.0.5/libraryI noticed that my version of finetune still comes from CRAN and not from GitHub, which is my installation was not well done. Let me know if you need extra info :)
from finetune.
juliasilge
commented on September 27, 2024
OK @Mayalaroz you need to install finetune from GitHub:
devtools::install_github("tidymodels/finetune")from finetune.
Mayalaroz
commented on September 27, 2024
Hello Julia,
I installed the finetune package from Github. I ran the example again:
library(finetune)
race_ctrl <-
control_race(
save_pred = TRUE,
parallel_over = "everything",
save_workflow = TRUE
)
system.time({race_results <-
all_workflows %>%
workflow_map(
"tune_race_anova",
seed = 1503,
resamples = concrete_folds,
grid = 25,
control = race_ctrl
)})
The output is:
i Creating pre-processing data to finalize unknown parameter: mtry
! Fold04, Repeat1: internal: A correlation computation is required, but `estimate` is constant and has 0 standard deviation, resulting in a divide by 0 error. `NA` wi...
! Fold04, Repeat1: internal: A correlation computation is required, but `estimate` is constant and has 0 standard deviation, resulting in a divide by 0 error. `NA` wi...
! Fold06, Repeat1: internal: A correlation computation is required, but `estimate` is constant and has 0 standard deviation, resulting in a divide by 0 error. `NA` wi...
! Fold06, Repeat1: internal: A correlation computation is required, but `estimate` is constant and has 0 standard deviation, resulting in a divide by 0 error. `NA` wi...
! Fold10, Repeat1: internal: A correlation computation is required, but `estimate` is constant and has 0 standard deviation, resulting in a divide by 0 error. `NA` wi...
! Fold10, Repeat1: internal: A correlation computation is required, but `estimate` is constant and has 0 standard deviation, resulting in a divide by 0 error. `NA` wi...
user system elapsed
5924.17 21.53 5971.81
Warning message:
The `...` are not used in this function but one or more objects were passed: 'param'
I have less warnings since I took the package from Github. Are these warnings a problem? The tune_race_anova function still takes 1h30 to complete it, which is the same amount of time than the grid search.
from finetune.
topepo
commented on September 27, 2024
In regards to the warning, those are fine.
from finetune.
juliasilge
commented on September 27, 2024
Those warnings from the example are probably from a situation where the model predicts a single value for all observations, so not a problem, like Max says.
If you'd like to get to the bottom of comparing the times, we do that by comparing:
full_results_time <-
system.time(
grid_results <-
all_workflows %>%
workflow_map(seed = 1503, resamples = concrete_folds, grid = 25,
control = grid_ctrl, verbose = TRUE)
)to:
race_results_time <-
system.time(
race_results <-
all_workflows %>%
workflow_map("tune_race_anova",
seed = 1503, resamples = concrete_folds, grid = 25,
control = race_ctrl)
)It's almost 4 times faster, because it doesn't train as many models.
from finetune.
Mayalaroz
commented on September 27, 2024
Hello ,
I run again the example and I get the following results,
With the grid search:
full_results_time
user system elapsed
31042.28 142.73 31387.05
With the racing approach:
race_results_time
user system elapsed
9518.37 38.97 9556.34
Indeed, the racing approach is 3.26 faster than the grid search. However, these methods take much more time than one hour and a half but I assume that it depends on the number of worker nodes and also the IT setup (I work on a remote desktop with a shared server). Thanks for the help you provided to me.
from finetune.
juliasilge
commented on September 27, 2024
Great @Mayalaroz! I really glad you were able to run the examples. 👍 Let us know if you have further questions in the future.
from finetune.
github-actions
commented on September 27, 2024
This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.
from finetune.
Related Issues (20)
- Error using tune_race_anova() with spatialsample objects and parallel computing HOT 3
- integer x axis ticks for plot_race() HOT 1
- note initial resampling when `verbose_elim = TRUE`
- audit alt text
- type check inputs
- transition to cli errors
- name the evaluation time for survival models
- Improve checking of metric used in `show_best.tune_race()` HOT 2
- `tune_race_anova()` throws same warning twice
- `tune_race_win_loss()` fails to handle `eval_time` when it's provided but not needed
- Feature: collect_race() or collect_metrics() for the race HOT 2
- Pass call from top-level functions
- revdepchecks for finetune v1.1.1
- Release finetune 1.2.0
- Update metric info on racing pages
- better error when passing a preprocessor with a workflow
- transition from deprecated `map_dfr()` in `autoplot()` internals
- Error that occurs when using the custom metric during parallel hyperparameter tuning. HOT 6
- Groupwise metrics not working with finetune HOT 1
- error with simulated annealing and censored regression
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from finetune.