Comments (2)
You are getting this error because your custom metric ccc_with_bias()
returned a tibble with .metric
value of ccc
where it should have returned a value of ccc_with_bias()
.
# What it returns
ccc_with_bias(solubility_test, solubility, prediction)
#> # A tibble: 1 × 3
#> .metric .estimator .estimate
#> <chr> <chr> <dbl>
#> 1 ccc standard 0.937
# What it should return
ccc_with_bias(solubility_test, solubility, prediction)
#> # A tibble: 1 × 3
#> .metric .estimator .estimate
#> <chr> <chr> <dbl>
#> 1 ccc_with_bias standard 0.937
I modified ccc_with_bias()
for you, and now it works as it should.
library(tidymodels)
library(finetune)
data(ames)
ames <- mutate(ames, Sale_Price = log10(Sale_Price))
set.seed(502)
ames_split <- initial_split(ames, prop = 0.80, strata = Sale_Price)
ames_train <- training(ames_split)
ames_test <- testing(ames_split)
ames_folds <- vfold_cv(ames_train, v = 10)
ames_rec <-
recipe(Sale_Price ~ Neighborhood + Gr_Liv_Area + Year_Built + Bldg_Type +
Latitude + Longitude, data = ames_train) %>%
step_log(Gr_Liv_Area, base = 10) %>%
step_other(Neighborhood, threshold = 0.01) %>%
step_dummy(all_nominal_predictors()) %>%
step_interact( ~ Gr_Liv_Area:starts_with("Bldg_Type_") ) %>%
step_ns(Latitude, Longitude, deg_free = 20)
rf_model <-
rand_forest(trees = tune()) %>%
# rand_forest(trees = 1000) %>%
set_engine("ranger") %>%
set_mode("regression")
rf_wflow <-
workflow() %>%
add_formula(
Sale_Price ~ Neighborhood + Gr_Liv_Area + Year_Built + Bldg_Type +
Latitude + Longitude) %>%
add_model(rf_model)
grid <- parameters(trees(c(10, 100))) %>%
grid_max_entropy(size = 10)
ccc_with_bias <- function(data, truth, estimate, na_rm = TRUE, case_weights = NULL, ...) {
res <- ccc(
data = data,
truth = !!rlang::enquo(truth),
estimate = !!rlang::enquo(estimate),
# set bias = TRUE
bias = FALSE,
na_rm = na_rm,
case_weights = !!rlang::enquo(case_weights),
...
)
res$.metric <- "ccc_with_bias"
res
}
# Use `new_numeric_metric()` to formalize this new metric function
ccc_with_bias <- new_numeric_metric(ccc_with_bias, "maximize")
model_metric <- metric_set(ccc_with_bias)
tune_res_anova <- tune_race_anova(
rf_wflow,
ames_folds,
grid = grid,
metrics = model_metric
)
Created on 2023-05-26 with reprex v2.0.2
from finetune.
As this issue hasn't seen any activity in a while, I'm going to go ahead and close. Thanks for the issue!
from finetune.
Related Issues (20)
- [Feature Request] Is it possible to include the genetic algorithm to tune hyper-parameters in Tidymodels? HOT 5
- Upkeep for finetune HOT 1
- Release finetune 1.1.0 HOT 1
- Error using tune_race_anova() with spatialsample objects and parallel computing HOT 3
- integer x axis ticks for plot_race() HOT 1
- note initial resampling when `verbose_elim = TRUE`
- audit alt text
- type check inputs
- transition to cli errors
- name the evaluation time for survival models
- Improve checking of metric used in `show_best.tune_race()` HOT 2
- `tune_race_anova()` throws same warning twice
- `tune_race_win_loss()` fails to handle `eval_time` when it's provided but not needed
- Feature: collect_race() or collect_metrics() for the race HOT 2
- Pass call from top-level functions
- revdepchecks for finetune v1.1.1
- Release finetune 1.2.0
- Update metric info on racing pages
- better error when passing a preprocessor with a workflow
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from finetune.