adefazio / classifierplots Goto Github PK
View Code? Open in Web Editor NEWGenerates a visualization of binary classifier performance as a grid of diagonstic plots with just one function call
License: Other
Generates a visualization of binary classifier performance as a grid of diagonstic plots with just one function call
License: Other
The manuals are poorly written, may I say, to the point of being non-existent...
In the calibration.R, qbeta function was used to calculate true probability in the calibration_plot, such as "qbeta(c(llb=0.025, lb=0.25, y=0.5, ub=0.75, uub=0.965), 0.5+positive, 0.5+bucket_size-positive)". Sorry I can't understand that. Could you please provide some expanations or some papers.
Thank you!
y_test_class = [0,0,0,0,0]
prob_class = [0.01,0.02,.45,0.36,0.001]
classifierplots(y_test_class,prob_class)
"""
Error in check_classifier_input_and_init(test.y, pred.prob) :
test.y had more than 2 unique values: 1
"""
An initial look into this shows a few issues. The tar
command references a folder in a directory structure that's different to the R.template assumed layout. Also, library(devtools)
(as per R.profile
) doesn't exist in CI.
Does this repo require artefacts to be uploaded
Or
Should the scripts be corrected to work propery
Hi, I just installed the CRAN version.
I train a classifier using various packages and when I finally want to call classifierplots
I get the error
alpha level NA, not in [0,1]
I can not share my whole code and data here. If I run the base example after doing my calculations I get the error too.
So there might be an incompatibility with the packages already loaded.
Do you have any idea if you look at the list of packages loaded.
Is there anything I can check?
Thank you!
library(classifierplots)
classifierplots(example_predictions$test.y, example_predictions$pred.prob)
and got the error:
[1] "Calculating AUC ..."
[1] "(AUC) Sorting data ..."
[1] "(AUC) Calculating ranks ..."
[1] "AUC: 90.5603213507625"
[1] "Bootstrapping ROC curves"
[1] "Eval AUC"
[1] "Producing ROC plot"
[1] "Generating score density plot"
Error in grDevices::rgb(col[1, ], col[2, ], col[3, ], alpha) :
alpha level NA, not in [0,1]
In addition: Warning message:
In grDevices::rgb(col[1, ], col[2, ], col[3, ], alpha) :
NAs introduced by coercion
my sessionInfo()
is
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252attached base packages:
[1] stats graphics grDevices utils datasets methods baseother attached packages:
[1] earth_4.4.7 plotmo_3.2.0 TeachingDemos_2.10 plotrix_3.6-3 classifierplots_1.3.2
[6] data.table_1.10.4 mlr_2.10 ParamHelpers_1.10 reshape_0.8.6 fastICA_1.2-0
[11] corrplot_0.77 e1071_1.6-8 dplyr_0.5.0 purrr_0.2.2 readr_1.0.0
[16] tidyr_0.6.1 tibble_1.2 ggplot2_2.2.1 tidyverse_1.1.1loaded via a namespace (and not attached):
[1] httr_1.2.1 jsonlite_1.1 splines_3.3.1 foreach_1.4.3 modelr_0.1.0 gtools_3.5.0
[7] LiblineaR_1.94-2 assertthat_0.1 stats4_3.3.1 deepnet_0.2 backports_1.0.3 lattice_0.20-33
[13] quantreg_5.29 checkmate_1.8.2 rvest_0.3.2 minqa_1.2.4 colorspace_1.3-2 Matrix_1.2-6
[19] plyr_1.8.4 psych_1.6.12 broom_0.4.1 SparseM_1.74 haven_1.0.0 caret_6.0-73
[25] corpcor_1.6.8 scales_0.4.1 parallelMap_1.3 gdata_2.17.0 sda_1.3.7 fdrtool_1.2.15
[31] MatrixModels_0.4-1 lme4_1.1-12 mgcv_1.8-12 car_2.1-3 xgboost_0.6-4 pacman_0.4.1
[37] ROCR_1.0-7 nnet_7.3-12 lazyeval_0.2.0 pbkrtest_0.4-6 mnormt_1.5-5 survival_2.39-4
[43] magrittr_1.5 readxl_0.1.1 nlme_3.1-128 MASS_7.3-45 gplots_3.0.1 forcats_0.2.0
[49] xml2_1.1.0 foreign_0.8-66 class_7.3-14 tools_3.3.1 hms_0.3 BBmisc_1.10
[55] stringr_1.1.0 munsell_0.4.3 entropy_1.2.1 caTools_1.17.1 grid_3.3.1 nloptr_1.0.4
[61] iterators_1.0.8 labeling_0.3 bitops_1.0-6 gtable_0.2.0 ModelMetrics_1.1.0 codetools_0.2-14
[67] DBI_0.5-1 reshape2_1.4.2 R6_2.1.3 gridExtra_2.2.1 lubridate_1.6.0 KernSmooth_2.23-15
[1] "Calculating AUC ..."
[1] "(AUC) Sorting data ..."
[1] "(AUC) Calculating ranks ..."
[1] "AUC: 72.3611244722356"
[1] "Bootstrapping ROC curves"
[1] "Eval AUC"
[1] "Producing ROC plot"
[1] "Generating score density plot"
Error in alpha * 255 : non-numeric argument to binary operator
Classifier plots roc curve drawing appears to have visual artefacts when scores aren't spread out well (there are a large number of instances with the same score, leading to empty deciles).
I was linked to this by @BrendanVR. Nice package! I was wondering if you would consider exporting the stuff in metrics.R? There is a real lack of support for scoring binary classifiers in modelr
and these functions would be a good start! Unless you think they belong in a separate package for this purpose?
The individual pieces that make up a classifierplot
object render fine when you add them in an Rmd chunk, but when you include a the actual classifierplots
function, it just prints out the message.
This works
density_plot(test_dat$Outcome,
test_dat$PredEns)
This does not
grobEns <- classifierplots(test_dat$Outcome,
test_dat$PredEns)
grobEns
And neither does this
gridExtra::grid.arrange(grobEns,
top=textGrob("Ensemble model",
gp=gpar(fontsize=16,font=1)))
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.