Comments (8)
Thanks, keep them coming :).
I'll try to add documentation soon, should be easy to do. Btw, this is a list of all evaluation measures, not only the ones the user can optimise for. Do you think we need a separate list for that?
Also, if you want to add new measures to the list, let me know: give me a name and description. We could add a service for adding new measures, but that may create some chaos :/.
Cheers,
Joaquin
On 27 Jan 2013, at 19:41, berndbischl [email protected] wrote:
http://expdb.cs.kuleuven.be/expdb/api/?f=openml.evaluation.measures
For quite a lot of the measures it is unclear what they mean exactly, or it is, but it does not make sense to ask the client to optimize them in a task.
Examples:
a) How is kohavi_wolpert_bias_squared defined exactly?b) Ho is the client supposed to optimize for "confusion_matrix"?
Solution:
Document measures, at least by providing links to definitions.
Remove the ones which do not make sense or explain to the user why they do.
—
Reply to this email directly or view it on GitHub.
from openml.
Btw, this is a list of all evaluation measures, not only the ones the user can optimise for.
Do you think we need a separate list for that?
I know, but the client programmer needs at least to know what he can possibly encounter in a task.xml.
If this is stated in the API docs somewhere, this is ok IMHO as well. Or make a 2nd list.
Also, if you want to add new measures to the list, let me know: give me a name and description
Not now, let's feature freeze as much as reasonable / doable for now.
Later on: Yes, possibly.
from openml.
Added descriptions of all evaluation measures. Currently queriable from the math_function table.
from openml.
Could you please add at least one sentence somewhere in the API docs how a description can currently be obtained?
I looked just now and could not find it...
from openml.
Sure I still get your messages :). Do you think these descriptions should be returned by the api, to show them in your system? Or should they only be looked up on the website?
In either case, you can query them on the website:
http://expdb.cs.kuleuven.be/expdb/query.php -> SQL tab -> query "select name,description from math_function"
Or programmatically (returned as json):
http://expdb.cs.kuleuven.be/expdb/expdbsqljson.php?q=select%20name,description%20from%20math_function
If you'd rather have a dedicated API calls, let me know, this will be easy to add.
On the website there will be more intuitive pages with overviews of algorithms, datasets and evaluation measures. I will do this once Jan is finished with his refactoring on monday.
from openml.
For now (and probably forever) I just want to be able to read them on the webpage, I don't care about a dedicated API call. Whether the more GUI oriented guys see this differently I don't know.
I don't know whether this is much work, but could you simply create a list / table of all results of your above mentioned queries and add a link to this tabke where the measures are covered in the API docs?
from openml.
I will fold this into the work on the website (search results, overviews, tutorials). Probably early next week.
from openml.
This is available from the website at openml.org, just add the metric in the search box or browse them under search>metrics
from openml.
Related Issues (20)
- Give back error messages when something with a dataset upload went wrong HOT 1
- Return error code 404 instead of 412 when dataset is not found
- [Server Error] HOT 1
- Issues uploading to test server (production works fine)
- Feature Request: Active Classification Task
- `Code 215: Database error. Setup search query failed - None`
- list_tasks() return task less than shown on the website HOT 1
- Dead link in README: "Citation and Honor Code" HOT 1
- Provide test datasets where labels are hidden HOT 2
- `parquet_url` incorrectly provided for non-arff formats
- Server error when creating a new task HOT 1
- Missing error message on blank fields
- Same dataset can be uploaded multiple times
- ServerError: server failed with HTTP status code 503 HOT 9
- Bucket does not exist or is private. Failed to download parquet, fallback on ARFF. HOT 2
- Cannot edit dataset HOT 1
- ElasticSearch user mapping: first name and last name should use a normalizer to allow case-insensitive sort
- JSON endpoint can return XML
- Weird task state for 361162?
- Delete HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openml.