Comments (6)
Evaluating the density of arbitrary points would be absolutely a nice feature.
You mentioned "high-dimensional interpolation on equidistant grids". To my knowledge, one easy and fast solution is to use scipy.ndimage.map_coordinates
.
FYI, in case it might help, I have written a wrapper to this function for my own use which is available at https://github.com/syrte/handy/blob/master/interpolate.py
from kdepy.
I'm not against it at all.
I guess we would need to evaluate on arbitrary grid points, in order to evaluate a score on the data. But FFTKDE generates equidistant grid points and throws away the original data. This would mean doing high-dimensional interpolation. I haven't looked into how hard or easy that might be, or if there is some other clever way of doing it.
from kdepy.
Agree that the scoring function may be non-trivial. But adding all other functionality would allow users to write their own scoring functions, which may be reasonable for some cases.
from kdepy.
Agreed. The sklearn API is very nice in it's own right, and a lot of people are ac customized to it and to some degree expect estimators to follow the API. I imitated it, but the APIs differ a little bit.
I'll leave the issue open. Up for grabs for anyone. I might look into it when (if?) I have time in the future.
from kdepy.
Hi
I just sent a PR with a proposal for a solution to this issue, based on a grid_search_cv
function which is a simplified version of sklearn
's GridSearchCV
class. Since FFTKDE doesn't allow evaluating on a arbitrary grid, the method only works for NaiveKDE and TreeKDE. Of course the performance won't be as good, but for my work this is good enough, and of course you can first optimize BW with a TreeKDE and the use it to create a FFTKDE.
In the past I tried implementing an integration directly with sklearn
, but I found 2 problems:
sklearn
doesn't takes into account test weights. This issue was pointed out and is being discussed here, so possibly will be fixed some day.- By the way
sklearn
makes cross validation, it can't work with variable bandwidths. As I understood, they first clone the model, which would copy the full variable BW, and then fit it with each train fold, wich is a subset of the original dataset, resulting in models with more bandwidths than data points.
The first problem could be tolerated, but I couldn't think of a solution for the second problem, and doesn't look like sklearn
will provide a solution. My conclusion was that the most reasonable solution was to implement the cross validation by hand (using only numpy
). It isn't that hard if you restrict the model you pass to the grid search: instead of implementing a get_params
method, I just clone a KDE model with model.__class__(model.kernel, model.bw, model.norm)
(or replace model.bw
by each BW of the grid).
from kdepy.
might be of interest: scikit-learn/scikit-learn#26896
from kdepy.
Related Issues (20)
- density estimation on 4D data HOT 3
- Grid definitions for multivariant KDE HOT 1
- gird points HOT 3
- Cytopy needs KDEpy version 1.0.10, but pip says it doesn't exist. HOT 2
- Saving NaiveKDE-objects with pickle HOT 1
- Installation fails on M1 mac HOT 16
- How to cite this package HOT 3
- Grid must be sorted error HOT 2
- Custom grid in FFTKDE of 3D (x, y, z) data HOT 1
- Return the value of bandwidth defined automatically with ISJ method HOT 3
- python3.11 compatibility
- Citation for your implementation HOT 2
- Add a JIT compiler? (feature request) HOT 1
- Including KDEpy in BSD licensed project HOT 4
- Some problems HOT 1
- Is it possible to fit and save the state of the FFTKDE? HOT 4
- Can `bw_selection.py` return a value when root finding did not converge? HOT 1
- Installing CytoPy HOT 1
- how to get pseudo-uniform samples HOT 1
- Unable to solve for support numerically. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kdepy.