Comments (2)
any suggestion to implement it by myself appreciated. tnx
from pix-plot.
@mostafa8026 Good question!
The data processing pipeline has a few steps, the first of which transforms each image into a vector. The image vectors are computed and cached (in outputs/data/image-vectors
) and so can be read directly after the first run, which should greatly expedite processing.
It's also worth noting that one can use a GPU to accelerate the creation of those image vectors. See the segments of the README on CUDA acceleration if that's an option for you.
From there, we need to project the vectors down to 2D for visualization. Right now we create a new UMAP model for this projection each time a user runs the pixplot
command. But we could cache the model from the first run and then use it for subsequent runs. The tradeoff here is between model accuracy and performance--using a cached model will make the data less expressive and could potentially refrain from displaying some patterns that are latent in the distribution, but will run faster, while creating a new model each run maximizes data expressivity but slows down processing...
If you're interested in the idea, check out the UMAP docs on projecting new data with an extant model. We have some code for saving models and loading saved models you could consult if you wanted to try using cached models when processing data. If that sounds interesting, please feel free to send a PR and we'll be happy to review and help it get accepted!
from pix-plot.
Related Issues (20)
- Dependency Requirements Issue HOT 1
- Image not displayed on iPad
- Download metadata for additional metadata fields HOT 1
- Add option to download metadata file for UMAP clusters like that for manually created clusters
- Best way to make a smaller "PixPlot" HOT 2
- metadata / custom fields HOT 1
- Bug with get_heightmap and umap layouts HOT 3
- HDBSCAN not available HOT 5
- Non-OK status: "platform is already registered with name: "METAL"
- Non-OK status: "platform is already registered with name: 'METAL'"
- Improper duplicate name validation in filter_images() HOT 5
- gbk code issue
- How would YaleDH like PixPlot to be ited?
- How would YaleDH like Pixplot to be cited?
- Associating filenames and clusters HOT 1
- Demo websites are down
- ThreeJS error when selecting lat/long view
- Sample data downloads are broken HOT 3
- Could not build wheels for scipy, HOT 6
- Tensorflow Issue? terminating due to uncaught exception of type google::protobuf::FatalException: HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pix-plot.