Comments (7)
use Overpass or other approach to getting OSM data, instead of hacking up PBF extracts with Osmium
@andrewljohnson have you considered OSM QA Tiles?
from deeposm.
@anandthakker I'm probably wrong, but I decided it doesn't really help me to use "tiled" data, because my source imagery isn't projected to Mercator, or tiled into a TMS pyramid?
When I did my first run at this, I used Mapzen vector tiles, plus TMS imagery tiles (since they align). But when I switched to NAIPs, it seemed to make more sense (less code) to clip some line strings to the bounds of my NAIPs or arbitrary tiles.
So that's also why I think my end solution is a planet DB with an API, and maybe that API is Overpass, or something simple I just cook up for this use case?
from deeposm.
Ah -- yeah, I see what you mean. I went the tiled route for skynet-data
basically because between existing tiled satellite source (Mapbox Satellite, DG, etc.) and OSM QA tiles, I figured I could skip a lot of the work. Since you've already handled chopping up the NAIP images, I agree that using tiles provides less of a benefit... Although, I guess one upside to going the tiled route would be that maybe it would be easier, in the future, to swap in a different imagery source (without your having to host/maintain/process it as much). Might not be worth it if you don't see yourself going that route, though
from deeposm.
Have you considered running the analysis at Digital Ocean and pushing the results into S3 from there? The compute at DO is significantly cheaper than at AWS.
from deeposm.
@brandongalbraith The analysis runs at home on a Linux box I have.
also: I might try out some Google ML infrastructure, just got my invite yesterday.
from deeposm.
Sorry about that! I inferred from above:
Actual work includes:
move the analysis to AWS, run on a cycle
that the analysis was planned to run in AWS. Happy to help break everything apart to scale it out, my day gig is devops/infrastructure.
from deeposm.
@brandongalbraith i realized that after I answered too fast :)
I guess the right answer is I hadn't much thought about it.
from deeposm.
Related Issues (20)
- todos in progress
- INSTALL ISSUES HOT 5
- I think there is errata in "false positives" words HOT 1
- Deeposm.org UI: Add "Flag as fixed" control HOT 1
- Posssible to work with Google Earth Imagery HOT 2
- Fail to run train_neural_net.py HOT 3
- Paths issues HOT 1
- Getting Error while training the neural net !!! HOT 7
- gdal version mismatch? HOT 4
- Getting warning while Training neural net HOT 13
- Cannot download NAIP images on s3 using boto HOT 3
- Getting the import error HOT 2
- Pillow version is low, cause error when saving JPEG files HOT 2
- [Suggest] Can you develop new function that generates website content?
- Clock in docker is wrong, so aws doesn't allow download HOT 1
- Is web mercator transform correct? HOT 1
- Simple offline prediction
- Training stops with 'NoneType' object has no attribute 'name' HOT 1
- Pretrained models
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deeposm.