Comments (29)
The Elevation API is not yet open to the public/research. May happen in a few months. Can't really count on it.
from kdb.
In the mean time, this is still open: https://maps.google.com/cbk?output=xml&ll=45.508457,-73.532738&dm=1
And we can generate such an image using https://github.com/cc-ai/floods-gans/blob/master/various_scripts/depth_map.py
Now open questions:
- can we map an exact StreetView position to such a query?
- can we get the original image from which this is computed?
- can we specify pitch, camera view, angle and so on
- is the code fast enough + scalable?
- how to compute a water plane from this data?
- how to embed said plane into original image?
Using GANs to simulate how a house would look like, had it suffered from a flood - cc-ai/floods-gans
from kdb.
Before going into matching depth-map and street view images automatically, I would like to know if you found GPS coordinates where you managed to match it qualitatively playing on streetview? If so could you please communicate them? I can't find a single place where I can match it visually.
from kdb.
I agree I have not and it is an open question since the API is not documented (or is it?)
from kdb.
@DatCorno on the backend we are using the same API or it's different?
from kdb.
Reconstructing the spherical img
There are some steps to automatically reconstruct the spherical image relative to a pano_id that should be developped but in the meantime this App do the work for us.
Reconstructing 3D point cloud
Here is a notebook colab to explore depth-map where I tested if we can superpose depth map to the panoramic img without success and where I translated some medium advices to generate the 3D point cloud (there is work to do before having a height in meter)
from kdb.
Ok so this is not good enough for us to work with right? This API is still open but basically it's not as informative as we expected?
from kdb.
I would not say that, I believe we should write our own code following the steps in this paper explaining how to reconstruct the panorama ourself from the tiles and see if it match the depth map: There is a lot of potential in this approach.
from kdb.
@gcosne how long it will take in your opinion, if we write our own code?
from kdb.
Somehow I find a panorama that almost match the depth map :
Remark : I discover one thing: Height of the Lidar is about 2.5 meters. Using the formula Normalized/minimum(height) * Lidar-Height allow to translate the normalized coordinates in meters.
Then I tried to" flood "the image until a certain height but I think the lidar doesn't have a lot of points below 1.5m. Hence the segmentation map generated is good when height >2.m but get sparse when we go below 1.7m.
Flood up to 1.7m
from kdb.
Pas mal du tout. La question qui nous reste sur les bras c'est: est-ce qu'on peut travailler avec ces panoramiques? i.e. est-ce que c'est suffisamment "relatable" pour en faire qqch et est-ce qu'on peut l'avoir pour n'importe quel endroit / si non lesquels?
Il faut en parler à Alice et mettre au point un premier groupe de testing utilisateur pour se faire un avis, imho
from kdb.
oups French, I meant: this is not bad at all! we ought to research that with users to understand if that's usable / relatable for our purpose + figure out if there are restrictions on localizations for which we can get this depth map + original panorama
from kdb.
from kdb.
very interesting, good job, so you found the magic number I was asking :)
for the flooding part maybe we can find another technick, like reconstructing the scene in 3D and flood afterwards ... but not sure for now, maybe too heavy, we can think more about it
from kdb.
Testing with depth estimation : Using very light single image depth estimator rather than 'google api', we almost have usable masks.
To Be Done:
- Identify if we have region where google api works for us.
- Calibrating depth estimator with google api.
Qualitative Results flooding up to 1.5m :
from kdb.
well, the flood part doesn't work ...
btw, what about people on street view images? should we "drown" them?
from kdb.
from kdb.
Update :
I translated DownloadPanorama C++ function from this incredible project in python.
We can now automatically extract xml information and generate both depthmap and panoramic images within a script. I think I will explore more of PaulWagener project, I sent him an email.
Cropping the image so they are the same size rather than resizing them actually work better. Not perfect yet but I will fix that soon.
from kdb.
from kdb.
Not yet but I do trust it will, I have to play with yaw heading.
from kdb.
from kdb.
Update : We can almost call it a success.
I said almost because the size of the tiles are not consistent, sometimes the 360° equirectangular image is already built (no overlapping between tiles), sometimes it's not. The good news are that we can identify it by the amount of black padding in the image.
360° panorama without overlapping More than 360° panoramic image
from kdb.
from kdb.
I assume so, nothing of an issue here but just a thing to be aware.
Next steps :
- Finish the work in low resolution.
- Create the stitching function for already built panoramic images in higher resolution.
from kdb.
from kdb.
cool, good job ;)
from kdb.
Done: Low and High resolution are working. I will clean and commit the code soon.
Next steps : Make it work with the 3D.
- Flood the ground.
- Flood a certain height.
from kdb.
from kdb.
3D reconstruction: There is an echo around the car that makes the ground plane estimation tricky
from kdb.
Related Issues (20)
- React to issues/PRs you have seen! HOT 1
- Total Variational Regularization for MaskGen
- Mask-conditioned encoder reduced water quality HOT 2
- Hardcoded domain adaptation classifier in MUNIT/SPADE HOT 4
- Next steps to improve SPADE + AdaIN architecture
- Tianyu's TODO list from April 29th
- Ground/Flood masks TO DO List HOT 9
- Ground/Flood masks postprocessing HOT 1
- Masker Next Ideas HOT 1
- Self-Supervised Relative Depth Learning for Urban Scene Understanding
- High-Fidelity Image Generation With Fewer Labels
- Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game
- Organising and documenting stuff HOT 2
- Segmentation maps for real dataset HOT 3
- Using Cityscapes Validation to Test ADVENT HOT 3
- Comparing the ADVENT pretrained with ADVENT non-pretrained HOT 1
- Exploring other extreme weather events HOT 2
- Omnigan current tasks
- Current OmniGAN tasks HOT 1
- Week-long Climate Hackathon HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kdb.