Giter Club home page Giter Club logo

supersplat's Introduction

SuperSplat

SuperSplat is a free and open source tool for inspecting and editing 3D Gaussian Splats. It is built on web technologies and runs in the browser so there's nothing to download and install.

supersplat

See https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/ for more information on gaussian splats.

A live version of this tool is available at: https://playcanvas.com/supersplat/editor

Loading Scenes

To load a Gaussian splat PLY file, drag & drop it onto the application page.

Alternatively, use the File menu at the top left of the screen and choose "Open".

SuperSplatFileMenu

If you disable the "Load all PLY data" option before loading the file, then the PLY data not required by the editor is excluded (for example the spherical harmonic data). This can save on browser memory.

Editing Scenes

Once a PLY file is loaded you can use the selection tools to modify splat selection and then delete splats from the scene.

You can also reorient the scene using the SCENE Position/Rotation/Scale controls.

Saving Results

Once you're done editing the scene, click the Export -> "Ply file" button to export the edited splat scene to the local file system.

Current Limitations

This editor is in beta and so currently has some limitations:

  • Only supports Gaussian splat PLY files
  • Spherical harmonic data is not rotated on export

Local Development

The steps required to clone the repo and run a local development server are as follows:

git clone https://github.com/playcanvas/supersplat.git
cd supersplat
npm i
npm run develop

The last command npm run develop will build and run a local version of the editor on port 3000. Changes to the source are detected and the editor is automatically rebuilt.

To access the local editor instance, open a browser tab and navigate to http://localhost:3000.

supersplat's People

Contributors

gotoeasy avatar isumygin-sc avatar kemchenj avatar marklundin avatar slimbuck avatar willeastcott avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

supersplat's Issues

Browse button overlaps other items on Firefox

image
it appears the element for drag/drop or file uploads needs some css to fix this
In fact, there is no obvious drag/drop/upload button or zone anywhere on the page from what i am seeing

[Feature Request] Points that indicate approximate range

Currently the points are all the same size and colour. It is frequently the case that we have folds or awkward shapes where it is very difficult to arrange the model to eliminate the possibility of points on 'the other side' overlapping our selection area.
This suggestion is to draw the points as small circles, with radii ranging from perhaps 6 pixels at the very largest to 1 pixel as the smallest, based on distance.
This would enable quick eyeball identification of problematic overlaps, without needing to jiggle the camera up and down to view the parralax motions.
I believe it would work very well in combination with the previously suggested 'depth' selection mode. #29

merge two gaussian splatting scene

Hi, I have question about how to merge two gaussian splatting scene? when I load two scene into super-splat, they are sperated to differnt coordinate(cannot merge), and cannot be export to one ply.

[Feature request] Measurement Tools in Splat Viewer

Hello Team,

I am writing to express my admiration for the Splat Viewer you have developed; it's truly impressive and has been fascinating to use. The intuitive design and functionality have significantly enhanced my experience.

While exploring the viewer, I was reminded of the measurement tools available in Potree, which I found incredibly useful for detailed analysis. In this context, I would like to inquire if there are any plans to incorporate similar measurement functionalities into the Splat Viewer.

To provide a clearer idea of what I'm referring to, I have attached a screenshot showing the measurement tools in Potree. These tools allow for precise distance, area, and volume measurements within the 3D space, adding a valuable layer of interaction and data analysis.

image

Adding such features to Splat Viewer could greatly enhance its utility, especially for users who require detailed spatial analysis and measurements.

I am excited about the potential of integrating these functionalities and would be happy to provide more feedback or ideas if helpful.

Thank you for your time and the incredible work you are doing.

Support file download in remoteStorage mode

When super-splat is launched in "remoteStorage" mode it only provides an option for exporting to remote storage. We still need the option in this case to download the file as well.

blender ply

Use ply exported from blender4.0 to load the unrecognized property data tyoe kust ub ply header

[FEATURE REQUEST] Lasso Selector

It would be great for more precise selection of points along straight surfaces, to be able to use a

  • Lasso-Tool
  • Polygonal Lasso Tool <- more important

image

Camera sensitivity on mobile

The sensitivity of the camera movement on mobile phone is too low. I am speaking of the two fingers control to move the camera sideways or up/down.

RPReplay_Final1699277931.MP4

Question: want dependency instead of git submodule ?

I'm still in shock by how well this works and how fast you've made it a reality. Thinking of random ways to contribute while reading it:

Engine 1.67 brings in support for splats. Want a PR to replace submodule with minimal dependency ?

I'll make sure you can dev off engine source directly too.

Is it possible to add a nodejs script to support converting to compressed ply file from the command line?

The current editor can load both the original .ply file and .compressed.ply files, but the original .ply files are too large. It would be helpful to have a script that can automatically convert the original .ply files to compressed ply files via the command line for easier transmission.

Is it possible to add such a conversion script to enable automatic conversion?
For example, using a command like this for conversion:
node convert-ply-compressed.js /path/to/file.ply /path/to/file.compressed.ply

Luma PLY files not working?

Having some difficulty getting any of the downloadable PLY files from LumaLabs to work with your editor. When adding the file, it seems to indicate that processing is happening but then the screen remains empty.

Have attempted a few different scenes using each of the various PLY download options available.

[Feature Request] Export colored point cloud

Hi! Thanks for the wonderful work. Is it possible to extract the colored pointcloud without any other features like scale in ply? How could I convert the spherical harmonics into the rgb points?

can't find 'f_rest' properties

Thanks for your great work.
I read the ply file's properties like this:

from plyfile import PlyData
plydata2 = PlyData.read('path/to/plyfile')
print(plydata1.elements[0])

I found that the ply file exported from super-splat only output the following information:

property float x
property float y
property float z
property float f_dc_0
property float f_dc_1
property float f_dc_2
property float opacity
property float scale_0
property float scale_1
property float scale_2
property float rot_0
property float rot_1
property float rot_2
property float rot_3

Compared with the format saved by 3DGS, there seem to be 45 properties starting with 'f_rest' missing. Is this caused by the way PlyData reads files? How should I find the 'f_rest' information?

Spherical Harmonics

Has the option for importing Spherical Harmonics been removed? I can see the option available in v 0.10.3 from one of the screenshots, but it appears to have been removed since then?

"IMPORTANT NOTE

the editor viewport does not render the spherical harmonic lighting data (please submit issue if you'd like this to be added)"

.... YES PLEASE!!!!

What is the license of this project

Very nice tool!!! Thank you so much for all the effort. Just curious what is the license of this project? I couldn't find it anywhere.

Thanks

[Feature request] Change selection depth for easier cleaning

This tool is really awesome. Thanks so much for making it.

However, I'm finding it super hard to easily clean up models because the selection tool selects everything under the selection tool, without any reference to depth; and since the gaussian models include background in all directions, it is quite hard to only select the noise points I want to delete that are near the camera, without also selecting everything behind it that I don't want to get rid of. If it was possible to set a depth of selection (maybe as in relation to the distance from the camera?) this might make cleaning up the model way easier.

I suspect the really easiest way to make cleanup easy would be to make a webVR interface so users could just go into the model in 3D and mark up and remove the bad splats by hand, but I'm assuming that is not technically easy to implement :)

Want to Recenter Pivot/Origin

While editing a Splat, mostly removing a lot of them because they are unwanted, i often end up with a scene where the origin is far away from the actual model. Rotating the scene becomes akward, especially after scaling the scene up. Is there a quick way to center the pivot/origin into the center of the remaining splats and keep it there?

[Feature request] save state in localstorage

As a user editing a splat I may close a tab without intending to which will result in loss of data such as selection area or previous edits.

Workaround: don't close a tab
Alternative: a warning menu prior to closing a tab using system dialog box

Instead:

  • save state in localstorage including current ply / splat being displayed, camera position, selected splats, scene origin, splat size #, it is ok if currently selected tool state is not retained.
  • upon reloading application, check to see if state is saved and reload if exists

RangeError. Out of memory

The error "RangeError. Out of memory" occurs when attempting to load a 1.4GB PLY file in the Safari or Chrome environment on iOS. This error indicates that there is not enough available memory to handle the file.
IMG_3404 PNG

[Feature Request] Improved colour combination for the points

The blue dots on a grey background are really hard to see.
The yellow for selected dots is great.
I suggest white, cyan, or a very pale red/green/blue would all work well and make selecting loose points at the edge of the object much easier.

[Feature Request] Filter region by base colour

I'm finding that I often get a white fringe around the edge of the focus object - probably lifted from the walls or ceiling via a tracking inaccuracy.
To remove that kind of effect, it would be really helpful to be able to filter the currently selected dots by a colour range.
The dots would need to change colour as you adjust the colour range to show what will and won't be affected.

In the soft toy model, this feature would permit easy clean-up of all the white fuzz underneath (although I've just realised that may have been left there on purpose to indicate snow!)

https://playcanvas.com/viewer?default&debug.grid=false&cameraPosition=-1,1.1,-2.8&camera.multisample=false&camera.hq=false&ui.active=false&load=https://dl.dropboxusercontent.com/scl/fi/50sfqcl0a12dksq3szx6u/knitted-toy.ply?rlkey=euwuabnqmavtlcx37m99orpmv

White fringe along the top of a black T-Shirt:
image

[Feature Request] Click to select a Gaussian's point individually

After some thorough cleaning, I tend to find that there are a number of large gaussian splats sticking out from surfaces and edges where an origin point is embedded somewhere in a tight group of valid points.
This suggestion is that it be possible to switch to a 'gaussian select' mode, and then individual mouse clicks will highlight the gaussian and it's origin point.
The approach would need to take into account alpha (ignore all overlapping gaussians which are too faint) and the gaussian value would need to be sufficiently strong (80%+) to avoid too many overlaps, but weak enough so that it's not necessary to find the middle too accurately (which would defeat the purpose).
Given that often it's just a single big splat sticking out, I'd be happy if it only ever picked the gaussian which is contributing the highest amount of colour to the region clicked... combining both requirements into one.

Three large 'bad' splats with difficult to isolate points:
image

Position Rotation Scale - missing SCENE tab

hi,
is there a reason why with the new version we can't rotate the file inserting the values in the SCENE tab?
sometimes you just need to rotate 90 degrees, and you can't have a good control using transform gizmos

supersplat

local development environment

I've cloned the repo and installed the submodules and packages, however can't seem to get the project to run locally, running npm run develop returns a 404.

npm run build seems to work, but it's unclear how to setup the development environment currently, any tips here?

this would be great to add to the readme ๐Ÿ™๐Ÿป

Thanks!

Segment-Anything to remove noise

I am not familiar with 3D splats and the feasibility of this proposition but would it be possible to use segment-anything to recognise the main object on 2D projection with a click and remove all noise outside coordinates of the contour area?

[Feature Request] Apply new limits to the splat min/max

For the giant splats that stick out from surfaces and edges, a useful option would be the ability to apply local limits on the min/max size of each splat. Some of the splats generated by Poly.cam are incredibly large (10cm or more), which will be very efficient when they're orientated correctly, but is a major hassle when they are not.
This suggestion is to add some edit boxes/sliders showing the current min/max limits in use, and the ability to adjust them for the current selection. This would probably cross over with another suggestion about rescaling the model.
#31

[Feature Request] Possibility to scale the model

Hi, first of all, thank you for this tool ! Currently, most of my models are way too big (maybe this tool, or my alignment) but my scans are one or two order of magnitude too big.

Would it be possible to show a grid that would show scale (like the grid can be 200cm by default but maybe editable via slider)
image

Something like that could be useful :
image
That way I move the scene center and set scale more easily.

Leverage lightgaussian for 15 times smaller PLY files

lightgaussian has a very small effect on quality (PSNR) and reduce size by 15 times, which btw improve FPS
https://github.com/VITA-Group/LightGaussian
it is made of 3 components

  1. gaussian pruning
  2. SH distilation
  3. vecquantization informed on gaussian importance weights from 1) + octree losless compression on position data and FP16

gaussian pruning is the only component with code released for now
both 1 and 2 are unique in that they selectively remove gaussians and SH with almost no loss of accuracy
most compression alternatives online act on step 3, as mostly non selective compression on attributes
the process though can be further optimized on the 40% unquantized gaussians if we follow aras-p blog https://aras-p.info/blog/2023/09/13/Making-Gaussian-Splats-smaller/
via BC7 partitioning and k-means on SH for truly SOTA compression but this is a digression as no one has combined both solutions ATM

SIBR_gaussianViewer_app.exe cannot view exported .ply files

When I export ply file and try to view it in SIBR_gaussianViewer_app it returns the following

F:\Downloads\viewers\bin\SIBR_gaussianViewer_app.exe --model-path d:\export\point_cloud.ply 

[SIBR] --  INFOS  --:   Initialization of GLFW                                                                                           
[SIBR] --  INFOS  --:   OpenGL Version: 4.6.0 NVIDIA 536.23[major: 4, minor: 6]                                                          
[SIBR] ##  ERROR  ##:   FILE C:\projects\gauss2\SIBR_viewers\src\projects\gaussianviewer\apps\gaussianViewer\main.cpp                                            
                                           LINE 140, FUNC main
                                           Could not find config file 'cfg_args' at d:\export\point_cloud.ply

Volinga Support?

From my tests, Volinga files are not yet supported, right? (Being new to all this; I might most likely is understand the difference in format - at the moment I think it is also a point cloud that gets exported...)

(Not sure if this is a place to ask questions. If not, would you please let me know, where the proper place for that is? Thank you!)

[Bug] Gaussian splats don't face the camera

I might have misunderstood the process, but my best interpretation of the gaussian splatting explainers is that we should always project the 3D gaussian as a 2D shape facing the camera.
When moving around in Super Splat it is common to see larger splats edge-on and it is then required to move the camera to view it's contribution properly.

Facing the camera:
image
After moving the camera upwards:
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.