yunshengtian / assemble-them-all Goto Github PK

View Code? Open in Web Editor NEW

121.0 121.0 14.0 15.43 MB

[SIGGRAPH Asia 2022] Assemble Them All: Physics-Based Planning for Generalizable Assembly by Disassembly

Home Page: http://assembly.csail.mit.edu

License: MIT License

Python 18.72% CMake 0.22% C++ 80.54% C 0.11% GLSL 0.41%

assembly dataset path-planning physics-based-simulation

assemble-them-all's People

Contributors

Stargazers

Watchers

Forkers

quacrobat peterzs jackzhousz nemkumar cyun9601 cbjones33 mmlchang jost0 idontsudo digitalbotlab wangpengcv davidgmota1 zheny5 hiyyg

assemble-them-all's Issues

Dataset Consulting

Your open source dataset is generated based on Fusion 360 Gallery, so is it possible to provide a way to find the file corresponding to this assembly in Fusion 360 Gallery by one of the assemblies in this dataset?

/subdivide_batch.py // edge_length_dict // dose not exist

Hello, thank you very much for the work you have done. Can you help me with subdeviding the meshes.

for all processes I get this:

Process Process-6: Traceback (most recent call last): File "/opt/conda/envs/assembly/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/conda/envs/assembly/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/mnt/Assemble-Them-All/utils/parallel.py", line 6, in parallel_worker result = worker(*args) File "/mnt/Assemble-Them-All/assets/subdivide.py", line 98, in subdivide_assembly new_mesh = subdivide_to_size(mesh, max_edge) File "/mnt/Assemble-Them-All/assets/subdivide.py", line 34, in subdivide_to_size longest_edge_length = edge_length_dict[longest_edge] KeyError: (12338, 12300)

Would this multi-part-assembly/disassembly method work with other type of datasets (e.g. the breaking bad dataset)

Hi @yunshengtian,

Congrats for this nice work and thank you making it public. I am trying to make my first steps to the multi-part-assembly research field and I am happy that I have discovered your work.

Recently I came upon the Breaking bad dataset and benchmark which I am not sure whether you are aware and I was playing with it. In principle it is a dataset that the authors made for object fracturing and assembly. I was checking on their dataset for quite some time now and I am curious whether your approach would be able to address such fracturing and assembly/disassembly models.

I would appreciate your feedback and whether you have already tried something similar or not.

Thanks.

failed running the test_joint_sim.py | throws error [ERROR] Body type error : SDF

I ran : python examples/test_joint_sim.py --dir joint_assembly_dense --id 00016 --gravity 9.8 --steps 2000
Error:
[Error] Body type error: SDF
Traceback (most recent call last):
File "examples/test_joint_sim.py", line 103, in
sim = redmax.Simulation(xml_file_path)
RuntimeError: Caught an unknown exception!

Changed the type to MESH and CUBOIDS as well. Did not work. Kindly provide help regarding the same. I am unable to load the xml file that creates /uses sdf objects.

what does output mean --save-dir?

Hello, thank you very much for the work you have done. I'm trying to integrate your toolpath solution into my automated assembly project. The --save-dir argument generates a set of arrays with a length of 100 nested arrays on my test build. In the example, I reduced it to three nested arrays. Could you explain what the numbers in these arrays mean?

This is the output of the t_plan variable from https://github.com/yunshengtian/Assemble-Them-All/blob/main/baselines/run_joint_plan.py#LL310C21-L310C25

[[ 1.00000000e+00  0.00000000e+00  0.00000000e+00  2.64617134e-32]
 [ 0.00000000e+00  1.00000000e+00  0.00000000e+00 -3.78024540e-32]
 [ 0.00000000e+00  0.00000000e+00  1.00000000e+00  1.53810015e+00]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]
[[ 1.00000000e+00  0.00000000e+00  0.00000000e+00  4.52211259e-32]
 [ 0.00000000e+00  1.00000000e+00  0.00000000e+00 -6.46016192e-32]
 [ 0.00000000e+00  0.00000000e+00  1.00000000e+00  2.62850025e+00]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]
[[ 1.00000000e+00  0.00000000e+00  0.00000000e+00  4.66938017e-32]
 [ 0.00000000e+00  1.00000000e+00  0.00000000e+00 -6.67054421e-32]
 [ 0.00000000e+00  0.00000000e+00  1.00000000e+00  2.71410026e+00]
 [ 0.00000000e+00  0.00000000e+00  0.00000000e+00  1.00000000e+00]]

Setup python issue

LINK : warning LNK4044: Unrecognized option '/ldl'; ignored [E:\gitASP\Assemble-Them-All\simulation\build\temp.win-amd64-3.7\Release\redmax\redmax.vcxproj]

python setup.py

When running "python setup.py"，“subprocess. CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '/m']' returned non-zero exit status 1”

XML Modeling Documentation

I'm really impressed by this project, but I'm wondering where I can find more information about the XML input to the simulation. Where can I find a description of the different expected XML fields that are supported by the RedMax Simulation object? I can see the code that is parsing the XML data, but I would love some reference material if it is available. How did you know what to put in the XML files that you in your examples?

void redmax::Simmulation::forward(int, bool, bool):Assertion (q - _q_his[_q_his.size() - 1]).norm() < 1e-7 && (qdot - _qdot_his[_qdot_his.size() - 1]).norm() < 1e-7; failed 已放弃（核心已转储）

can not install in Linux

My PC info:

Linux project33-System-Product-Name 5.15.0-53-generic #59~20.04.1-Ubuntu SMP Thu Oct 20 15:10:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Python 3.7.0
cmake version 3.16.3
pybind version (2, 10, 1)

(assembly) ~$ python setup.py install

all
running install
running bdist_egg
running egg_info
creating redmax_py.egg-info
writing redmax_py.egg-info/PKG-INFO
writing dependency_links to redmax_py.egg-info/dependency_links.txt
writing top-level names to redmax_py.egg-info/top_level.txt
writing manifest file 'redmax_py.egg-info/SOURCES.txt'
reading manifest file 'redmax_py.egg-info/SOURCES.txt'
writing manifest file 'redmax_py.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning (dev) at /usr/share/cmake-3.16/Modules/FindOpenGL.cmake:275 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  CMakeLists.txt:20 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so   
CMake Error at externals/CMakeLists.txt:25 (add_subdirectory):
  The source directory

    /mnt/sde2/test/AE/test/Assemble-Them-All/simulation/externals/libigl

  does not contain a CMakeLists.txt file.


CMake Error at externals/CMakeLists.txt:34 (add_subdirectory):
  The source directory

    /mnt/sde2/test/AE/test/Assemble-Them-All/simulation/externals/pybind11

  does not contain a CMakeLists.txt file.


-- The imported target "vtkParseOGLExt" references the file
  "/usr/bin/vtkParseOGLExt-7.1"
but this file does not exist.  Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
  "/usr/lib/cmake/vtk-7.1/VTKTargets.cmake"
but not all the files it references.

-- The imported target "vtkRenderingPythonTkWidgets" references the file
  "/usr/lib/x86_64-linux-gnu/libvtkRenderingPythonTkWidgets.so"
but this file does not exist.  Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
  "/usr/lib/cmake/vtk-7.1/VTKTargets.cmake"
but not all the files it references.

-- The imported target "vtk" references the file
  "/usr/bin/vtk"
but this file does not exist.  Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
  "/usr/lib/cmake/vtk-7.1/VTKTargets.cmake"
but not all the files it references.

-- The imported target "pvtk" references the file
  "/usr/bin/pvtk"
but this file does not exist.  Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
  "/usr/lib/cmake/vtk-7.1/VTKTargets.cmake"
but not all the files it references.

CMake Deprecation Warning at externals/glfw/CMakeLists.txt:10 (cmake_policy):
  The OLD behavior for policy CMP0042 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so  
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- Using X11 for window creation
-- Found X11: /usr/include   
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
CMake Error at redmax/CMakeLists.txt:9 (pybind11_add_module):
  Unknown CMake command "pybind11_add_module".


-- Configuring incomplete, errors occurred!
See also "/mnt/sde2/test/AE/test/Assemble-Them-All/simulation/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeOutput.log".
See also "/mnt/sde2/test/AE/test/Assemble-Them-All/simulation/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeError.log".
Traceback (most recent call last):
  File "setup.py", line 66, in <module>
    zip_safe=False,
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/test/anaconda3/envs/assembly/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 32, in run
    self.build_extension(ext)
  File "setup.py", line 57, in build_extension
    subprocess.check_call(['cmake', ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env)
  File "/anaconda3/envs/assembly/lib/python3.7/subprocess.py", line 328, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/mnt/sde2/test/AE/test/Assemble-Them-All/simulation', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=Assemble-Them-All/simulation/build/lib.linux-x86_64-3.7', '-DPYTHON_EXECUTABLE=/anaconda3/envs/assembly/bin/python', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.

Multi-part assembly planning - single assembly issue

Hello @yunshengtian ,I have successfully run the project, but there seems to be a slight deviation in the results. This is the command I used: python examples/run_multi_plan.py --seq-planner prog-queue --path-planner bfs --dir multi_assembly --id 00031 --render. However, the results appear to be somewhat different from the expected outcome in the example. I'm not sure where the issue might be occurring.I tried to save the GIF using the --save-dir option, but it failed. Therefore, I will use images as a substitute for now.These two images represent two different steps.

May I ask if this algorithm only supports the OBJ format? Or does it also support step format and so on?

May I ask if there is a way to visualize SDF in the paper if I want to visualize my own model, like the figure 4 in your paper

Suggestion with environment

Hi there,

I wanted to use your repo in my project and I realized that you are using python 3.7.

To my experience, Apple M1/M2 machines have problems with python <3.9. Also, since it is python 3.7, I had a lot of other issues, like library not found etc (not sure if it is my system or general problem). But a lot of dependencies are outdated and most likely some will start to get deprecated/not supported soon.

So, I updated your environment.yml file little bit and once the installation was completed, I exported the new yml file. (added txt at the extension now to be able to upload it to git)

environment2.yml.txt

The other issue is that python setup.py install is deprecated. Instead I used pip install .

I tested the new environment on Mac M2, it works fine. I am pretty sure it should be fine in other OSs too. But it is better to test.

Look up poses of objects

I would like to look up the pose of an object in the simulator. When I attempt this using get_joint_q it does not match up with external validation (Open3D). In the animations below I would expect the red mesh (right visual) to match up with the end location in the redmax replay but this is not the case

Redmax	Open3D (Start location ✔️ End location ❌ )

Questions

Is this the correct function to use? I see there are other variants get_joint_q(..., world_frame=True), get_body_E0i. It's not clear which one to use
Am I not using the result correctly? I was thinking the result from get_joint_q could be relative to the start location but after applying that it still didn't look right

This example uses an object from this dataset

import numpy as np
import open3d as o3d
import redmax_py as redmax
from scipy.spatial.transform import Rotation

def translation_euler_angles_to_matrix(translation, euler_angles):
    T = np.eye(4)
    T[:3, :3] = Rotation.from_euler("xyz", euler_angles).as_matrix()
    T[:3, -1] = translation

    return T

np.random.seed(12)
translation = np.array([0, 0, 4.0])

euler_angles = np.random.rand(3)
quaternion = Rotation.from_euler("xyz", euler_angles).as_quat()
# Convert from XYZW -> WXYZ
quaternion = np.roll(quaternion, 1)

translation_string = " ".join([str(x) for x in translation])
quaternion_string = " ".join([str(x) for x in quaternion])
object_file = "assets/joint_assembly/00000/0.obj"

xml_string = f"""
<redmax model="assemble">
<option integrator="BDF1" timestep="1e-3" gravity="0. 0. -9.8"/>
<ground pos="0 0 0" normal="0 0 1"/>
<default>
    <ground_contact kn="1e6" kt="1e3" mu="0.8" damping="5e1"/>
    <general_SDF_contact kn="1e5" kt="1e3" mu="0.0" damping="3e1" />
</default>
<robot>
    <link name="part">
        <joint name="part" type="free3d-euler" axis="0. 0. 0." pos="{translation_string}" quat="{quaternion_string}" frame="WORLD" damping="0"/>
        <body name="part" type="SDF" filename="{object_file}" pos="0 0 0" quat="1 0 0 0" scale="1 1 1" transform_type="OBJ_TO_JOINT" density="1" dx="0.1" mu="0"/>
    </link>
</robot>
<contact>
    <ground_contact body="part"/>
</contact>
</redmax>
"""

simulation = redmax.Simulation(xml_string, "")
simulation.reset()
simulation.forward(5000)

# Read translation and euler angles from sim
translation_euler_angles = simulation.get_joint_q("part")
simulation.replay()

# Transform the mesh to the requested start position
start_location_mesh = o3d.io.read_triangle_mesh(object_file)
start_pose = translation_euler_angles_to_matrix(translation, euler_angles)
start_location_mesh.transform(start_pose)

# Transform the mesh to the expected end position
end_location_mesh = o3d.io.read_triangle_mesh(object_file)
end_pose = translation_euler_angles_to_matrix(translation_euler_angles[:3], translation_euler_angles[3:])
end_location_mesh.transform(end_pose)

o3d.visualization.draw_geometries(
    [
        # End mesh is red, start mesh is green
        end_location_mesh.paint_uniform_color((1, 0, 0)),
        start_location_mesh.paint_uniform_color((0, 1, 0)),
        # Ground plane and coordinate frame for reference
        o3d.geometry.TriangleMesh.create_coordinate_frame(),
        o3d.geometry.TriangleMesh.create_box(width=10, height=10, depth=0.0001)
    ]
)

SimRenderer gives me black images

Hi @yunshengtian,

I am trying to render the states output by setting the --render and the --record-dir parameters but the output is a black .gif file. With a quick debugging I've noticed that the output from the the sim.replay() here is back images:

I am not sure though what could be causing this, any idea?

linux installation error

I followed all the steps but when I execute the "python setup.py install" command, I get the following error:

subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j8']' returned non-zero exit status 2.

For reference, my python version is 3.7.0 and cmake version is 3.16.3. Also note that I do not have any access right issues.
Any help would be highly appreciated.