threedworld-mit / tdw Goto Github PK

View Code? Open in Web Editor NEW

493.0 493.0 75.0 430.83 MB

ThreeDWorld simulation environment

License: BSD 2-Clause "Simplified" License

Dockerfile 0.03% Python 99.88% Shell 0.09%

tdw's People

Stargazers

Watchers

tdw's Issues

Impact sound when robot colliding with objects?

Hello, thank you for this great work.
Currently I am interested with the sound simulation with robots settings. I saw detailed python example scripts about impact sound. Regarding collision, is it possible to produce collision sound when a robot arm interacting with an object?
I have tried to add some robot part to object_names and set p.set_default_audio_info(object_names=object_names). But it seems it dose not work.
And another question is can we implement (or alternatively) friction sound in TDW?

Thank you very much for your help.

Dictionary key error when adding the same object ID twice

Linux build doesn't have write permissions

UDP heartbeat connection is sometimes refused on a server

Desktop is 2560 x 1600 @ 60 Hz       
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll.so
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/Crypt32.dll
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.so
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/Crypt32
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll.so
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll.so
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/Crypt32.dll
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.so
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/Crypt32
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll.so
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libCrypt32.dll
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libKernel32
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libKernel32.so
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/Kernel32
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libKernel32
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libKernel32.so
Fallback handler could not load library /root/tdw_build/TDW/TDW_Data/Mono/libKernel32
SocketException: Connection refused                                                     
  at System.Net.Sockets.Socket.Send (System.Byte[] buffer, System.Int32 offset, System.Int32 size, System.Net.Sockets.SocketFlags socketFlags) [0x00016] in <aa976c2104104b7ca9e1785715722c9d>:0                                                         
  at System.Net.Sockets.UdpClient.Send (System.Byte[] dgram, System.Int32 bytes) [0x00045] in <aa976c2104104b7ca9e1785715722c9d>:0
  at TDWInput.StartUdp+<UDP>d__8.MoveNext () [0x00056] in <8b280ca777e243d7b67c8c9c4ff94723>:0
  at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00026] in <6c3ec4200b3e425baf3e35cde6d27f4c>:0                                                                              
Waiting on threads to park on joinable thread list timed out.

tactile simluation integration/support

Hello,

thank you for this rich-featured interactive simulation engine.

I am wondering if there are any plans for supporting tactile simulations?
For example: integrating it/supporting tactile simulator like TACTO (https://github.com/facebookresearch/tacto)

Missing json in big lists on Linux

Multiple Display Errors when launching the TDW docker image in a remote server

Hi,

I am trying to use the TDW docker image in a remote Ubuntu server. Due to the display limitation of my remote server, I have to launch a specialized docker container in my server and run an X server there so that I could connect the display to my local desktop through VNC. This workflow works fine for other applications so I was thinking about launching the TDW docker image from my container.

My docker container for display has Ubuntu version 18.04.3. When initializing it, I passed options

-v /var/run/docker.sock:/var/run/docker.sock
-v /usr/bin/docker:/bin/docker

into the RUN command.

However, when I build the TDW docker image, the building hangs in the last step: RUN ./TDW/TDW.x86_64.

I suspected that maybe ./TDW/TDW.x86_64 is an application with GUI, so I skipped the last step and launch the TDW image within my container. This is the command I use:

sudo docker run -it \
    --rm \
    --gpus all \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e DISPLAY=$DISPLAY \
    --network host \
    tdw:v1.6.0

However, when I run ./TDW/TDW.x86_64, the prompt still hangs there with no response, neither could I see anything from my display. I tried to run xclock but it says Error: Can't open display: :0. I tried to do some troubleshooting and found that my X11 socket file /tmp/.X11-unix/X0 is successfully passed into the TDW container, but it is identified as a directory instead of a socket.

In the TDW container

In my display container

I am a newbie in this and I couldn't figure out what happened.

After I failed to launch the TDW container with the Docker within Docker approach, I tried to install an X server and VNC display directly in the TDW container. I successfully see the display on my local desktop. Running xclock in the TDW container also pops up a window in the display. However, when I tried to run ./TDW/TDW.x86_64, the following error shows up:

X Error of failed request:  BadValue (integer parameter out of range for operation)
  Major opcode of failed request:  150 (GLX)
  Minor opcode of failed request:  3 (X_GLXCreateContext)
  Value in failed request:  0x0
  Serial number of failed request:  87
  Current serial number in output stream:  88

It seems to be an OpenGL problem and I have no idea how to solve it. Any feedback or suggestion on this is much appreciated! Thank you for the time!

Feature Request: Directly manipulate forces/torques of joints

Hi there,
thank you for starting such an ambitious and cool project!
We'd love to use tdw in our lab, but since we are working on autonomously learning low-level robotics tasks it is necessary for us to directly control forces and torques of robot joints.

Unless I've missed it, the robotics API only allows for target-setting of joints. Are you willing to implement a set_joint_torque method at some point, or should we apply for access to the C# backend?

Best regards,

zqm permission denied

Hi,

When I run the starter python code on my WINDOWS laptop,
in
# c = Controller()
the code breaks down and it says

Traceback (most recent call last):
  File ".\twd_test.py", line 3, in <module>
    c = Controller(port=1071, launch_build=False)
  File "C:\Users\25457\AppData\Roaming\Python\Python37\site-packages\tdw\controller.py", line 48, in __init__
    self.socket.bind('tcp://*:' + str(port))        
  File "zmq/backend/cython/socket.pyx", line 550, in zmq.backend.cython.socket.Socket.bind
  File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
    raise ZMQError(errno)
zmq.error.ZMQError: Permission denied

I guess the problem has something to do with the firewall, do you know any fixation to it?

Currently I'm running the binary with twd.exe -port 10080 and change the error line into c = Controller(port=10080, launch_build=False), seems it works, but I hopt there is a better way to avoid this

Concave flex primitives have convex colliders

Flex primitives have convex colliders in order to work in Flex but as a result they act weirdly in rigidibody simulations. They should have multiple hull colliders.

Test controller:

from tdw.librarian import ModelLibrarian
from tdw.tdw_utils import TDWUtils
from tdw.controller import Controller

c = Controller(launch_build=False)
c.start()
c.communicate([TDWUtils.create_empty_room(12, 12),
               c.get_add_object(model_name="bowl",
                                position={"x": 0, "y": 0, "z": 0},
                                object_id=0,
                                library="models_flex.json"),
               c.get_add_object(model_name="sphere",
                                position={"x": 0, "y": 2, "z": 0},
                                object_id=1,
                                library="models_flex.json"),
               {"$type": "scale_object",
                "id": 1,
                "scale_factor": {"x": 0.4, "y": 0.4, "z": 0.4}}])
for i in range(1000):
    c.communicate([])

Audio Video Synchronization

Hi, I have a question about the impact sound. Does it align the collision with the sound happening at the same time accurately?

Some general questions

Hi All,
Thank you and well done for developing such a great simulation tool!
I have a few related questions:

Which graphic engine do you use under the hood? (e.g. Maya, UE, Blender, Unity)
What are the audio rendering abilities of this simulation? Does it support doppler effect \ impulse response?
Is it possible to anchor a pair of microphones to a specific object?
Do you support multiple microphones and sound sources in a single scene?

Thanks in advance.

Unexpected behavior for some object models

Hello. I'm trying to place a stationary object in the center of a room and take pictures of it from a camera circling the room and looking at it. For many object models this works fine (like the wood_chair):

But some object models, like backpack, round_bowl_large_metal_perf, poliform_park_low_arm_sofa, and bed01_red show unexpected behavior, like the object intersecting with the floor or moving around.

Here is the simple script that produces these results:

import os
import csv
import math
import json
from tqdm import tqdm


from pathlib import Path
from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils
from tdw.librarian import ModelLibrarian
from tdw.output_data import OutputData, Bounds, Images, Collision, EnvironmentCollision

num_frames = 50
dimensions = [10, 10]

lib = ModelLibrarian(library="models_core.json")

name = "backpack"#"round_bowl_large_metal_perf" "poliform_park_low_arm_sofa" "bed01_red"

c = Controller(launch_build=True)
c.communicate({"$type": "set_render_quality", "render_quality": 3})
c.communicate({"$type": "simulate_physics", "value": True})
c.communicate({"$type": "set_time_step", "time_step": 1})

resp = c.communicate([{"$type": "load_scene", "scene_name": "ProcGenScene"},
TDWUtils.create_empty_room(*dimensions)])

x = 0
y = 0
z = 0
resp = c.communicate([c.get_add_object(model_name = name, object_id = 0, position = {"x": x, "y": y, "z": z})])

avatar_id = "a"
resp = c.communicate([{"$type": "create_avatar",
"type": "A_Img_Caps_Kinematic",
"avatar_id": avatar_id},
{"$type": "look_at",
"avatar_id": avatar_id,
"object_id": 0},
{"$type": "set_pass_masks",
"avatar_id": avatar_id,
"pass_masks": ["_img"]}])

radius = 3
angles = [2*math.pi * i/num_frames for i in range(num_frames)]

for frame in range(num_frames):
    angle = angles[frame]
    camera = {"x": radius * math.cos(angle), "y": 2.5, "z": radius * math.sin(angle)}
    resp = c.communicate([{"$type": "teleport_avatar_to", "position": camera, "avatar_id": avatar_id},
                          {"$type": "look_at","avatar_id": avatar_id, "object_id": 0},
                          {"$type": "send_images", "frequency": "once"}])

    for r in resp:
        r_id = OutputData.get_data_type_id(r)
        if r_id == "imag":
            img = Images(r)

            TDWUtils.save_images(img, filename=f"{name}_{frame:03}")

c.communicate({"$type": "terminate"})

I am using the latest version of TDW (version 1.8.24). We would appreciate an insight into what it going on. Thanks!

Fix tdw PyPi module

There's a PyPi module of TDW (pip3 install tdw) It doesn't work yet. Don't use it! See the "Getting Started" guide for the correct way to install TDW.

Getting started example

Hi, I'm following the Getting Started example locally on my laptop, and while everything works, the image I get looks nothing like in that documentation:

There is nothing special visible on the output:

Your installed tdw Python module is up to date with PyPi.
Build version 1.6.1
Unity Engine 2019.2.21f1
Python tdw module version 1.6.1

Since this is a laptop, I tried both the integrated Intel GPU, as well as the dedicated NVIDIA GPU (by running primusrun ./TDW.x86_64 and using Controller(launch_build=False)) but both give the same result. I also tried running the build in OpenGL 4.2 according to your FAQ (-force-glcore42) but it persists.

I know that this laptop and linux install can properly run 3D, as I both use it for gaming, as well as programming in Unity myself.

Here is the player log, there are a few suspicious entries (especially Couldn't deserialize command(s):), but I'm not entirely sure what to make of it, since this is new to me:

Mono path[0] = '/home/beyer/tdw_build/TDW/TDW_Data/Managed'
Mono config path = '/home/beyer/tdw_build/TDW/TDW_Data/MonoBleedingEdge/etc'
Preloaded 'libaudiopluginresonanceaudio.so'
Preloaded 'ScreenSelector.so'
Unable to preload the following plugins:
	libflexUtils.so
Display 0 '0': 3840x2160 (primary device).
Desktop is 3840 x 2160 @ 60 Hz
Initialize engine version: 2019.2.21f1 (9d528d026557)
[XR] Discovering subsystems at path /home/beyer/tdw_build/TDW/TDW_Data/UnitySubsystems
GfxDevice: creating device client; threaded=1
Renderer: GeForce GTX 960M/PCIe/SSE2
Vendor:   NVIDIA Corporation
Version:  4.5.0 NVIDIA 440.100
GLES:     0
 GL_AMD_multi_draw_indirect GL_AMD_seamless_cubemap_per_texture GL_ARB_arrays_of_arrays GL_ARB_base_instance GL_ARB_bindless_texture GL_ARB_blend_func_extended GL_ARB_buffer_storage GL_ARB_clear_buffer_object GL_ARB_clear_texture GL_ARB_clip_control GL_ARB_color_buffer_float GL_ARB_compressed_texture_pixel_storage GL_ARB_conservative_depth GL_ARB_compute_shader GL_ARB_compute_variable_group_size GL_ARB_conditional_render_inverted GL_ARB_copy_buffer GL_ARB_copy_image GL_ARB_cull_distance GL_ARB_debug_output GL_ARB_depth_buffer_float GL_ARB_depth_clamp GL_ARB_depth_texture GL_ARB_derivative_control GL_ARB_direct_state_access GL_ARB_draw_buffers GL_ARB_draw_buffers_blend GL_ARB_draw_indirect GL_ARB_draw_elements_base_vertex GL_ARB_draw_instanced GL_ARB_enhanced_layouts GL_ARB_ES2_compatibility GL_ARB_ES3_compatibility GL_ARB_ES3_1_compatibility GL_ARB_ES3_2_compatibility GL_ARB_explicit_attrib_location GL_ARB_explicit_uniform_location GL_ARB_fragment_coord_conventions GL_ARB_fragment_layer_viewport GL_ARB_fragm
ent_program GL_ARB_fragment_program_shadow GL_ARB_fragment_shader GL_ARB_framebuffer_no_attachments GL_ARB_framebuffer_object GL_ARB_framebuffer_sRGB GL_ARB_geometry_shader4 GL_ARB_get_program_binary GL_ARB_get_texture_sub_image GL_ARB_gl_spirv GL_ARB_gpu_shader5 GL_ARB_gpu_shader_fp64 GL_ARB_gpu_shader_int64 GL_ARB_half_float_pixel GL_ARB_half_float_vertex GL_ARB_imaging GL_ARB_indirect_parameters GL_ARB_instanced_arrays GL_ARB_internalformat_query GL_ARB_internalformat_query2 GL_ARB_invalidate_subdata GL_ARB_map_buffer_alignment GL_ARB_map_buffer_range GL_ARB_multi_bind GL_ARB_multi_draw_indirect GL_ARB_multisample GL_ARB_multitexture GL_ARB_occlusion_query GL_ARB_occlusion_query2 GL_ARB_parallel_shader_compile GL_ARB_pipeline_statistics_query GL_ARB_pixel_buffer_object GL_ARB_point_parameters GL_ARB_point_sprite GL_ARB_polygon_offset_clamp GL_ARB_program_interface_query GL_ARB_provoking_vertex GL_ARB_query_buffer_object GL_ARB_robust_buffer_access_behavior GL_ARB_robustness GL_ARB_sample_shading GL_ARB_sa
mpler_objects GL_ARB_seamless_cube_map GL_ARB_seamless_cubemap_per_texture GL_ARB_separate_shader_objects GL_ARB_shader_atomic_counter_ops GL_ARB_shader_atomic_counters GL_ARB_shader_ballot GL_ARB_shader_bit_encoding GL_ARB_shader_clock GL_ARB_shader_draw_parameters GL_ARB_shader_group_vote GL_ARB_shader_image_load_store GL_ARB_shader_image_size GL_ARB_shader_objects GL_ARB_shader_precision GL_ARB_shader_storage_buffer_object GL_ARB_shader_subroutine GL_ARB_shader_texture_image_samples GL_ARB_shader_texture_lod GL_ARB_shading_language_100 GL_ARB_shading_language_420pack GL_ARB_shading_language_include GL_ARB_shading_language_packing GL_ARB_shadow GL_ARB_sparse_buffer GL_ARB_sparse_texture GL_ARB_spirv_extensions GL_ARB_stencil_texturing GL_ARB_sync GL_ARB_tessellation_shader GL_ARB_texture_barrier GL_ARB_texture_border_clamp GL_ARB_texture_buffer_object GL_ARB_texture_buffer_object_rgb32 GL_ARB_texture_buffer_range GL_ARB_texture_compression GL_ARB_texture_compression_bptc GL_ARB_texture_compression_rgtc GL_
ARB_texture_cube_map GL_ARB_texture_cube_map_array GL_ARB_texture_env_add GL_ARB_texture_env_combine GL_ARB_texture_env_crossbar GL_ARB_texture_env_dot3 GL_ARB_texture_filter_anisotropic GL_ARB_texture_float GL_ARB_texture_gather GL_ARB_texture_mirror_clamp_to_edge GL_ARB_texture_mirrored_repeat GL_ARB_texture_multisample GL_ARB_texture_non_power_of_two GL_ARB_texture_query_levels GL_ARB_texture_query_lod GL_ARB_texture_rectangle GL_ARB_texture_rg GL_ARB_texture_rgb10_a2ui GL_ARB_texture_stencil8 GL_ARB_texture_storage GL_ARB_texture_storage_multisample GL_ARB_texture_swizzle GL_ARB_texture_view GL_ARB_timer_query GL_ARB_transform_feedback2 GL_ARB_transform_feedback3 GL_ARB_transform_feedback_instanced GL_ARB_transform_feedback_overflow_query GL_ARB_transpose_matrix GL_ARB_uniform_buffer_object GL_ARB_vertex_array_bgra GL_ARB_vertex_array_object GL_ARB_vertex_attrib_64bit GL_ARB_vertex_attrib_binding GL_ARB_vertex_buffer_object GL_ARB_vertex_program GL_ARB_vertex_shader GL_ARB_vertex_type_10f_11f_11f_rev GL_
ARB_vertex_type_2_10_10_10_rev GL_ARB_viewport_array GL_ARB_window_pos GL_ATI_draw_buffers GL_ATI_texture_float GL_ATI_texture_mirror_once GL_S3_s3tc GL_EXT_texture_env_add GL_EXT_abgr GL_EXT_bgra GL_EXT_bindable_uniform GL_EXT_blend_color GL_EXT_blend_equation_separate GL_EXT_blend_func_separate GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_compiled_vertex_array GL_EXT_Cg_shader GL_EXT_depth_bounds_test GL_EXT_direct_state_access GL_EXT_draw_buffers2 GL_EXT_draw_instanced GL_EXT_draw_range_elements GL_EXT_fog_coord GL_EXT_framebuffer_blit GL_EXT_framebuffer_multisample GL_EXTX_framebuffer_mixed_formats GL_EXT_framebuffer_multisample_blit_scaled GL_EXT_framebuffer_object GL_EXT_framebuffer_sRGB GL_EXT_geometry_shader4 GL_EXT_gpu_program_parameters GL_EXT_gpu_shader4 GL_EXT_multi_draw_arrays GL_EXT_multiview_texture_multisample GL_EXT_multiview_timer_query GL_EXT_packed_depth_stencil GL_EXT_packed_float GL_EXT_packed_pixels GL_EXT_pixel_buffer_object GL_EXT_point_parameters GL_EXT_polygon_offset_clamp GL_E
XT_provoking_vertex GL_EXT_rescale_normal GL_EXT_secondary_color GL_EXT_separate_shader_objects GL_EXT_separate_specular_color GL_EXT_shader_image_load_formatted GL_EXT_shader_image_load_store GL_EXT_shader_integer_mix GL_EXT_shadow_funcs GL_EXT_stencil_two_side GL_EXT_stencil_wrap GL_EXT_texture3D GL_EXT_texture_array GL_EXT_texture_buffer_object GL_EXT_texture_compression_dxt1 GL_EXT_texture_compression_latc GL_EXT_texture_compression_rgtc GL_EXT_texture_compression_s3tc GL_EXT_texture_cube_map GL_EXT_texture_edge_clamp GL_EXT_texture_env_combine GL_EXT_texture_env_dot3 GL_EXT_texture_filter_anisotropic GL_EXT_texture_integer GL_EXT_texture_lod GL_EXT_texture_lod_bias GL_EXT_texture_mirror_clamp GL_EXT_texture_object GL_EXT_texture_shadow_lod GL_EXT_texture_shared_exponent GL_EXT_texture_sRGB GL_EXT_texture_sRGB_R8 GL_EXT_texture_sRGB_decode GL_EXT_texture_storage GL_EXT_texture_swizzle GL_EXT_timer_query GL_EXT_transform_feedback2 GL_EXT_vertex_array GL_EXT_vertex_array_bgra GL_EXT_vertex_attrib_64bit GL_
EXT_window_rectangles GL_EXT_x11_sync_object GL_EXT_import_sync_object GL_NV_robustness_video_memory_purge GL_IBM_rasterpos_clip GL_IBM_texture_mirrored_repeat GL_KHR_context_flush_control GL_KHR_debug GL_EXT_memory_object GL_EXT_memory_object_fd GL_KHR_parallel_shader_compile GL_KHR_no_error GL_KHR_robust_buffer_access_behavior GL_KHR_robustness GL_EXT_semaphore GL_EXT_semaphore_fd GL_KHR_shader_subgroup GL_KTX_buffer_region GL_NV_alpha_to_coverage_dither_control GL_NV_bindless_multi_draw_indirect GL_NV_bindless_multi_draw_indirect_count GL_NV_bindless_texture GL_NV_blend_equation_advanced GL_NV_blend_equation_advanced_coherent GL_NV_blend_minmax_factor GL_NV_blend_square GL_NV_command_list GL_NV_compute_program5 GL_NV_conditional_render GL_NV_copy_depth_to_color GL_NV_copy_image GL_NV_depth_buffer_float GL_NV_depth_clamp GL_NV_draw_texture GL_NV_draw_vulkan_image GL_NV_ES1_1_compatibility GL_NV_ES3_1_compatibility GL_NV_explicit_multisample GL_NV_feature_query GL_NV_fence GL_NV_float_buffer GL_NV_fog_dista
nce GL_NV_fragment_program GL_NV_fragment_program_option GL_NV_fragment_program2 GL_NV_framebuffer_multisample_coverage GL_NV_geometry_shader4 GL_NV_gpu_program4 GL_NV_internalformat_sample_query GL_NV_gpu_program4_1 GL_NV_gpu_program5 GL_NV_gpu_program5_mem_extended GL_NV_gpu_program_fp64 GL_NV_gpu_shader5 GL_NV_half_float GL_NV_light_max_exponent GL_NV_multisample_coverage GL_NV_multisample_filter_hint GL_NV_occlusion_query GL_NV_packed_depth_stencil GL_NV_parameter_buffer_object GL_NV_parameter_buffer_object2 GL_NV_path_rendering GL_NV_pixel_data_range GL_NV_point_sprite GL_NV_primitive_restart GL_NV_query_resource GL_NV_query_resource_tag GL_NV_register_combiners GL_NV_register_combiners2 GL_NV_shader_atomic_counters GL_NV_shader_atomic_float GL_NV_shader_atomic_int64 GL_NV_shader_buffer_load GL_NV_shader_storage_buffer_object GL_NV_shader_subgroup_partitioned GL_NV_texgen_reflection GL_NV_texture_barrier GL_NV_texture_compression_vtc GL_NV_texture_env_combine4 GL_NV_texture_multisample GL_NV_texture_rec
tangle GL_NV_texture_rectangle_compressed GL_NV_texture_shader GL_NV_texture_shader2 GL_NV_texture_shader3 GL_NV_transform_feedback GL_NV_transform_feedback2 GL_NV_uniform_buffer_unified_memory GL_NV_vdpau_interop GL_NV_vdpau_interop2 GL_NV_vertex_array_range GL_NV_vertex_array_range2 GL_NV_vertex_attrib_integer_64bit GL_NV_vertex_buffer_unified_memory GL_NV_vertex_program GL_NV_vertex_program1_1 GL_NV_vertex_program2 GL_NV_vertex_program2_option GL_NV_vertex_program3 GL_NVX_conditional_render GL_NV_gpu_multicast GL_NVX_progress_fence GL_NVX_gpu_memory_info GL_NVX_nvenc_interop GL_NV_shader_thread_group GL_NV_shader_thread_shuffle GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OVR_multiview GL_OVR_multiview2 GL_SGIS_generate_mipmap GL_SGIS_texture_lod GL_SGIX_depth_texture GL_SGIX_shadow GL_SUN_slice_accum
OPENGL LOG: Creating OpenGL 4.5 graphics device ; Context level  <OpenGL 4.5> ; Context handle 47258424
Begin MonoManager ReloadAssembly
- Completed reload, in  0.075 seconds
Default vsync count 0
requesting resize 256 x 256
resizing window to 256 x 256
Desktop is 3840 x 2160 @ 60 Hz
UnloadTime: 0.917060 ms
Setting up 4 worker threads for Enlighten.
  Thread -> id: 7f81fcc9d700 -> priority: 1 
  Thread -> id: 7f81e7fff700 -> priority: 1 
  Thread -> id: 7f81e77fe700 -> priority: 1 
  Thread -> id: 7f81e6ffd700 -> priority: 1 
requesting resize 256 x 256
resizing window to 256 x 256
Desktop is 3840 x 2160 @ 60 Hz
Fallback handler could not load library /home/beyer/tdw_build/TDW/TDW_Data/Mono/libKernel32
Fallback handler could not load library /home/beyer/tdw_build/TDW/TDW_Data/Mono/libKernel32.so
Fallback handler could not load library /home/beyer/tdw_build/TDW/TDW_Data/Mono/Kernel32
Fallback handler could not load library /home/beyer/tdw_build/TDW/TDW_Data/Mono/libKernel32
Fallback handler could not load library /home/beyer/tdw_build/TDW/TDW_Data/Mono/libKernel32.so
Fallback handler could not load library /home/beyer/tdw_build/TDW/TDW_Data/Mono/libKernel32
WARNING: Shader Unsupported: 'Standard' - Pass 'META' has no vertex shader
WARNING: Shader Unsupported: 'Standard' - Pass 'META' has no vertex shader
Couldn't deserialize command(s): [{"$type": "add_object", "name": "iron_box", "url": "https://tdw-private.s3.amazonaws.com/models/linux/2018-2019.1/iron_box", "scale_factor": 1.0, "position": {"x": 0, "y": [-0.00037562177749350667, 0.8929160833358765, -7.67991878092289e-07], "z": 0}, "rotation": {"x": 0, "y": 0, "z": 0}, "category": "box", "id": 1}]
 
(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

WARNING: Shader Unsupported: 'Hidden/PostProcessing/FinalPass' - Pass '' has no vertex shader
WARNING: Shader Unsupported: 'Hidden/PostProcessing/FinalPass' - Pass '' has no vertex shader
WARNING: Shader Unsupported: 'Hidden/PostProcessing/Uber' - Pass '' has no vertex shader
WARNING: Shader Unsupported: 'Hidden/PostProcessing/Uber' - Pass '' has no vertex shader
Could not find object of ID 1
 
(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

This made me realize that you are using some "private" model for the example, probably not a good idea. However, when I change the lib.get_record("iron_box") to lib.get_record("puzzle_box_composite") (and use models_core.json), it still does not work, with pretty much the same logfile, still failing to deserialize the command:

Couldn't deserialize command(s): [{"$type": "add_object", "name": "puzzle_box_composite", "url": "https://tdw-public.s3.amazonaws.com/models/linux/2019.2//puzzle_box_composite", "scale_factor": 1, "position": {"x": 0, "y": [-0.00037562177749350667, 0.8929160833358765, -7.67991878092289e-07], "z": 0}, "rotation": {"x": 0, "y": 0, "z": 0}, "category": "toy", "id": 1}]

Any ideas?

Some models might be missing materials on OS X

always waiting in socket.recv()

I tried to follow getting_started.md in my laptop, but it waited in

File "my.py", line 3, in
c = Controller()
File "/home/sy/.local/lib/python3.6/site-packages/tdw/controller.py", line 51, in init
self.socket.recv()
File "zmq/backend/cython/socket.pyx", line 791, in zmq.backend.cython.socket.Socket.recv
File "zmq/backend/cython/socket.pyx", line 827, in zmq.backend.cython.socket.Socket.recv
File "zmq/backend/cython/socket.pyx", line 186, in zmq.backend.cython.socket._recv_copy
File "zmq/backend/cython/checkrc.pxd", line 13, in zmq.backend.cython.checkrc._check_rc

What's the problem?

Add an optional parameter to rotate objects around centroid

./TDW.x86_64 exists without any information

Running ./TDW.x86_64 exists immediately without any information. c = Controller() hangs forever at self.socket.recv(). The system is Ubuntu 18 and tdw is installed with pip install tdw. Is there any way to debug into such cases? Thanks!

When Unity window is not visible, output turns out black or "white noise"-like

I'm trying to use TDW to generate an image dataset, similar to one of your described use-cases. In my current setup, I don't always start X with access to my graphics card, so as to save battery when I'm not using it. But since image rendering is faster on the GPU, I start a tty with a GPU-enabled X session. I start the script, see the Unity window pop-up, see some images being generated, all good so far.

I then switch to another tty (running a different X session, not GPU-enabled), leaving the script running in the GPU-enabled tty. The images that are generated while I'm on the GPU-less tty turn out black or like gaussian noise.

Does the Unity window always need to be visible while rendering?

Crash to desktop when requesting collision data with a UR5 or UR10 robot

This is due to one of the colliders, ee_link, being inside another collider.

Controller:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils
from tdw.collisions import Collisions

c = Controller(launch_build=False)
c.start()
c.communicate([TDWUtils.create_empty_room(12, 12),
               c.get_add_object(model_name="rh10",
                                position={"x": 0, "y": 10, "z": 0},
                                object_id=1),
               c.get_add_robot(name="ur5", robot_id=0)])
c.communicate({"$type": "send_collisions"})
c.communicate({"$type": "pause_editor"})
for i in range(200):
    resp = c.communicate([])
    collisions = Collisions(resp=resp)
    for q in collisions.obj_collisions:
        print(q)
c.communicate({"$type": "terminate"})

Inaccurate end effector position when use inverse kinematic solution

Hello, I refered Magnebot codes for robot control but ran into problems of inaccurate end_effector positions.
As is in Magnebot, ikpy is used to calculate the inverse kinematic solution. I use it too, but for a ur5 arm.
See codes below for details (some of the codes are also cherry-picked from tdw-transport-challenge).

from pathlib import Path
from typing import Dict, List, Optional, Union

import ikpy.chain
import numpy as np
from magnebot.joint_static import JointStatic
from magnebot.util import check_version, get_data
from tdw.controller import Controller
from tdw.librarian import ModelLibrarian, RobotLibrarian
from tdw.output_data import Bounds, Images, OutputData, Robot, StaticRobot
from tdw.robot_creator import RobotCreator
from tdw.tdw_utils import AudioUtils, TDWUtils

lib_path = "resources/robot_librarian.json"
if not Path(lib_path).exists():
    RobotLibrarian.create_library(path=lib_path, description="Custom Robot Librarian")
lib = RobotLibrarian(lib_path)
robot_name = "ur5"

robot_urdf_urls = {
    "ur5": "https://github.com/ros-industrial/robot_movement_interface/blob/master/dependencies/ur_description/urdf/ur5_robot.urdf.xacro",
}
robot_creator_base_path = "/Users/xufeng/robot_creator/Assets/robots/"
chain_urdf_files = {
    "ur5": robot_creator_base_path + "ur5_robot.urdf",
}
urdf_base_elements = {
    "ur5": ["base_link"],
}
if lib.get_record(robot_name) is None:
    r = RobotCreator()
    record = r.create_asset_bundles(
        urdf_url=robot_urdf_urls[robot_name],
    )
    lib.add_or_update_record(record=record, overwrite=False, write=True)


class SceneState:
    def __init__(self, resp: List[bytes], only_get_img=False):
        r = get_data(resp=resp, d_type=Robot)
        self.joint_positions: Dict[int, np.array] = dict()
        self.joint_angles: Dict[int, np.array] = dict()
        for i in range(r.get_num_joints()):
            j_id = r.get_joint_id(i)
            self.joint_positions[j_id] = r.get_joint_position(i)
            self.joint_angles[j_id] = r.get_joint_positions(i)


class RobotStatic:
    def __init__(
        self,
        static_robot: StaticRobot,
        chain_urdf_file=None,
        urdf_base_element=None,
    ):
        self.joints: Dict[int, JointStatic] = dict()
        self.joint_name_id: Dict[str, int] = dict()
        for i in range(static_robot.get_num_joints()):
            joint_id = static_robot.get_joint_id(i)
            # Cache the body parts.
            joint_static = JointStatic(sr=static_robot, index=i)
            self.joints[joint_id] = joint_static
            joint_name = joint_static.name
            self.joint_name_id[joint_name] = joint_id
        self.chain = (
            ikpy.chain.Chain.from_urdf_file(
                chain_urdf_file, base_elements=urdf_base_element
            )
            if chain_urdf_file is not None
            else None
        )


class Test(Controller):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.robot_id: int = 0

    def get_robot(self, resp: List[bytes]) -> Robot:
        robot: Optional[Robot] = None
        for i in range(len(resp) - 1):
            r_id = OutputData.get_data_type_id(resp[i])
            if r_id == "robo":
                r = Robot(resp[i])
                if r.get_id() == self.robot_id:
                    robot = r
                    break
        return robot

    def trial(self):
        def get_ik_commands(
            target_position, scene_state: SceneState, robot_static: RobotStatic
        ) -> np.array:
            def convert_for_ik(x, y, z):
                return z, -x, y

            angles = robot_static.chain.inverse_kinematics(
                convert_for_ik(*target_position),
            )
            angles = [float(np.rad2deg(angle)) for angle in angles]
            ik_commands = []
            for angle, joint_id in zip(angles, scene_state.joint_positions):
                if robot_static.joints[joint_id].joint_type != "revolute":
                    continue
                ik_commands.append(
                    {
                        "$type": "set_revolute_target",
                        "id": robot_id,
                        "joint_id": joint_id,
                        "target": angle,
                    }
                )
            return ik_commands

        commands = [
            TDWUtils.create_empty_room(12, 12),
            {"$type": "destroy_all_objects"},
            {"$type": "set_screen_size", "width": 800, "height": 800},
            {"$type": "set_target_framerate", "framerate": 100},
        ]
        commands.extend(
            TDWUtils.create_avatar(
                avatar_type="A_Img_Caps_Kinematic",
                position={"x": 0.5, "y": 1, "z": 0.5},
                look_at={"x": 0.3, "y": 0, "z": 0.3},
            )
        )
        resp = self.communicate(commands)
        robot_id = self.robot_id
        robot_commands = [
            self.get_add_robot(
                name=robot_name,
                robot_id=robot_id,
                library=lib_path,
            ),
            {"$type": "send_static_robots", "frequency": "once"},
            {"$type": "send_robots", "frequency": "always"},
        ]
        commands.extend(robot_commands)
        resp = self.communicate(commands)
        self.robot_static = RobotStatic(
            static_robot=get_data(resp=resp, d_type=StaticRobot),
            chain_urdf_file=chain_urdf_files[robot_name],
            urdf_base_element=urdf_base_elements[robot_name],
        )
        state_0 = SceneState(resp=resp)
        next_tip_position = [0.3, 0.3, 0.3]
        ik_commands = get_ik_commands(next_tip_position, state_0, self.robot_static)
        resp = self.communicate(ik_commands)
        # waiting
        for i in range(200):
            print(".", end="")
            resp = self.communicate([])
        state_1 = SceneState(resp=resp)
        tip_pos = state_1.joint_positions[self.robot_static.joint_name_id["ee_link"]]
        print(f"End effector position error: f{next_tip_position - tip_pos}.")

    def run(self):
        self.start()
        self.trial()
        self.communicate({"$type": "terminate"})


if __name__ == "__main__":
    Test().run()

Output:

........................................................................................................................................................................................................End effector position error: f[ 0.12968929 -0.0595466   0.03743686].

Note that after 200 frames, the robotic arm is almost being static, but the position error between desired and real is quite large (while we always suppose a quick response and a small error like 1e-4).
PS: I have coordinate mapping problems when directly utilizing ikpy, so I defined a function called convert_for_ik() to convert. According to the movement of the arm, it works well.

Object ID collisions (with avatars?)

Upgrade Ubuntu version in Docker container

We have Ubuntu 16 in the Docker container because as far as we know, Flex doesn't run on later versions of Ubuntu due to missing CUDA drivers.

"send_images" command surprisingly slow

Hi,

I'm playing around with a very small example derived from the "getting started" tutorial: an empty room with a table, and the "puzzle" object falling onto that table. Full code in this gist: https://gist.github.com/lucasb-eyer/7468124163ce69b7e8231a8e6884f803

I am running this locally on a gaming laptop, i.e. with a good GPU.

What I noticed, is that if I simply loop like this:

for s in range(100):
    resp = c.communicate([])

I can see the Unity window, and the whole animation is really nice and smooth. However, if I now ask for images:

for s in range(100):
    resp = c.communicate([{"$type": "send_images", "frequency": "once", "avatar_id": avatar_id}])

then the whole animation (in the Unity window) gets really slow and laggy, feels like it's making somewhere around 1-5 FPS.
This only happens for higher resolution, such as the 1280x720 I select here.

Since the animation is fast without the send_images request, and since all of this is running on localhost, I believe that neither communication, nor my GPU are the problem here. It seems to me that one of the following would cause such slowdown, given my observations:

Acquiring the pixel-buffer in a bad way. I don't know how you're doing that on the build, but there should be a way of getting the pixel buffer that is not slow.
It seems that the image is sent to the controller as a png encoded image. I believe this is where the slowdown comes from. It is not possible to encode an image into png format in real-time for higher resolutions in my experience.

What do you think, does this guess sound reasonable? Am I missing something completely different?

If you agree with my guess, would it be possible to add an encoding flag to send_images with the option of selecting raw? When running both on localhost, png/jpg encoding really doesn't make sense and wastes resources, especially if it's decoded back into a numpy array on the controller side right away anyways.

Can't run TDW on OS X by double-clicking the .app

rotate_object_by command api not as expected

Hi,

Using the following code snippet, I generated some images with different rotations for the object yaw with 'use_centroid' : True. I expected the object to rotate about its centroid but it seems to be about a point far off its center.

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils
from tdw.librarian import ModelLibrarian, MaterialLibrarian, ModelRecord, MaterialRecord, HDRISkyboxLibrarian, SceneLibrarian
from tdw.output_data import Images, Environments
from pathlib import Path
from tqdm import tqdm
import json
import numpy as np
import os

class MyScreenShotter(Controller):
    """
    Base class for screenshotter controllers.
    """

    def __init__(self, library: str, scene_name: str ='tdw_room', port=1071, launch_build=True):
        self.model_lib = ModelLibrarian(library + '.json')
        self.material_lib = MaterialLibrarian('materials_high.json')
        self.hdri_skybox_lib = HDRISkyboxLibrarian()
        self.scene_lib = SceneLibrarian()
        self.scene_name = scene_name
        self.materials = self.material_lib.records
        self._lib = self._get_librarian(library)
        self.log = Path("log.txt")
        if self.log.exists():
            self.log.unlink()
        output_dir = self._get_output_directory()
        if not output_dir.exists():
            output_dir.mkdir(parents=True)
        self.output_dir = str(output_dir.resolve())
        Path(self.output_dir).joinpath("records.json").write_text(
            json.dumps(self._get_visualizer_metadata()), encoding="utf-8")

        super().__init__(port, launch_build=launch_build)
        if library == 'models_full':
            self.communicate({"$type": "use_pre_signed_urls", "value": True})

        self.add_scene()
        self.envs = Environments(self.communicate({"$type": "send_environments",
                                                   "frequency": "once"})[0])
        self.set_opts1()
        self.communicate(TDWUtils.create_avatar(
            avatar_type='A_Img_Caps_Kinematic',
            position=self._get_avatar_position()
        ))
        self.set_opts2()

        self._set_post_processing()

    def add_scene(self):
        add_scene_command = self.get_add_scene(scene_name=self.scene_name)
        self.communicate(add_scene_command)

    def set_opts1(self):
        self.communicate([{"$type": "simulate_physics",
                           "value": False},
                          {"$type": "set_screen_size",
                           "height": 1024,
                           "width": 1024},
                          {"$type": "set_render_quality",
                           "render_quality": 5},
                          {"$type": "set_shadow_strength",
                           "strength": 1.0},
                          {"$type": "set_img_pass_encoding",
                           "value": False}])

    def set_opts2(self):
        self.communicate([{"$type": "set_pass_masks",
                           "avatar_id": "a",
                           "pass_masks": ["_img"]},
                          {"$type": "send_images",
                           "frequency": "always"}])

     def _get_output_directory(self):
        """
        :return: The output directory for the images.
        """
        return Path.home().joinpath("TDWImages/ModelImages")

    def run(self):
        record = self.model_lib.records[0]
        o_id = Controller.get_unique_id()
        resp = self.communicate([{"$type": "add_object",
                                  "name": record.name,
                                  "url": record.get_url(),
                                  "scale_factor": 2 * TDWUtils.get_unit_scale(record),
                                  "rotation": record.canonical_rotation,
                                  'position': {'x': 0, 'y': 1, 'z': 0},
                                  "id": o_id},
                                 {"$type": "look_at",
                                  "avatar_id": "a",
                                  "object_id": o_id,
                                  "use_centroid": True}
                                 ])
        TDWUtils.save_images(
            Images(resp[0]), '{}'.format(record.name),
            output_directory=os.path.join(self.output_dir, 'rotation'), append_pass=False)
        

        for i, angle in enumerate(np.arange(0, 1, 0.1)):
            resp = self.communicate([{'$type': 'rotate_object_by',
                                      "angle": angle,
                                      'axis': 'yaw',
                                      'is_world' : False,
                                      'use_centroid': True,
                                      'id': o_id}])
            TDWUtils.save_images(
                Images(resp[0]), '{}_angle{}'.format(record.name, i),
                output_directory=os.path.join(self.output_dir, 'rotation'), append_pass=False)

        # Destroy the model and unload the asset bundle.
        self.communicate([{"$type": "destroy_object",
                           "id": o_id},
                          {"$type": "unload_asset_bundles"}])
        self.communicate({"$type": "terminate"})

if __name__ == "__main__":
    from argparse import ArgumentParser
    parser = ArgumentParser()
    parser.add_argument("--type", default="models_core", choices=["models_full", "models_core"],
                        help="The type of screenshotter and the asset library.")
    parser.add_argument("--target", nargs="?", type=str, help="The name of a single record.")
    args = parser.parse_args()

    M = MyScreenShotter(args.type)
    M.run()

The following shows a gif of the generated images

"S" key turns clockwise and "D" key does nothing in keyboard_controls.py

Was flipping through some demos to get a feel for tdw and ran into controls being bugged as described in title. "W" and "A" key work as intended though. The effect is mirrored for the arrow keys as well, so I think something is going on with the "move" and "turn" functions. Here is my environment.

Build version 1.8.25
Unity Engine 2020.2.7f1
Python tdw module version 1.8.25

Python version: 3.9.6

Fix Docker container

How does engine parse the commands and play the audio

Hi, I have one question in get_audio_commands(). It detects the collision and generates the impact sound by get_audio_commands(). It is executed by every frame, which is 0.01s in the setup. The default sampling rate is 44100.
I try to log the generated sound in each frame as below and find something confusing. Each generated sound is an impact response, but how to concatenate them is at the engine side.

each frame can generate more than 44100*0.01=440 samples. How do you concatenate these samples?
One collision can be detected in consecutive frames, such as 53 and 54. How do you concatenate these frames?

frame number53, the wave shape (8502,)
frame number54, the wave shape (8781,)
frame number88, the wave shape (8696,)
frame number92, the wave shape (8762,)
frame number93, the wave shape (9214,)
frame number99, the wave shape (8701,)
frame number109, the wave shape (8514,)
frame number110, the wave shape (8550,)
frame number111, the wave shape (8759,)
frame number112, the wave shape (8616,)
frame number114, the wave shape (8704,)
frame number115, the wave shape (8610,)
frame number119, the wave shape (8871,)
frame number123, the wave shape (8863,)
frame number124, the wave shape (9031,)
frame number126, the wave shape (8716,)```

RobotArm example does not launch the build.

Hi,
First of all, thanks for making this awesome tool public!

What happens:
Running example_controllers/robot_arm.py hangs, failing to display the robot arm. There is no error.

Probable cause:
launch_build is set to False:
if __name__ == "__main__": RobotArm(launch_build=False).run()

Suggested fix.

Create composite object with different mass/friction.. for each subobject

Dear authors,

Thanks for your great work, it looks quite interesting. Before I decide to use it in my work, I'm wondering if I can use it to create some random composite object and assign each subpart with different mass, friction, etc.. And maybe ultimately I can export the composite object into a single file (urdf, xml or whatever). I see there is a part about composite objects in the documentation, but not really answer my question.

Thanks in advance :)

AssetBundleCreator only generates assets for windows

Hi, I ran local_object.py on Windows 10 and got the following error:

Converting a .fbx file to .obj
Running V-HACD on a .obj file (this might take awhile).
Created: cube.wrl
Removed V-HACD log file.
Removed superfluous .obj file: cube.obj
Converting .wrl to .obj
Created C:\Users\dsp\dev\projects\tdw\tdw\Python\example_controllers\cube_colliders.obj from .wrl file.
Copying files into the Unity project.
Copied files into the Unity project.
Creating the prefab...
Created the prefab.
Creating local asset bundles
Traceback (most recent call last):
  File ".\local_object.py", line 45, in <module>
    LocalObject.run()
  File ".\local_object.py", line 18, in run
    asset_bundle_paths, record_path = AssetBundleCreator().create_asset_bundle("cube.fbx", True, 123, "", 1)
  File "C:\Users\dsp\dev\projects\tdw\venv\lib\site-packages\tdw\asset_bundle_creator.py", line 97, in create_asset_bundle
    asset_bundle_paths = self.prefab_to_asset_bundle(prefab_path, model_name)
  File "C:\Users\dsp\dev\projects\tdw\venv\lib\site-packages\tdw\asset_bundle_creator.py", line 536, in prefab_to_asset_bundle
    assert bundle_path.exists(), f"Missing asset bundle: {asset_bundle_platform}"
AssertionError: Missing asset bundle: StandaloneOSX

In asset_bundle_creator\Assets\NewAssetBundles\cube I can see StandaloneOSX and StandaloneLinux64, but they are empty folders. StandaloneWindows64 contains files for the new model.

Did I miss something?

Crash on OS X when using ResonanceAudio

Mono path[0] = '/Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Resources/Data/Managed'
Mono config path = '/Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/MonoBleedingEdge/etc'
Initialize engine version: 2020.2.7f1 (c53830e277f1)
Plugins: Couldn't open OculusXRPlugin, error: dlopen(OculusXRPlugin, 2): image not found

[Subsystems] Discovering subsystems at path /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Resources/Data/UnitySubsystems
[Subsystems] No descriptors matched for  examples in UnitySubsystems/OculusXRPlugin/UnitySubsystemsManifest.json.
[Subsystems] 1 'inputs' descriptors matched in UnitySubsystems/OculusXRPlugin/UnitySubsystemsManifest.json
[Subsystems] 1 'displays' descriptors matched in UnitySubsystems/OculusXRPlugin/UnitySubsystemsManifest.json
[Subsystems] No descriptors matched for  meshings in UnitySubsystems/OculusXRPlugin/UnitySubsystemsManifest.json.
[Subsystems] No descriptors matched for  examples in UnitySubsystems/WindowsMRXRSDK/UnitySubsystemsManifest.json.
[Subsystems] 1 'inputs' descriptors matched in UnitySubsystems/WindowsMRXRSDK/UnitySubsystemsManifest.json
[Subsystems] 1 'displays' descriptors matched in UnitySubsystems/WindowsMRXRSDK/UnitySubsystemsManifest.json
[Subsystems] 1 'meshings' descriptors matched in UnitySubsystems/WindowsMRXRSDK/UnitySubsystemsManifest.json
GfxDevice: creating device client; threaded=1
2021-06-28 15:04:26.701 TDW[46658:22500974] Color LCD preferred device: AMD Radeon Pro 5300M (high power)
2021-06-28 15:04:26.701 TDW[46658:22500974] Metal devices available: 2
2021-06-28 15:04:26.701 TDW[46658:22500974] 0: AMD Radeon Pro 5300M (high power)
2021-06-28 15:04:26.701 TDW[46658:22500974] 1: Intel(R) UHD Graphics 630 (low power)
2021-06-28 15:04:26.702 TDW[46658:22500974] Using device AMD Radeon Pro 5300M (high power)
Initializing Metal device caps: AMD Radeon Pro 5300M
Begin MonoManager ReloadAssembly
- Completed reload, in  0.175 seconds
XRGeneral Settings awakening...

UnloadTime: 2.504494 ms
Setting up 6 worker threads for Enlighten.
  Thread -> id: 700010c95000 -> priority: 1 
  Thread -> id: 700010d18000 -> priority: 1 
  Thread -> id: 700010e1e000 -> priority: 1 
  Thread -> id: 700010b8f000 -> priority: 1 
  Thread -> id: 700010c12000 -> priority: 1 
  Thread -> id: 700010d9b000 -> priority: 1 
Metal RecreateSurface[0x10ec99100]: surface size 256x256
TDW v1.8.15

Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/audiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/audiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
DllNotFoundException: audiopluginresonanceaudio
  at (wrapper managed-to-native) ResonanceAudio.SetRoomProperties(intptr,single[])
  at ResonanceAudio.DisableRoomEffects () [0x00000] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoomManager.UpdateRoomEffects () [0x0000c] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoomManager.UpdateRoom (ResonanceAudioRoom room) [0x0000c] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoom.OnEnable () [0x00000] in <3bce3af48a8b465298049f820126dc66>:0 
UnityEngine.GameObject:Internal_AddComponentWithType(Type)
UnityEngine.GameObject:AddComponent(Type)
UnityEngine.GameObject:AddComponent()
UtilExtensionMethods:GetOrAddComponent(GameObject)
TDWInput.CreateReverbSpaceCommand:Do()
NetworkManager:Unpack(NetMQMessage&)
Req:Update()
NetworkManager:Update()

Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/audiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/Kernel32
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/audiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libKernel32
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/audiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
DllNotFoundException: audiopluginresonanceaudio
  at (wrapper managed-to-native) ResonanceAudio.SetListenerGain(single)
  at ResonanceAudio.UpdateAudioListener (ResonanceAudioListener listener) [0x0001b] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioListener.Update () [0x0001d] in <3bce3af48a8b465298049f820126dc66>:0 

DllNotFoundException: audiopluginresonanceaudio
  at (wrapper managed-to-native) ResonanceAudio.SetListenerGain(single)
  at ResonanceAudio.UpdateAudioListener (ResonanceAudioListener listener) [0x0001b] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioListener.Update () [0x0001d] in <3bce3af48a8b465298049f820126dc66>:0 

Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/audiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.dylib
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.so
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio.bundle
Fallback handler could not load library /Users/vinayakagarwal/tdw_build/TDW/TDW.app/Contents/Frameworks/MonoEmbedRuntime/osx/libaudiopluginresonanceaudio
DllNotFoundException: audiopluginresonanceaudio
  at (wrapper managed-to-native) ResonanceAudio.SetRoomProperties(intptr,single[])
  at ResonanceAudio.UpdateRoom (ResonanceAudioRoom room) [0x0003b] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoomManager.UpdateRoomEffects () [0x00037] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoomManager.UpdateRoom (ResonanceAudioRoom room) [0x0000c] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoom.Update () [0x00000] in <3bce3af48a8b465298049f820126dc66>:0 

DllNotFoundException: audiopluginresonanceaudio
  at (wrapper managed-to-native) ResonanceAudio.SetRoomProperties(intptr,single[])
  at ResonanceAudio.DisableRoomEffects () [0x00000] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoomManager.UpdateRoomEffects () [0x0000c] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoomManager.RemoveRoom (ResonanceAudioRoom room) [0x00007] in <3bce3af48a8b465298049f820126dc66>:0 
  at ResonanceAudioRoom.OnDisable () [0x00000] in <3bce3af48a8b465298049f820126dc66>:0 

Waiting on threads to park on joinable thread list timed out.

no /Python folder for pip installation

I may be missing something obvious, but upon extracting the TDW folder for Windows or Linux, there is no Python folder from which to run pip install.

On Windows the tree is :
├───MonoBleedingEdge
│ ├───EmbedRuntime
│ └───etc
│ └───mono
│ ├───2.0
│ │ └───Browsers
│ ├───4.0
│ │ └───Browsers
│ ├───4.5
│ │ └───Browsers
│ └───mconfig
└───TDW_Data
├───Managed
├───Plugins
└───Resources

And on Linux:
└───TDW_Data
├───Managed
├───MonoBleedingEdge
│ ├───etc
│ │ └───mono
│ │ ├───2.0
│ │ │ └───Browsers
│ │ ├───4.0
│ │ │ └───Browsers
│ │ ├───4.5
│ │ │ └───Browsers
│ │ ├───mconfig
│ │ └───registry
│ └───x86_64
├───Plugins
│ └───x86_64
└───Resources

None of the subfolders contain anything I've found that python can play with. Help pls? :)

AssetBundleCreator().create_asset_bundle() not work?

I'm using Ubuntu 18.04, with Unity 2019.4.15f1. I try to convert some .obj files from ShapeNet to asset bundle so that I can load my custom objects into TDW.

However, I firstly find some codes in asset_bundle_creator.py not suitable for Unity in Linux. I changed the path to executable file of Unity, then comment the asset bundle production for Windows and MACOS, since Unity in Linux cannot install the build tool for other platforms. I will put the asset_bundle_creator.py that I've modified here.

After these modifications, I successfully run local_object.py, but the unity window shows nothing but white, which means my custom asset bundle has not been successfully loaded. I don't know whether anyone face this situation, or any one can help me?

The following is asset_bundle_creator.py that I have modified. There's only a few modifications, you can copy it and compare with git tools.

from pathlib import Path
import platform
from typing import List, Dict, Optional, Tuple, Union
from subprocess import call, check_output, CalledProcessError
import os
import shutil
from tdw.librarian import ModelRecord
from tdw.backend import paths
import json
import pkg_resources
import re
import distutils.dir_util
import distutils.file_util
from tdw.backend.platforms import S3_TO_UNITY, SYSTEM_TO_UNITY, UNITY_TO_SYSTEM


class AssetBundleCreator:
    """
    Given a .fbx file or a .obj file, and (optionally) Materials and/or Textures folder adjacent to that file,
    create asset bundles for Windows, OS X, and Linux.

    Usage:

    ```python
    from tdw.asset_bundle_creator import AssetBundleCreator

    a = AssetBundleCreator()

    # Typically this is the only function you'll want to call.
    asset_bundle_paths, record_path = a.create_asset_bundle("cube.fbx", cleanup=True)
    ```

    For more information, see: `Documentation/misc_frontend/add_local_object.md`.
    """

    UNITY_VERSION = "2019.4"

    def __init__(self, quiet: bool = False, display: str = ":0"):
        """
        :param quiet: If true, don't print any messages to console.
        :param display: The display to launch Unity Editor on. Ignored if this isn't Linux.
        """

        # Get the binaries path and verify that AssetBundleCreator will work on this platform.
        system = platform.system()

        self.env = os.environ.copy()

        # libgconf needs to be installed the Editor to work.
        if system == "Linux":
            try:
                check_output(["dpkg", "-l", "libgconf-2-4"])
            except CalledProcessError as e:
                raise Exception(f"{e}\n\nRun: sudo apt install libgconf-2-4")
            # Set the display for Linux.
            self.env["DISPLAY"] = display

        self.binaries: Dict[str, str] = dict()
        binary_path = f"binaries/{system}"

        # Cache the binaries.
        self.binaries["assimp"] = f"{binary_path}/assimp/assimp"
        self.binaries["meshconv"] = f"{binary_path}/meshconv/meshconv"
        self.binaries["vhacd"] = f"{binary_path}/vhacd/testVHACD"

        for binary in self.binaries:
            # Add the .exe suffix for Windows.
            if system == "Windows":
                self.binaries[binary] += ".exe"
            # Run chmod +x on everything.
            else:
                call(["chmod", "+x", pkg_resources.resource_filename(__name__, self.binaries[binary])])

        self.quiet = quiet

        self.project_path = self.get_unity_project()
        assert self.project_path.exists(), self.project_path

        self.unity_call = self.get_base_unity_call()

    def create_asset_bundle(self, model_path: Union[Path, str], cleanup: bool, wnid: int = -1, wcategory: str = "", scale: float = 1) -> (List[Path], Path):
        """
        Create an asset bundle for each operating system. Typically, this is the only function you'll want to use.
        This function calls in sequence: `fbx_to_obj()`, `obj_to_wrl()`, `wrl_to_obj()`, `move_files_to_unity_project()`, `create_prefab()`, `prefab_to_asset_bundle()`, and `create_record()`.

        :param model_path: The path to the model.
        :param cleanup: If true, remove temporary files after usage.
        :param wnid: The WordNet ID.
        :param wcategory: The WordNet category.
        :param scale: The scale of the object.

        :return The paths to each asset bundle as Path objects (from pathlib) and the path to the metadata record file as a Path object (from pathlib).
        """

        model_path = self.get_model_path(model_path)

        model_name = model_path.stem

        # Create the asset bundles.
        obj_path, is_new = self.fbx_to_obj(model_path)
        wrl_path = self.obj_to_wrl(model_path)
        obj_colliders_path = self.wrl_to_obj(wrl_path, model_name)
        copied_file_paths = self.move_files_to_unity_project(obj_colliders_path, model_path)
        prefab_path, report_path = self.create_prefab(f"{model_name}_colliders.obj", model_name, model_path.suffix)
        asset_bundle_paths = self.prefab_to_asset_bundle(prefab_path, model_name)

        # Parse the URLs.
        urls = self.get_local_urls(asset_bundle_paths)

        # Create the metadata record.
        record_path = self.create_record(model_name, wnid, wcategory, scale, urls)

        # Remove the temporary files.
        if cleanup:
            paths = [wrl_path, obj_colliders_path, prefab_path, report_path]
            paths.extend(copied_file_paths)
            if is_new:
                paths.append(obj_path)

            assert model_path not in paths

            for path in paths:
                if not path.exists():
                    continue
                path.unlink()

            # Remove all materials.
            materials_directory = self.get_resources_directory().joinpath("Materials")
            if materials_directory.exists():
                shutil.rmtree(str(materials_directory.resolve()))

            if not self.quiet:
                print("Removed temporary files.")

        if not self.quiet:
            print("DONE!")

        return asset_bundle_paths, record_path

    @staticmethod
    def get_model_path(model_path: Union[Path, str]) -> Path:
        """
        Check if the model path is valid.

        :param model_path: The path to the model. Can be a Path object, or a string representing the absolute file path.

        :return The path as a Path object if there are no problems.
        """

        if isinstance(model_path, str):
            model_path = Path(model_path)
        assert model_path.exists(), f"Model path doesn't exist: {model_path}"
        assert model_path.is_file(), f"Model path isn't a file: {model_path}"
        assert model_path.suffix in [".fbx", ".obj", ".prefab"], "Model file must be .fbx, .obj, or .prefab"

        return model_path

    def get_assets_directory(self) -> Path:
        """
        :return The path to `<home>/asset_bundle_creator/Assets/`
        """

        assets_directory = self.project_path.joinpath("Assets")
        assert assets_directory.exists(), f"Assets directory not found: {assets_directory.resolve()}"

        return assets_directory

    def get_resources_directory(self) -> Path:
        """
        :return The path to `<home>/asset_bundle_creator/Assets/Resources`
        """

        resources_directory = self.get_assets_directory().joinpath("Resources")
        assert resources_directory.exists(), f"Resources directory not found: {resources_directory.resolve()}"

        return resources_directory

    @staticmethod
    def get_editor_path() -> Path:
        """
        :return The path to the Unity Editor executable.
        """

        system = platform.system()

        # Get the path to the Editor executable.
        if system == "Windows":
            editor_path = Path('C:/Program Files/Unity/Hub/Editor/')

            # Sometimes Unity Hub is installed here instead.
            if not editor_path.exists():
                editor_path = Path('C:/Program Files/Unity Hub/')
        elif system == "Darwin":
            editor_path = Path("/Applications/Unity/Hub/Editor")
        elif system == "Linux":
            editor_path = Path.home().joinpath("Unity/2019.4.15f1/Editor")
        else:
            raise Exception(f"Platform not supported: {system}")

        assert editor_path.exists(), f"Unity Hub not found: {editor_path}"

        # Get the expected Unity version.
        # ds = []
        # re_pattern = AssetBundleCreator.UNITY_VERSION + ".(.*)"
        # for d in editor_path.iterdir():
        #     if AssetBundleCreator.UNITY_VERSION not in d.stem:
        #         continue
        #     re_search = re.search(re_pattern, str(d.resolve()))
        #     if re_search is None:
        #         continue
        #     ds.append(d)
        # ds = sorted(ds, key=lambda version: int(re.search(re_pattern, str(version.resolve())).group(1), 16))
        editor_version = '2019.4.15f1'
        # editor_path = editor_path.joinpath(editor_version)

        if system == "Windows":
            editor_path = editor_path.joinpath("Editor/Unity.exe")
        elif system == "Darwin":
            editor_path = editor_path.joinpath("Unity.app/Contents/MacOS/Unity")
        elif system == "Linux":
            editor_path = editor_path.joinpath("Unity")
        else:
            raise Exception(f"Platform not supported: {system}")
        assert editor_path.exists(), f"Unity Editor {editor_version} not found. %s" % (editor_path)

        return editor_path

    def get_base_unity_call(self) -> List[str]:
        """
        :return The call to launch Unity Editor silently in batchmode, execute something, and then quit.
        """

        return [str(AssetBundleCreator.get_editor_path().resolve()),
                "-projectpath",
                str(self.project_path.resolve()),
                "-quit",
                "-batchmode"]

    def get_unity_project(self) -> Path:
        """
        Build the asset_bundle_creator Unity project.

        :return The path to the asset_bundle_creator Unity project.
        """

        unity_project_path = Path.home().joinpath("asset_bundle_creator")

        # If the project already exists, stop.
        if unity_project_path.exists():
            return unity_project_path

        if not self.quiet:
            print(f"Creating: {unity_project_path.resolve()}")
            
        call([str(AssetBundleCreator.get_editor_path().resolve()),
              "-createProject",
              str(unity_project_path.resolve()),
              "-quit",
              "-batchmode"], env=self.env)
        assert unity_project_path.exists(), unity_project_path.resolve()
        if not self.quiet:
            print(f"Created new Unity project: {str(unity_project_path.resolve())}")

        # Add the .unitypackage to the new project.
        package_name = "asset_bundle_creator.unitypackage"
        filepath = pkg_resources.resource_filename(__name__, package_name)
        assert Path(filepath).exists(), filepath

        # Import the package.
        call([str(AssetBundleCreator.get_editor_path().resolve()),
              "-projectPath",
              str(unity_project_path.resolve()),
              "-importPackage",
              filepath,
              "-quit",
              "-batchmode"], env=self.env)
        if not self.quiet:
            print("Imported asset_bundle_creator.unitypackage into the new project.")

        return unity_project_path

    def fbx_to_obj(self, model_path: Path) -> Tuple[Path, bool]:
        """
        Convert a .fbx file to a .obj file with assimp

        :param model_path: The path to the model.

        :return The path to the new object, and True if it's a new file (False if it's the existing base file).
        """

        if model_path.suffix != ".fbx":
            assert model_path.suffix == ".obj"
            if not self.quiet:
                print("Model is already a .obj file. Skipping the conversion to .obj")
            return model_path, False

        if not self.quiet:
            print("Converting a .fbx file to .obj")

        # Create the .obj file.
        obj_filename = model_path.stem + ".obj"

        assimp = pkg_resources.resource_filename(__name__, self.binaries["assimp"])
        assert Path(assimp).exists(), assimp

        # Run assimp to create the .obj file.
        call([assimp,
              "export",
              str(model_path.resolve()),
              obj_filename],
             stdout=open(os.devnull, "wb"))

        obj_path = Path(obj_filename)
        assert obj_path.exists(), "Failed to convert .fbx to .obj"

        mtl_path = Path(obj_filename + ".mtl")
        if mtl_path.exists():
            mtl_path.unlink()
            if not self.quiet:
                print("Removed superfluous .obj.mtl file")

        return obj_path, True

    def obj_to_wrl(self, model_path: Path, vhacd_resolution: int = 8000000) -> Path:
        """
        Convert a .obj file to a .wrl file with testVHACD

        :param model_path: The path to the model.
        :param vhacd_resolution: The V-HACD voxel resolution. A higher number will create more accurate physics colliders, but it will take more time to initially create the asset bundle.

        :return The path to the .wrl file.
        """

        # Get the target .obj file.
        if model_path.suffix == ".obj":
            obj_path = str(model_path.resolve())
        else:
            obj_path = model_path.stem + ".obj"
        assert Path(obj_path).exists(), f"Target .obj doesn't exist: {obj_path}"

        wrl_filename = model_path.stem + ".wrl"
        if not self.quiet:
            print("Running V-HACD on a .obj file (this might take awhile).")

        # Run V-HACD.
        vhacd = pkg_resources.resource_filename(__name__, self.binaries["vhacd"])

        assert Path(vhacd).exists(), vhacd
        call([vhacd,
              "--input", obj_path,
              "--resolution", str(vhacd_resolution),
              "--output", wrl_filename],
             stdout=open(os.devnull, "wb"))

        assert Path(wrl_filename).exists(), "Failed to create .wrl file. " \
                                            "Check your original file to make sure that is 3D " \
                                            "(not just a flat plane) and isn't corrupted."

        if not self.quiet:
            print(f"Created: {wrl_filename}")

        # Remove the log, if any.
        if Path("log.txt").exists():
            os.remove("log.txt")

            if not self.quiet:
                print("Removed V-HACD log file.")

        # Remove the superfluous .obj file.
        if model_path.suffix != ".obj":
            os.remove(obj_path)

            if not self.quiet:
                print(f"Removed superfluous .obj file: {obj_path}")

        return Path(wrl_filename)

    def wrl_to_obj(self, wrl_filename: Path, model_name: str) -> Path:
        """
        Convert a .wrl file back into a .obj file with meshconv

        :param wrl_filename: The to the .wrl file.
        :param model_name: The name of the model (minus its file extension).

        :return The path to the .obj file.
        """

        assert wrl_filename.exists(), f"Missing .wrl file: {wrl_filename}"

        if not self.quiet:
            print("Converting .wrl to .obj")

        # Run meshconv.
        meshconv = pkg_resources.resource_filename(__name__, self.binaries["meshconv"])
        assert Path(meshconv).exists(), meshconv
        call([meshconv,
              str(wrl_filename.resolve()),
              "-c",
              "obj",
              "-o",
              f"{model_name}_colliders",
              "-sg"],
             stdout=open(os.devnull, "wb"))

        obj_path = Path(f"{model_name}_colliders.obj")
        assert obj_path.exists(), "Failed to create .obj file from .wrl file."

        if not self.quiet:
            print(f"Created {obj_path.resolve()} from .wrl file.")

        return obj_path

    def move_files_to_unity_project(self, obj_colliders: Optional[Path], model_path: Path, sub_directory: str = "") -> List[Path]:
        """
        Moves all required files to the Unity project.

        :param obj_colliders: The path to the colliders .obj file. May be None.
        :param sub_directory: Optional subdirectory to move the files to.
        :param model_path: The path to the model. Can be a Path object, or a string representing the absolute file path.

        :return A list of paths of files in the Unity project.
        """

        if obj_colliders is not None:
            assert obj_colliders.exists(), f"Colliders .obj file not found: {obj_colliders.resolve()}"

        resources_directory = self.get_resources_directory()

        if not self.quiet:
            print("Copying files into the Unity project.")

        mtl = model_path.parent.joinpath(Path(model_path.stem + ".mtl"))

        dests = []
        sources = [model_path, model_path.parent.joinpath("Materials"), model_path.parent.joinpath("Textures"), mtl]
        if obj_colliders is not None:
            sources.append(obj_colliders)

        # Open the mtl file and try to parse for textures
        if mtl.exists():
            mtl_txt = mtl.read_text()
            for mtl_line in mtl_txt.split("\n"):
                if ".jpg" in mtl_line:
                    tex_path = model_path.parent.joinpath(mtl_line.split(" ")[1])
                    assert tex_path.exists(), f"Missing texture: {tex_path}"
                    sources.append(tex_path)
        for src in sources:
            if src is None or not src.exists():
                continue
            if sub_directory == "":
                dest = resources_directory.joinpath(src.stem + src.suffix)
            else:
                dest = resources_directory.joinpath(sub_directory).joinpath(src.stem + src.suffix)
            # Copy to the destination.
            if src.is_file():
                if not dest.parent.exists():
                    Path(dest.parent).mkdir(parents=True)
                distutils.file_util.copy_file(str(src.resolve()), str(dest.resolve()))
            else:
                distutils.dir_util.copy_tree(str(src.resolve()), str(dest.resolve()))
            dests.append(dest)
        if not self.quiet:
            print("Copied files into the Unity project.")
        return dests

    def create_prefab(self, colliders: str, model_name: str, model_extension: str) -> Tuple[Path, Path]:
        """
        Create a prefab from the files existing in the Unity project folder.

        :param colliders: The colliders filename.
        :param model_name: The name of the model, minus its file extension.
        :param model_extension: The file extension of the model (e.g. ".obj").

        :return The path to the prefab and the path to the report (if any).
        """

        report = self.get_assets_directory().joinpath("report.txt")
        if report.exists():
            os.remove(str(report.resolve()))
            if not self.quiet:
                print("Removed old report.")

        # Create the prefab.
        if not self.quiet:
            print("Creating the prefab...")
        prefab_call = self.unity_call[:]
        prefab_call.extend(["-executeMethod",
                            "AssetBundleCreator.CreatePrefab",
                            "-filename=" + model_name,
                            "-modelname=" + model_name,
                            "-extension=" + model_extension,
                            "-colliders=" + colliders
                            ])
        call(prefab_call, env=self.env)
        prefab_path = self.get_resources_directory().joinpath(f"prefab/{model_name}.prefab")
        assert prefab_path.exists(), "Failed to create prefab."

        # Check the report.
        if report.exists():
            raise Exception(f"Created the prefab with errors: {report.read_text()}")

        if not self.quiet:
            print("Created the prefab.")

        return prefab_path, report

    def prefab_to_asset_bundle(self, prefab_path: Path, model_name: str, platforms: List[str] = None) -> List[Path]:
        """
        Given a .prefab, create asset bundles and write them to disk.

        :param prefab_path: The path to the .prefab file.
        :param model_name: The name of the model, minus its file extension.
        :param platforms: Platforms to build asset bundles for. Options: "windows", "osx", "linux". If None, build all.

        :return The paths to the asset bundles.
        """

        if platforms is None:
            #platforms = ["windows", "osx", "linux"]
            platforms = ["linux"]

        assert prefab_path.exists(), f"Missing prefab: {prefab_path.resolve()}"

        if not self.quiet:
            print("Creating local asset bundles")
        asset_bundle_call = self.unity_call[:]
        platforms_call = ""
        for p in platforms:
            platforms_call += p + ","
        platforms_call = platforms_call[:-1]
        asset_bundle_call.extend(["-executeMethod",
                                  "AssetBundleCreator.BuildAssetBundle",
                                  "-modelname=" + model_name,
                                  "-platforms=" + platforms_call
                                  ])
        call(asset_bundle_call, env=self.env)
        new_asset_bundles_directory = self.get_assets_directory().joinpath("NewAssetBundles")
        new_asset_bundles_directory = new_asset_bundles_directory.joinpath(model_name)
        assert new_asset_bundles_directory.exists(), f"No asset bundles found: {new_asset_bundles_directory.resolve()}"

        paths = []

        # Iterate through all target platforms and build asset bundles.
        for platform_key in platforms:
            asset_bundle_platform = S3_TO_UNITY[platform_key]
            bundle_path = new_asset_bundles_directory.joinpath(asset_bundle_platform).joinpath(model_name)
            assert bundle_path.exists(), f"Missing asset bundle: {asset_bundle_platform}"
            paths.append(bundle_path)

        if not self.quiet:
            print("Created local asset bundles.")

        return paths

    @staticmethod
    def get_local_asset_bundle_path(model_name: str) -> Path:
        """
        :param model_name: The name of the model, minus its file extension.

        :return The expected path of the local asset bundle for this platform.
        """

        return Path.home().joinpath("asset_bundle_creator/Assets/NewAssetBundles").\
            joinpath(model_name + "/" + SYSTEM_TO_UNITY[platform.system()] + "/" + model_name)

    def create_record(self, model_name: str, wnid: int, wcategory: str, scale: float, urls: Dict[str, str], record: Optional[ModelRecord] = None, write_physics: bool = False) -> Path:
        """
        Create a local .json metadata record of the model.

        :param model_name: The name of the model.
        :param wnid: The WordNet ID.
        :param wcategory: The WordNet category.
        :param scale: The default scale of the object.
        :param urls: The finalized URLs (or local filepaths) of the assset bundles.
        :param record: A pre-written metadata record. If not None, it will override the other parameters.
        :param write_physics: If true, launch the build to write the physics quality. (This is optional).

        :return The path to the file with the metadata record.
        """

        # Write the record.
        if not self.quiet:
            print("Creating a record.")

        if record is None:
            record = ModelRecord()
            record.name = model_name
            record.wnid = f'n{wnid:08d}'
            record.wcategory = wcategory
            record.urls = urls
            record.scale = scale

        # Append asset bundle sizes.
        local_path = Path.home().joinpath("asset_bundle_creator/Assets/NewAssetBundles").joinpath(record.name)
        print(local_path)
        for os_dir in local_path.iterdir():
            if not os_dir.is_dir():
                continue
            if str(os_dir) != '/home/franka/asset_bundle_creator/Assets/NewAssetBundles/StandaloneLinux64':
                continue
            asset_bundle_platform = UNITY_TO_SYSTEM[os_dir.stem]
            size = os_dir.joinpath(record.name).stat().st_size
            record.asset_bundle_sizes[asset_bundle_platform] = size

        # Assemble a dictionary of just the data that we don't need the Editor for.
        record_data = {"name": record.name,
                       "urls": record.urls,
                       "wnid": record.wnid,
                       "wcategory": record.wcategory,
                       "scale_factor": record.scale_factor,
                       "do_not_use": record.do_not_use,
                       "do_not_use_reason": record.do_not_use_reason,
                       "canonical_rotation": record.canonical_rotation,
                       "physics_quality": -1,
                       "asset_bundle_sizes": record.asset_bundle_sizes}

        # Serialize the record.
        record_data = json.dumps(record_data)
        # Remove the last } and replace it with , to keep serializing with Unity.
        record_data = record_data[:-1] + ","

        record_path = self.get_assets_directory().joinpath(model_name + ".json")
        record_path.write_text(record_data, encoding="utf-8")

        record_call = self.unity_call[:]
        record_call.extend(["-executeMethod",
                            "RecordCreator.WriteRecord",
                            "-modelname=" + model_name,
                            "-scale=" + str(scale)])
        call(record_call, env=self.env)

        # Test the record.
        try:
            json.loads(record_path.read_text(encoding="utf-8"))
        except json.JSONDecodeError:
            raise Exception("Failed to deserialize: " + record_path.read_text(encoding="utf-8"))

        if not self.quiet:
            print("Wrote the record data to the disk.")

        if write_physics:
            self.write_physics_quality(record_path=record_path,
                                       asset_bundle_path=self.get_local_asset_bundle_path(model_name))

        return record_path

    @staticmethod
    def get_local_urls(asset_bundle_paths: List[Path]) -> Dict[str, str]:
        """
        Generate a dictionary of local URLs from the asset bundle paths.

        :param asset_bundle_paths: The asset bundle paths.

        :return A dictionary. Key = OS, Value = Path to the local file.
        """

        urls: Dict[str, str] = {}
        for ap in asset_bundle_paths:
            ap = "file:///" + str(ap.resolve()).replace("\\", "/")
            if "StandaloneWindows64" in ap:
                urls.update({"Windows": ap})
            elif "StandaloneOSX" in ap:
                urls.update({"Darwin": ap})
            elif "StandaloneLinux64" in ap:
                urls.update({"Linux": ap})
        return urls

    @staticmethod
    def write_physics_quality(record_path: Path, asset_bundle_path: Path) -> None:
        """
        Append the physics quality data to the temporary record file.
        This is an optional record field that records the percentage of the model encapsualted by colliders.

        :param record_path: The path to the temporary record file.
        :param asset_bundle_path: The URL to the local asset bundle.
        """

        # Get the path to the writer controller.
        writer_path = pkg_resources.resource_filename(__name__, "model_pipeline/write_physics_quality.py")
        writer_call = ["py", "-3", writer_path,
                       "--record_path", str(record_path.resolve()),
                       "--asset_bundle_path", str(asset_bundle_path.resolve())]

        call(writer_call)

    def validate(self, record_path: Path, asset_bundle_path: Path) -> Tuple[bool, str]:
        """
        Validate the asset bundle.

        :param record_path: The path to the temporary record file.
        :param asset_bundle_path: The URL to the local asset bundle.

        :return True if there aren't problems, and a string output report.
        """

        if not self.quiet:
            print("Validating asset bundle...")

        validator_path = pkg_resources.resource_filename(__name__, "model_pipeline/validator.py")
        validator_call = ["py", "-3", validator_path,
                          "--record_path", str(record_path.resolve()),
                          "--asset_bundle_path", str(asset_bundle_path.resolve())]
        call(validator_call)

        report = json.loads(paths.VALIDATOR_REPORT_PATH.read_text(encoding="utf-8"))
        if not report["ok"]:
            output = "There are problems with the asset bundle!"
            for problem in report["reports"]:
                output += "\n\t" + problem
            return False, output
        if not self.quiet:
            print("OK!")
        return True, ""

    def create_many_asset_bundles(self, library_path: str, cleanup: bool = True, vhacd_resolution: int = 8000000) -> None:
        """
        Create asset bundles in batch from .obj or .fbx files already in: asset_bundle_creator/Assets/Resources/models
        This function will create collider .obj files if there aren't any already
        (it will look for a file named <model>_colliders.obj).

        :param library_path: The path to the library file.
        :param cleanup: If true, remove all temp files when done.
        :param vhacd_resolution: The V-HACD voxel resolution. A higher number will create more accurate physics colliders, but it will take more time to initially create the asset bundle.
        """

        # Create for each model that doesn't have colliders.
        models_dir = self.get_resources_directory().joinpath("models")
        for ext in [".obj", ".fbx"]:
            for f in models_dir.rglob(f"*{ext}"):
                if f.stem.endswith("_colliders"):
                    continue
                colliders_path = f.parent.joinpath(f"{f.stem}_colliders.obj")
                # Don't regenerate a colliders file if one already exists.
                if colliders_path.exists():
                    continue
                # Don't recreate models that already have prefabs.
                prefab_path = self.get_resources_directory().joinpath(f"prefab/{f.stem}.prefab")
                if prefab_path.exists():
                    continue
                # Create the colliders.
                wrl_path = self.obj_to_wrl(f, vhacd_resolution=vhacd_resolution)
                # Move the collider .obj to the correct directory.
                wrl_to_obj_path = self.wrl_to_obj(wrl_path, f.stem)
                distutils.file_util.move_file(str(wrl_to_obj_path.resolve()), str(colliders_path.resolve()))
                # Remove the .wrl file.
                wrl_path.unlink()

        # Create the asset bundles.
        record_call = self.unity_call[:]
        record_call.extend(["-executeMethod",
                            "AssetBundleCreator.CreateManyAssetBundles",
                            "-library=" + library_path,
                            "-internal_materials"])
        call(record_call)

        if cleanup:
            root_dir = self.get_assets_directory()
            # Remove assets used to create asset bundles.
            for ext in [".obj", ".fbx", ".mtl", ".mat", ".jpg", ".prefab"]:
                for f in root_dir.rglob(f"*{ext}"):
                    f.unlink()
            # Remove asset bundle junk.
            distutils.dir_util.remove_tree(str(self.get_assets_directory().joinpath("NewAssetBundles").resolve()))
            # Remove models junk.
            for d in models_dir.iterdir():
                if "placeholder" in d.stem:
                    continue
                if d.is_file():
                    d.unlink()
                else:
                    distutils.dir_util.remove_tree(str(d.resolve()))

Feature request: singularity file

Hi,

Clusters tend not to support docker, but Singularity containers, which are less powerful -- and thus less dangerous.

Could you share a Singularity definition file?
A few tools exist for converting a Dockerfile into a Singularity definition file, but it will contain a few mistakes.

I can create such file once I will be more familiarized with TDW.

Thanks!

Can't draw Flex particles on Linux

It's currently not possible to draw Flex particles ("draw_particles" in the Command API) in Linux. Attempting to create a build with the Flex particle shaders for Linux results in a crash-to-desktop. This is new as of Unity 2020.2.2 and it is likely that a future Unity Engine upgrade (i.e. to Unity 2020.2.x) will fix it. As a matter of course, we apply minor Unity Engine updates to TDW whenever they're available.

`add_fixed_joint` to be deprecated?

I noticed that the add_fixed_joint command to attach one object to another is deprecated and slated for removal, though it hasn't been removed yet.

Is there another command that will replace it? Being able to rigidly attach two objects is a useful feature.

jupyter lab issue creating `Controller`

tdw fails initializing a Controller when it runs in a jupyter lab notebook. I do not have this issue when running from the python terminal.

I am running the first script under "Getting Started":

from tdw.controller import Controller

c = Controller()
print("Everything is OK!")

jupyter lab gives this stack trace:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-8a8dea7f6837> in <module>
----> 1 c = Controller()
      2 print("Everything is OK!")

~/anaconda3/envs/ai/lib/python3.6/site-packages/tdw/controller.py in __init__(self, port, check_version, launch_build)
     38         # If there is a difference, recommend an upgrade.
     39         if check_version:
---> 40             self._check_pypi_version()
     41 
     42         # Launch the build.

~/anaconda3/envs/ai/lib/python3.6/site-packages/tdw/controller.py in _check_pypi_version(v_installed_override, v_pypi_override)
    343         installed_tdw_version = PyPi.get_installed_tdw_version()
    344         # Get the latest version of the tdw module on PyPi.
--> 345         pypi_version = PyPi.get_pypi_version()
    346 
    347         # Apply overrides

~/anaconda3/envs/ai/lib/python3.6/site-packages/tdw/release/pypi.py in get_pypi_version(truncate)
     58 
     59         # From the list of available versions, get the last one (the most recent).
---> 60         v = PyPi._get_pypi_releases()[-1]
     61 
     62         # Strip the post-release suffix.

~/anaconda3/envs/ai/lib/python3.6/site-packages/tdw/release/pypi.py in _get_pypi_releases()
     46         stdout, stderr = p.communicate()
     47         # From the list of available versions, get the last one (the most recent).
---> 48         versions = re.search(r"\(from versions: (.*)\)", stderr.decode("utf-8")).group(1).split(",")
     49         return [v.strip() for v in versions]
     50 

AttributeError: 'NoneType' object has no attribute 'group'

My platform

Ubuntu 20.04.1 LTS

jupyter lab --version
2.2.6

python --version
Python 3.6.12 :: Anaconda, Inc.

Unexpected white cube during generation

TDW version: 1.8.7
Player.log file:
`Mono path[0] = 'C:/Users/eliwang/tdw_build/TDW/TDW_Data/Managed'
Mono config path = 'C:/Users/eliwang/tdw_build/TDW/MonoBleedingEdge/etc'
Initialize engine version: 2020.2.7f1 (c53830e277f1)
[Subsystems] Discovering subsystems at path C:/Users/eliwang/tdw_build/TDW/TDW_Data/UnitySubsystems
[Subsystems] No descriptors matched for examples in UnitySubsystems/OculusXRPlugin/UnitySubsystemsManifest.json.
[Subsystems] 1 'inputs' descriptors matched in UnitySubsystems/OculusXRPlugin/UnitySubsystemsManifest.json
[Subsystems] 1 'displays' descriptors matched in UnitySubsystems/OculusXRPlugin/UnitySubsystemsManifest.json
[Subsystems] No descriptors matched for meshings in UnitySubsystems/OculusXRPlugin/UnitySubsystemsManifest.json.
[Subsystems] No descriptors matched for examples in UnitySubsystems/WindowsMRXRSDK/UnitySubsystemsManifest.json.
[Subsystems] 1 'inputs' descriptors matched in UnitySubsystems/WindowsMRXRSDK/UnitySubsystemsManifest.json
[Subsystems] 1 'displays' descriptors matched in UnitySubsystems/WindowsMRXRSDK/UnitySubsystemsManifest.json
[Subsystems] 1 'meshings' descriptors matched in UnitySubsystems/WindowsMRXRSDK/UnitySubsystemsManifest.json
GfxDevice: creating device client; threaded=1
Direct3D:
Version: Direct3D 11.0 [level 11.1]
Renderer: NVIDIA GeForce GTX 1080 (ID=0x1b80)
Vendor:
VRAM: 8079 MB
Driver: 27.21.14.5671
Begin MonoManager ReloadAssembly

Completed reload, in 0.124 seconds
XRGeneral Settings awakening...

D3D11 device created for Microsoft Media Foundation video decoding.
[XR] OculusXRPlugin: registering subsystems
[Subsystems] OculusXRPlugin successfully registered Provider for oculus display
[XR] Oculus Input lifecycle provider registered.
[Subsystems] OculusXRPlugin successfully registered Provider for oculus input
[Subsystems] OculusXRPlugin successfully registered Provider for oculus display
Initializing input.

XInput1_3.dll not found. Trying XInput9_1_0.dll instead...
Input initialized.

Initialized touch support.

UnloadTime: 1.235900 ms
Setting up 24 worker threads for Enlighten.
Thread -> id: 64dc -> priority: 1
Thread -> id: 7654 -> priority: 1
Thread -> id: 9e5c -> priority: 1
Thread -> id: 96fc -> priority: 1
Thread -> id: 99b4 -> priority: 1
Thread -> id: 247c -> priority: 1
Thread -> id: 8bec -> priority: 1
Thread -> id: 70ac -> priority: 1
Thread -> id: 8a04 -> priority: 1
Thread -> id: a6e0 -> priority: 1
Thread -> id: 90a0 -> priority: 1
Thread -> id: a3c8 -> priority: 1
Thread -> id: 2bf8 -> priority: 1
Thread -> id: a334 -> priority: 1
Thread -> id: 6f34 -> priority: 1
Thread -> id: a688 -> priority: 1
Thread -> id: 92ec -> priority: 1
Thread -> id: 3154 -> priority: 1
Thread -> id: 8c8c -> priority: 1
Thread -> id: 4a1c -> priority: 1
Thread -> id: 5780 -> priority: 1
Thread -> id: 2388 -> priority: 1
Thread -> id: 9f28 -> priority: 1
Thread -> id: 8fd4 -> priority: 1
Object cube isn't a cached object. [TDW.CollisionStayListener]

Object cube isn't a cached object. [TDW.CollisionStayListener]

Object with ID 1 already exists in the cache. This object has been given a new ID: -968722629 [TDW.ProcGenSceneRoot]

ArgumentException: An item with the same key has already been added. Key: 1
at System.Collections.Generic.Dictionary2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) [0x000c1] in <eae584ce26bc40229c1b1aa476bfa589>:0 at System.Collections.Generic.Dictionary2[TKey,TValue].Add (TKey key, TValue value) [0x00000] in :0
at TDW.ProcGenSceneRoot.CacheObject (TDW.CachedObjectBase c) [0x00000] in <0692d6301a894e8d861ae9b3bb7ed472>:0
at TDW.ProcGenSceneRoot.CacheModel (TDW.CachedModel c) [0x00000] in <0692d6301a894e8d861ae9b3bb7ed472>:0
at TDWInput.AddModelCommand.End (UnityEngine.GameObject obj) [0x000c6] in <0692d6301a894e8d861ae9b3bb7ed472>:0
at TDWInput.AddObject.End (UnityEngine.GameObject obj) [0x00000] in <0692d6301a894e8d861ae9b3bb7ed472>:0
at TDWInput.AssetBundleCommand1+<DownloadAssetBundle>d__16[T].MoveNext () [0x0008f] in <0692d6301a894e8d861ae9b3bb7ed472>:0 at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00026] in <5f528563e9f04651878c6c899feba4b9>:0 UnityEngine.MonoBehaviour:StartCoroutineManaged2(IEnumerator) UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator) TDWInput.AssetBundleCommand1:Do()
NetworkManager:Unpack(NetMQMessage&)
Req:Update()

Segfaults in Linux

Possibly caused by error handling in v1.8.15?

single_object.py

Hi,

I just got started with TDW and it so far looks all great :)

I have however three questions to the single-object case - this is probably closest to what we plan to do:

it refers to "Martin Schrimpf's and Jonas Kubilius' ImageNet transfer research" is there a paper or something that shows me what that project was?
it renders in the highest possible quality, but then saves the files in jpg format with strong compression artifacts - is that intended?
I tried to start with --train=1 --val=1 and it seems like those values are just ignored?

Best
Bernhard

get_depth_values causes overflow

The depth_shader.py example does not render correct depth.

tdw/Python/tdw/tdw_utils.py

Line 382 in 8ad5c33

image = np.array(Image.open(io.BytesIO(image)))

image.dtype is uint8 by default, which causes overflow when multiplying 256*256

image = np.array(Image.open(io.BytesIO(image)), dtype="uint32") could solve the issue.

Unity is Black-and-White and Very Dark

Hello,

I've been running some of your sample scripts and the images in Unity (both in the run window and when saved) are black-and-white and very dark.

Here's an example from your Getting Started page.

Here are my versions:

Build version 1.6.3
Unity Engine 2019.2.21f1
Python tdw module version 1.6.3
Python version 3.6.5

Thanks,

Thomas

What is the capacity of container?

Hi,

I am looking for a detail that I did not find in the original paper -- what is the capacity of the containers that the agent can use to carry multiple objects? Is it infinite, a fixed value, or anything else?

Thanks,
Yash

Thanks!

Improve scene documentation

Include the names and images of every scene
Copy floorplan images from Magnebot

threedworld-mit / tdw Goto Github PK

tdw's People

Stargazers

Watchers

Forkers

tdw's Issues

My platform

Recommend Projects

Recommend Topics

Recommend Org