Hi guys, congrats on making such an interesting project.

How can we source the data from the example files? I got the vista_traces file but i dont know how to connect it to the program. When i import vista i get:
2022-09-04 19:37:06,783::WARNING::[vista.entities.sensors.EventCamera.<module>] Fail to import module for event camera. Remember to do source <some-dir>/openeb/build/utils/scripts/setup_env.shCan ignore this if not using it

There must be a simple solution to this problem, i searched the docs but i could not find the explaination for this.

Hi Amini, thanks for sharing the surprising and very useful project. I am very intresting in the data-driven simulation of RGB camera. So I read your paper 'Learning Robust Control Policies for End-to-End Autonomous Driving from Data-Driven Simulation'. This part make me confused
'From the single closest monocular image, a depth map is estimated using a convolutional neural network using self-supervision of stereo cameras [28]. Using the estimated depth map and camera intrinsics, our algorithm projects from the sensor frame into the 3D world frame'
Could you please give me more details about how to use the depth map and camera intrinsics to make a 3D world frame and use the relative transformation to count next observation.
Best Regards

Hi,

I have a couple of questions.

1. When I try to run examples/basic_usage/sim_event, it asks for a RIG.xml file. Where can I find this file?
2. How can I synthesize the view, and visualize it? Which script would be good to get started on generating views?

Best,

Hi, I have an image dataset, I want to use in vista. So I want to know if I can do this and how to do this if I can do.
Thanks in advance.

I am attempting to run sim_multi_car.py but I am missing a mesh directory.

Hello,

Congratulations on a great effort and thanks for sharing it with the public. I have a query regarding the sensor suite.
Query:

1. As there are handful of proprietery cameras like Intel D4115, D435, ZED1 and 2, OAK-D etc. how should someone be able to use those camera models into the current software stack of vista2.0.5?

Eager to know from you,

I've been trying to learn the control policy for custom datasets. And I can get quaternion values right in the file params.xml. I either get the wrong translation or even though visually the translations are more or less similar to the translations with the vista dataset. Nonetheless, the total reward eventually stuck around five and never recover afterward.

Hello All,
I am trying to run following basic code

import argparse
import numpy as np
import os
import cv2
import vista
from vista.utils import transform
from vista.entities.agents.Dynamics import tireangle2curvature
trace_path = ["/home/parvez_alam/Downloads/vista_traces/20210726-131322_lexus_devens_center"]

def main(trace_path):
   world = vista.World(trace_path, trace_config={'road_width': 4})
   car = world.spawn_agent(
   config={
        'length': 5.,
        'width': 2.,
        'wheel_base': 2.78,
        'steering_ratio': 14.7,
        'lookahead_road': True
    })

camera = car.spawn_camera(config={
    'size': (200, 320),
})
display = vista.Display(world)

world.reset()
display.reset()

while not car.done:
    action = follow_human_trajectory(car)
    car.step_dynamics(action)
    car.step_sensors()

    vis_img = display.render()
    cv2.imshow('Visualize RGB', vis_img[:, :, ::-1])
    cv2.waitKey(20)


def follow_human_trajectory(agent):
    action = np.array([
       agent.trace.f_curvature(agent.timestamp),
       agent.trace.f_speed(agent.timestamp)
    ])
    return action






main(trace_path)

But it is showing following error
2023-08-01 16:24:26,442::WARNING::[vista.entities.sensors.EventCamera.] Fail to import module for event camera. Remember to do source /openeb/build/utils/scripts/setup_env.shCan ignore this if not using it
2023-08-01 16:24:27,060::WARNING::[vista.core.Trace._divide_to_good_segments] No video_label.csv
2023-08-01 16:24:28,728::WARNING::[vista.entities.sensors.Camera.reset] No flow data
Traceback (most recent call last):
File "vista1.py", line 54, in
main(trace_path)
File "vista1.py", line 37, in main
vis_img = display.render()
File "/home/parvez_alam/.local/lib/python3.8/site-packages/vista/core/Display.py", line 227, in render
patch = PolygonPatch(patch,
File "/home/parvez_alam/.local/lib/python3.8/site-packages/descartes/patch.py", line 87, in PolygonPatch
return PathPatch(PolygonPath(polygon), **kwargs)
File "/home/parvez_alam/.local/lib/python3.8/site-packages/descartes/patch.py", line 62, in PolygonPath
vertices = concatenate([
File "/home/parvez_alam/.local/lib/python3.8/site-packages/descartes/patch.py", line 63, in
concatenate([asarray(t.exterior)[:, :2]] +
IndexError: too many indices for array: array is 0-dimensional, but 2 were indexed

I'm trying to run the sim_control.py and I'm encountering an error with the descartes package.

File "d:VISTA2.py", line 91, in
main()
File "d:VISTA2.py", line 34, in main
vis_img = display.render()
File "D:vista\lib\site-packages\vista\core\Display.py", line 227, in render
patch = PolygonPatch(patch,
File "D:vista\lib\site-packages\descartes\patch.py", line 87, in PolygonPatch
return PathPatch(PolygonPath(polygon), **kwargs)
File "D:vista\lib\site-packages\descartes\patch.py", line 62, in PolygonPath
vertices = concatenate([
File "D:vista\lib\site-packages\descartes\patch.py", line 63, in
concatenate([asarray(t.exterior)[:, :2]] +
IndexError: too many indices for array: array is 0-dimensional, but 2 were indexed

Any help would be appreciated

Description:

Dear contributors at MIT and the VISTA community, my name is Mark, and I am a computer science student in Taiwan. I have found out that some newcomers to the VISTA simulator couldn't run a basic example with ease. The issue of #6 and #4 have shown some problems with the environment set-up for the people who have never engaged with the VISTA simulator.

Solution:

Develop a Python script to automatically set up the environment needed for running the VISTA simulator and its examples.

@aamini and @zswang666 is there any channel for us to apply for the contributor for this repository? Or, could you specify the process to get involved in this project?

Mark Chen,
Student Researcher.

I have done !py -m pip install vista to install vista. When i try to do import vista i am getting this error.

If you have any idea how to get pass this error that would be amazing cause i am stuck at the moment. all of the other dependencies are installed. if you also have an idea for a better ide for this that would be great too. not a big fan of jupyter notebook.

Hello,
thanks for this work it's great !! For my project, I wanted to test the lidar sim functionality. I am running into some issues, seems like some resource files related to Lidar aren't pushed, which seems to be causing the issue.
the error I am getting.
self.avg_mask = np.load(str(rsrc_path / "Lidar/avg_mask2.npy"))
File "/home/amohan/python_env/vista_env/lib/python3.8/site-packages/numpy/lib/npyio.py", line 390, in load
fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: '/home/amohan/python_env/vista_env/lib/python3.8/site-packages/vista/resources/Lidar/avg_mask2.npy'

As you can see from the LidarSynthetisis code it requires some files

kindly push or provide a way to download those files if they already exist point me towards them.
Thanks in advance.

Hi,

I am trying to run basic examples (i.e sim_control.py, sim_lidar.py) after the installation, however, it gives me the following error "~/python3.8/site-packages/vista/core/World.py", line 32, in init
self._traces: List[Trace] = [
TypeError: 'NoneType' object is not iterable"

Any help would be appreciated.

Thanks.