mandizhao / robot-collab Goto Github PK
View Code? Open in Web Editor NEWCodebase for paper: RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
License: MIT License
Codebase for paper: RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
License: MIT License
Dear MandiZhao,when I try to run Task: Pack Grocery,often when I get past the first round of dialogue, the LLM keeps talking about the same behavior and can't plan the right action to execute.
the feedback likes as following:
INFO:root:Step: 13 LLM plan parsed, begin RRT planning
tograsp: {'Bob': ('banana', 'banana_top', 0), 'Alice': ('apple', 'apple_top', 0)} inhand: {'Bob': ('banana', 'banana_top', 'banana_joint'), 'Alice': ('apple', 'apple_top', 'apple_joint')} {'Bob': 'PLACE banana bin_front_left PATH [(-0.25, 0.40, 0.3), (0.0, 0.40, 0.3), (0.12, 0.40, 0.3), (0.25, 0.41, 0.3)]', 'Alice': 'PLACE apple bin_front_right PATH [(-0.73, 0.40, 0.3), (-0.48, 0.40, 0.3), (-0.23, 0.40, 0.3), (0.65, 0.41, 0.3)]'}
Given 4 waypoints, found 4 valid waypoints via IK
=====================================
init-sampling! 1
init-sampling! 2
init-sampling! 3
init-sampling! 4
failed to find a path, reason: RRT failed: ReasonTimeout_time221.55805015563965_iter9
Can you give me some advices?Thank you very much!
用gpt4尝试了grocery的任务,两个机械臂只能完成两个正确的step,后续的5个step一直存在collision,没法继续进行,我的gpt4余额都快花完了:(
python run_dialog.py --task pack -llm gpt-4
File "run_dialog.py", line 35
env: MujocoSimEnv,
^
SyntaxError: invalid syntax
Ubuntu20.04 System, I want to apply the command in my OS.
When I try to run python run_dialog.py --task rope
, I get the following error:
ValueError: Error: material 'stove-metal' not found in geom 145
Object name = groove_front, id = 145, line = 55, column = -1
Hello, I have an issue when run python run_dialog.py --task pack -llm gpt-4
WARNING:root:PackGroceryTask requires split parsed plans, and no failed waypoints, no direct waypoints
Traceback (most recent call last):
File "C:\Anaconda\envs\roco\run_dialog.py", line 494, in
main(args)
File "C:\Anaconda\envs\roco\run_dialog.py", line 412, in main
env = env_cl(
^^^^^^^
File "C:\Anaconda\envs\roco\rocobench\envs\task_pack.py", line 91, in init
super(PackGroceryTask, self).init(
File "C:\Anaconda\envs\roco\rocobench\envs\base_env.py", line 216, in init
self.physics = dm_mujoco.Physics.from_xml_path(filepath)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Anaconda\Lib\site-packages\dm_control\mujoco\engine.py", line 455, in from_xml_path
return cls.from_model(model)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Anaconda\Lib\site-packages\dm_control\mujoco\engine.py", line 419, in from_model
return cls(data)
^^^^^^^^^
File "C:\Anaconda\Lib\site-packages\dm_control\mujoco\engine.py", line 123, in init
self._reload_from_data(data)
File "C:\Anaconda\Lib\site-packages\dm_control\mujoco\engine.py", line 400, in _reload_from_data
model=index.struct_indexer(self.model, 'mjmodel', axis_indexers),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Anaconda\Lib\site-packages\dm_control\mujoco\index.py", line 623, in struct_indexer
attr = getattr(struct, field_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'MjModel' object has no attribute 'eq_active'. Did you mean: 'eq_active0'?
What should I do to revise it? Many thanks.
The bin and soda_can are missing in the rendered video of the pack_grocery task, and the blue cup is missing in the rendered video of the open_cabinet task.
I noticed their mesh is commented out in the XML file https://github.com/MandiZhao/robot-collab/blob/main/rocobench/envs/task_pack.xml#L140, but after I uncommented them, their physics is weird: the soda_can will roll down the table without contact, and the mug can never be placed on the coaster correctly.
There seems something wrong with the XML file, could you kindly update it to the correct one you used in your experiments or guide me on how to fix this issue? Thanks!
assert llm_source in ["gpt-4", "gpt-3.5-turbo", "claude"], f"llm_source must be one of [gpt4, gpt-3.5-turbo, claude], got {llm_source}"
AssertionError: llm_source must be one of [gpt4, gpt-3.5-turbo, claude], got gpt4
I didn't get claude api key, but I get gpt4, do I need to get both of them?thanks.
Can you take a look at this?
Traceback (most recent call last):
File "run_dialog.py", line 494, in <module>
main(args)
File "run_dialog.py", line 463, in main
runner.run(args)
File "run_dialog.py", line 380, in run
self.one_run(run_id)
File "run_dialog.py", line 223, in one_run
self.display_plan(plan, save_name=f"vis_llm_plan_{i}", save_dir=step_dir)
File "run_dialog.py", line 156, in display_plan
raise NotImplementedError
NotImplementedError
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.