Comments (3)
Hi sorry about the late reply -- the goal of the model is to learn pick and place one single block (given other already stacked blocks)
In the second setting -- the model predicts when to form contacts with individual blocks. When the model is not trained well, the contacts will be incorrectly added, which leads to very odd physics simulation.
Feel free to let me know if you have any other questions.
from diffuser.
@yilundu , Thank you very much for you reply. Unfortunately, I didn't understand.
Let me write my question in little bit detail.
The part before training
You have this kind of dataset , kuka_dataset/data_897.npy, kuka_dataset/data_898.npy .... and diffuser gets this dataset and trains the model.
When I check your dataset (e.g kuka_dataset/data_897.npy) , the dataset consists of 470x39 parameters.
(The first axis of the dataset) The 470 is timestep t
(The second axis of the dataset) 7 variables from the 39 dimension are the joint variables of the robot. The rest of the parameters (32 parameters) is the position as well as the orientation of the 4 cubes ( 3 pos and 4 rot for each cube)
Basically, the data has the trajectory points of each cube as well as the joint values for the robot in each timestep.
If I render the following datasets (without starting any training), I can already see that the robot picks objects and places them. ( In the following video)
tmp = np.load("kuka_dataset/data_897.npy")
renderer.renders(tmp)
Here is the rendering of the dataset data_897
2023-03-27_16-05-38.mp4
tmp = np.load("kuka_dataset/data_898.npy")
renderer.renders(tmp)
Here is the rendering of the dataset data_898
https://user-images.githubusercontent.com/81103451/227969534-5ec34a13-8f73-4c9a-ae01-4c5e204db0ea.mp4
So, the data was literally trained, and the robot learned how to pick and place. You start with a learned policy.
The part after training
Then I start training using the pretrained model model-650.pt, then the error is around 0.020
Then diffision model samples the trajectories points from the below code line ( here https://github.com/jannerm/diffuser/blob/kuka/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py#L806 )
When I check the dimension of the variable unnormed, It has size
unnormed.shape
torch.Size([1, 128, 39])
It means that It has joint values as well as pos and rot for each cube for 128 timesteps. When I render this data ( the sampled data from the diffusion model), I see the following behaviour;
2023-03-27_16-03-32.mp4
So, It is simply the similar behaviour before starting to the training.
You are giving some data from a learned policy(e.g robot pick and place the cube). The diffuser model gets trajectory points of the cube as well as joints of the robot, It trains it and obtains some trajectory points(still robot picks and place the cube). ( If I understand correctly)
So my question is;
The robot already picks and place the object before training. After the training, It has still the same behavior. What is the objective of the training?
from diffuser.
Ahh I see -- yes the objective of training is to learn how to pick / place blocks from a large dataset of existing demonstrations. The goal is to generalize to new configuration of blocks seen at test time, as well as pick and place blocks in a controllable manner.
from diffuser.
Related Issues (20)
- Doubt on Action Space HOT 1
- Inquiry About the Tool Used for 3D Plotting in Your Impressive Project HOT 1
- when i install gym==0.18.0 use pip, HOT 5
- Clarification Required for the Use of "if not is_last:" Condition
- OSError: [Errno 32] Broken pipe occured in running plan_guided.py
- Detail of maze2d experiment HOT 4
- Rendering problem with Maze2d environments HOT 8
- error running plan_guided.py
- About variable-length plans
- Error loading weights for maze_2d HOT 1
- A question about the antmaze bug
- The demo animation on the project website
- Maze2d Start Point
- I use the gym 0.19.0 nut it has wrongs whch says "No registered env with id: halfcheetah-medium-expert-v2" HOT 2
- conda env create -f environment.yml HOT 9
- Dataset availability HOT 2
- How did you generated datas for KUKA? HOT 2
- The question about "from tap import Tap" HOT 1
- How is conditioning implemented HOT 1
- error in kuka branch
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diffuser.