Visual Servoing: ToDOs about conq_python HOT 4 OPEN

Oversight:

1. Achieves only pose, and not orientation (PBVS accounts for only cartesian pose)

• Align to object/grasp pose (position, orientation)
• Future: Track Delta p_{t}^{t-1} Reference to tracking issue

2. Need to hover with offset along z_axis of grasp/object pose (preferably grasp pose)

• Do the math for hovering over object along z-axis of grasp pose
• Radius of sphere = offset

3. New objective of visual servoing:

• Gripper frame needs to decrease the error between the offset grasp pose on the sphere and current pose
• Might need to use the same controller used to decrease the error between robot current pose and robot measured pose
• When error is within a specific threshold, decrease z linearly to go to true grasp pose

from conq_python.

TODOs post discussion (11-March-2024):

• Get the centroid of object (roboflowv/ lang-prompted)
• Get waypoints of camera/hand pose based on generated grasp pose (Initial trajectory)
• Best View Point Planning #50
• • Objective is to maximize the view of the tool from ray-tracing until a specific threshold(distance between camera and object/grasp pose) is achieved (because further than that)
• • Modify trajectory based on the planner
• Perform normal visual-servoing controller

from conq_python.

Manipulation check with/without VisualServoing

• Precision and Repeatability check
• • Without visual servoing
• • With Visual servoing (Use AprilTag)
• • Maybe do same experiment with impedance with maximum stiffness?

from conq_python.

• XMem Object video segmentation
• • https://github.com/hkchengrex/XMem
• • Checkout colab notebook: https://colab.research.google.com/drive/1RXK5QsUo2-CnOiy5AOSjoZggPVHOPh1m?usp=sharing#scrollTo=qYv3kbaQT2w4
• Segment point-cloud based on this segmentation #49

from conq_python.