error: (-215:Assertion failed) (int)detectedCharucoVecMat[i].total() * detectedCharucoVecMat[i].channels() == 2 in function 'cv::aruco::CharucoBoardImpl::matchImagePoints' about opencv HOT 6 OPEN

I'd be more than happy to take over the issue if someone can do a basic code walkthrough for me, regarding what all places do I need to look into and such.

from opencv.

I have no specific ideas about what causes this to happen. I'd recommend starting at the failing assertion. I'd recommend setting up your environment so you can edit the code in that module, and setting up a little test program (Python or C++) to provoke the issue.

You don't need permission to work on this. If you want to, go ahead. Every amount of investigation is very welcome. I'm only reporting the issue so me "giving permission" means little.

from opencv.

I just looked into the code and from what I can see, detectMarkers generates detected_corners, a tuple of arrays. The shape of the array is 4x2 (Matrix of 4 rows and 2 cols). However, the function CharucoBoardImpl::matchImagePoints expects the Matrix of 1x2.

Possible fixes:

• Change the Python code to generate a new compatible detected_corners object

temp = []
for i in range(4):
    row = np.array([], dtype=np.float32)
    for c in detected_corners:
        assert len(c[0]) == 4
        row = np.append(row, c[0][i])
    row = row.reshape(24, 1, 2)
    temp.append(tuple(row))

print(f"{len(detected_ids)} detected")

for i in range(len(temp)):
    (obj_points, img_points) = board.matchImagePoints(
        detectedCorners=temp[i],
        detectedIds=detected_ids)
    print(f"obj_points: {obj_points}")

• Fix it in the code. Add a loop to make it generic to support the Matrix of n x 2

from opencv.

I wrote the Python code emulating a C++ example in the docs (think I linked to it earlier). I did not try to run the C++ example.

Either I did the language translation wrong, or that example itself is broken.

The input to one of these functions is a list of arucos and their four corners each.

I do not think that this function merely expects a reshaping of that. I think that code wants something else, which is the checkerboard corners, not the marker corners.

That is why I opened this issue.

If it were as simple as reshaping, which I tried (and that too failed with an assertion), even that would have to be documented and/or fixed.

Maybe this is something that can only be handled by @AleksandrPanov because he's the most knowledgeable w.r.t. the aruco module.

from opencv.

I wrote the Python code emulating a C++ example in the docs (think I linked to it earlier). I did not try to run the C++ example.

Either I did the language translation wrong, or that example itself is broken.

The input to one of these functions is a list of arucos and their four corners each.

I do not think that this function merely expects a reshaping of that. I think that code wants something else, which is the checkerboard corners, not the marker corners.

That is why I opened this issue.

If it were as simple as reshaping, which I tried (and that too failed with an assertion), even that would have to be documented and/or fixed.

Maybe this is something that can only be handled by @AleksandrPanov because he's the most knowledgeable w.r.t. the aruco module.

From the API's point of view, I can tell it expects a corner but not all four altogether. I tested this and I'm getting the same object point for all three corners. Maybe there is a step in between detecting the markers and getting the image point.

I also saw the drawDetectedMarkers and it picks the first point from all 4 corners.

Note: I'll test C++ example code and come back to you again.

from opencv.

I looked into examples relating to matchImagePoints and detectMarkers, and found something interesting.

The (only) class that is handling all four detected markers is GridBoard; however, CharucoBoard is putting a restriction on the number of markers to 1 per point.

Example 1 (GridBoard) => detect_board.cpp:

aruco::GridBoard board(Size(markersX, markersY), markerLength, markerSeparation, dictionary);
...
detector.detectMarkers(image, corners, ids, rejected);
...
board.matchImagePoints(corners, ids, objPoints, imgPoints);

Example 2 => aruco_detect_board_charuco.py

board = cv.aruco.CharucoBoard(board_size, square_len, marker_len, aruco_dict)
charuco_detector = cv.aruco.CharucoDetector(board) // This is different in your example
charuco_corners, charuco_ids, marker_corners, marker_ids = charuco_detector.detectBoard(image) // here they are using detectBoard rather than detectMarkers
...
obj_points, img_points = board.matchImagePoints(charuco_corners, charuco_ids)

In this example, detectBoard does both detectMarkers and interpolateCornersCharucoApproxCalib/interpolateCornersCharucoLocalHom because of this your markers are converted to a matrix of 1 x 2.

from opencv.