philipturner / arheadsetkit Goto Github PK

I am working on an API to export the raw data from ARHeadsetKit. It takes all geometry and color data present at one point in time, then reorders it into a simple serialization format. This is entirely lossless and should happen very quickly.

• To work with this format, you need a programming language with low-level access to memory layout. Python is not ideal; rather, use Swift or C/C++ compiled in release mode.

The next step is converting the YCbCr data to RGB (if you wish) or rearranging the geometry data into a different layout.

• Initially, you should convert the luma and chroma into PNG images, before converting to RGB. They are valid color spaces that can be viewed as images.
  • This can be done for a single triangle (a very tiny image).
• Next, read this: https://developer.apple.com/documentation/accelerate/conversion/understanding_ypcbcr_image_formats
• Apple has utilities in vImage for converting YCbCr -> RGB on the CPU: https://developer.apple.com/documentation/accelerate/1533189-vimageconvert_ypcbcrtoargb_gener
  • Concatenate all the chroma tiles into a massive array, do the same with luma, and send that through vImage.

cc @knightfork

Is it possible to import a custom 3D model into the scene?

Hi Philip, thank you so much for creating a great package/framework! I have been trying to make it work as a framework but it fails with 'Command ExtractAPI failed with a nonzero exit code' and it fails to find metal_stdlib file in the framework - "fatal error: 'metal_stdlib' file not found". I am working with iOS SDK 16.2. Not sure if that's something to do with it. I am new to metal (and to AR) so not quite sure how to fix this issue. I am quite sure I followed the tutorial for 'Reconfiguring ARHeadsetKit' to the T. I tried setting the header paths in the settings but to no avail. What might I be doing wrong?