GLTFRealityKitLoader bad material rendering on some GLB Files about gltfkit2 HOT 19 CLOSED

I can but GLB doesn't compress well and these files are bigger than the GitHub max attachment size

Google Drive link work?
https://drive.google.com/file/d/1MelF1Qo_kE83h4Ym_cYbWxqMWOzNSZYI/view?usp=share_link

from gltfkit2.

Thanks so much for the support here, @warrenm!

If you're interested, we'd be happy to give you free access to Polycam Pro. If you create a Polycam account, send me the email address you used to create it, and I'll upgrade it. My email is my first name "elliott" at polycam.ai

from gltfkit2.

Thanks for the report. This should be enough to start looking into the issue.

from gltfkit2.

This was due to a misunderstanding of how RealityKit maps the materials list in a ModelComponent to the corresponding MeshResource primitives. It should be fixed as of 95d6aaf.

If you're able to test this fix without me needing to spin a new release, I'd appreciate any feedback. Otherwise this fix will be incorporated into an upcoming release. Thanks again for the report.

from gltfkit2.

This was due to a misunderstanding of how RealityKit maps the materials list in a ModelComponent to the corresponding MeshResource primitives. It should be fixed as of 95d6aaf.

If you're able to test this fix without me needing to spin a new release, I'd appreciate any feedback. Otherwise this fix will be incorporated into an upcoming release. Thanks again for the report.

Thanks for taking a look Warren! We'll test it out and revert back.

from gltfkit2.

Thanks for the rapid turnaround

Still seeing issues with 95d6aaf

from gltfkit2.

Can you share a .glb that reproduces this issue? I suspect it's a separate bug.

from gltfkit2.

The immediate issue here is that we're trying to extract a color channel from a monochrome image as if it's an RGBA image. That's fairly easy to fix, but in the process of fixing it, I've encountered a lot of flakiness in the runtime behavior, and the perennial flakiness of Swift+lldb is making it hard to debug. I'll continue looking at this.

from gltfkit2.

The situation should be much improved with c5501dd. In addition to never attempting to extract a channel from a single-channel image, I switched to a more efficient method from the Accelerate framework to do the extraction when necessary. I also added some defensive image decoding code that only runs on visionOS to address some problems I encountered there.

You're likely to hit validation errors when running with the Metal API validation layer due to the use of 8K images for all material properties, but you can address that when and how you see fit.

from gltfkit2.

Tested with c5501dd

Things do seem improved, there's intermittency on some of the failures (not sure if that's this library or RealityKit/visionOS Simulator)

However some of the models (the jet engine) remain broken with that "zebra" white/gray material failure mode.

Here's is another model that consistently zebras: https://drive.google.com/file/d/1cQDxuaT44CjQHENjNUlJSmaspaRd8Srp/view?usp=share_link

Are you able to reproduce that issue with that model?

• Xcode 15.2
• visionOS Simulator

(edit:) my testing environment:

1. Out-of-the-box new visionOS App via Xcode
2. GLTFKit2.0 installed via SMP pointed to master
3. GLB loading done via the local filesystem
4. Resulting ModelEntity placed via RealityView in the default ImmersiveSpace.

Nothing too crazy here, I can provide the project if that helps with reproduction

from gltfkit2.

Be happy to take a look at a sample project if you can provide it. The table model seems to consistently render without obvious bugs for me with Xcode 15.2 + visionOS Simulator 1.0:

I do see one failure on the console:
callDecodeImage:2006: *** ERROR: decodeImageImp failed - NULL _blockArray
and the app crashes with the Metal validation layer enabled (due to the large image size), but without validation enabled, it renders as above.

from gltfkit2.

Sample project, very simple modifications to the default project setup from 15.2

Large size due to embedded gib files

Hope I'm just holding it wrong :)

https://drive.google.com/file/d/1gPBJwy8YBSyL7oc6mHnpwCjqufudWp9L/view?usp=share_link

from gltfkit2.

I think if you're consuming the framework as a Swift package (which I don't recommend to anyone, but grudgingly support), it uses the contents of the Package.swift in the checkout to determine how to build. In this case, because the package file points to the latest GitHub release, you're not actually building and linking against the code at top of tree; you're linking against version 0.5.7, which doesn't include the changes made to address this issue.

from gltfkit2.

Here's an XCFramework built from top of tree. It should work as a drop-in replacement for the SPM dependency.

https://drive.google.com/file/d/1SmnapYontNls7Wuwv2lyXmhs648derKV/view?usp=sharing

SHA256: 9caf87301f184abb6a268053f5e89f07d06002acba7a5532ec89bcb6b804b1cb

from gltfkit2.

Oh wow okay that "makes sense" but damn

So my project's package.swift points to GLTFKit master, and so the git history of the local GLTFKit copy shows your recent commits (which is why I thought I was using them)

But because GLTFKit's package.swift on master points to a specific release, it "overrides" my project's package.swift file, and what actually gets installed is the older release copy...

HUH. Good to know.

Okay that's great... I'll be able to test this later today.

Thanks Warren! Much appreciated

from gltfkit2.

Yep, you got it. Sorry for the confusion. I only distribute binaries because SPM still doesn't support mixed-language targets after all these years. Maybe we get that some time this decade.

from gltfkit2.

Perfection, thank you so much @warrenm

from gltfkit2.

Sweet! Thanks for sticking with it.

from gltfkit2.

@warrenm Awesome! Just upgraded your Polycam account - you should have access to Pro indefinitely.

from gltfkit2.