Hi thanks for doing this projects as I've been trying to get my head around Vulkan/Ope

Good idea with going through the history, and very strange that <code class="notransla

Did a clone/build, only got left eye rendering,about janhsimon/openxr-vulkan-example

Comments (17)

janhsimon commented on August 12, 2024 1

I did push the commit now but I didn't realize I had already pushed what I thought could have been the fix, so this newest commit probably won't fix anything for you.

from openxr-vulkan-example.

janhsimon commented on August 12, 2024 1

I can reproduce both the hanging on startup in the latest commit, as well as the eye/mirror view issue on my laptop with a 1080. I will dive into the problem now that I can reproduce it myself.

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

Hi madwax, super weird. I do have access to a WMR headset and will see if I can reproduce the issue. I am on a business trip this week however so I'll get back to you at some point next week.

If you do have some time, I would start by making sure I'm in debug mode, and check the console for any Vulkan or OpenXR validation layer warnings/errors.

from openxr-vulkan-example.

madwax commented on August 12, 2024

I've started to have a look and most of the errors come from xrEndFrame (validate set to break on problem) I'm seeing all these on that call.
VUID-VkImageCreateInfo-pNext-01443(ERROR / SPEC): msgNum: 416909302 - Validation Error: [ VUID-VkImageCreateInfo-pNext-01443 ] Object 0: handle = 0x1cf48626ed0, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x18d987f6 | vkCreateImage: VkImageCreateInfo pNext chain includes VkExternalMemoryImageCreateInfo with handleTypes 16 but pCreateInfo->initialLayout is VK_IMAGE_LAYOUT_PREINITIALIZED. The Vulkan spec states: If the pNext chain includes a VkExternalMemoryImageCreateInfo or VkExternalMemoryImageCreateInfoNV structure whose handleTypes member is not 0, initialLayout must be VK_IMAGE_LAYOUT_UNDEFINED (https://vulkan.lunarg.com/doc/view/1.3.231.1/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-01443)
Objects: 1
[0] 0x1cf48626ed0, type: 3, name: NULL
VUID-VkImageCreateInfo-pNext-01443(ERROR / SPEC): msgNum: 416909302 - Validation Error: [ VUID-VkImageCreateInfo-pNext-01443 ] Object 0: handle = 0x1cf48626ed0, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x18d987f6 | vkCreateImage: VkImageCreateInfo pNext chain includes VkExternalMemoryImageCreateInfo with handleTypes 16 but pCreateInfo->initialLayout is VK_IMAGE_LAYOUT_PREINITIALIZED. The Vulkan spec states: If the pNext chain includes a VkExternalMemoryImageCreateInfo or VkExternalMemoryImageCreateInfoNV structure whose handleTypes member is not 0, initialLayout must be VK_IMAGE_LAYOUT_UNDEFINED (https://vulkan.lunarg.com/doc/view/1.3.231.1/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-01443)
Objects: 1
[0] 0x1cf48626ed0, type: 3, name: NULL
UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout(ERROR / SPEC): msgNum: 1303270965 - Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x1cf4bedc1b0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] command buffer VkCommandBuffer 0x1cf4bedc1b0[] expects VkImage 0xfef35a00000000a0[] (subresource: aspectMask 0x1 array layer 1, mip level 0) to be in layout VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.
Objects: 1
[0] 0x1cf4bedc1b0, type: 6, name: NULL
UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout(ERROR / SPEC): msgNum: 1303270965 - Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x1cf4c0cc700, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] command buffer VkCommandBuffer 0x1cf4c0cc700[] expects VkImage 0xf9ddb900000000a3[] (subresource: aspectMask 0x1 array layer 1, mip level 0) to be in layout VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.
Objects: 1
[0] 0x1cf4c0cc700, type: 6, name: NULL
UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout(ERROR / SPEC): msgNum: 1303270965 - Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x1cf4bedc1b0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] command buffer VkCommandBuffer 0x1cf4bedc1b0[] expects VkImage 0x54fef00000000a6[] (subresource: aspectMask 0x1 array layer 1, mip level 0) to be in layout VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.
Objects: 1
[0] 0x1cf4bedc1b0, type: 6, name: NULL
UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout(ERROR / SPEC): msgNum: 1303270965 - Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x1cf4c0cc700, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] command buffer VkCommandBuffer 0x1cf4c0cc700[] expects VkImage 0xfef35a00000000a0[] (subresource: aspectMask 0x1 array layer 1, mip level 0) to be in layout VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.
Objects: 1

Will keep looking/learning :)

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

Interesting, the first two errors about VkImageCreateInfo probably originate from within the SteamVR OpenXR implemententation, I have seen those before. It should be safe to disregard these.

The rest, about the image layout, could be the cause of your issue. I will try to reproduce it with a Windows Mixed Reality headset this week. But here are some general thoughts:

The Headset class holds a render pass that transitions the color attachment of the eye image from VK_IMAGE_LAYOUT_UNDEFINED (ie. anything) to VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL when rendering into it. The mirror view then takes that, converts it from VK_IMAGE_LAYOUT_UNDEFINED (ie. anything) to VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, and reads from it in the blit to the mirror view.

Your errors occur in the call to vkQueueSubmit(), which is called in void Renderer::submit(), where it takes the command buffer it recorded with the aforementioned render pass. Even though the render pass expects its input to be in VK_IMAGE_LAYOUT_UNDEFINED, it looks to give this error. Presumably this only happens in frame 2 and later, since we haven't done the mirror view part yet when vkQueueSubmit() is called in the first frame.

I suggest confirming that the first from goes through without the error. If it is the second frame, then it's a pretty strong indication that the mirror view blitting does indeed cause the issue when drawing into the headset again.

One important note is that you're unfortunately not guaranteed to receive errors at the right time with Vulkan, since the CPU and GPU are not always in sync. So if your breakpoint is hit in xrEndFrame(), then that probably just means that the OpenXR implementation is submitting our work at that point, or it has a sync point in there, causing the breakpoint to show up there.

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

I just tried it out with a Lenovo Explorer WMR Headset and it is working fine on my end, with images in both eyes and the mirror view. What headset are you using?

I am using the latest Windows Mixed Reality portal (according to Windows), SteamVR 1.24.6, and the latest SteamVR for Windows Mixed Reality plugin 1.3.64 (according to Steam). And of course I have SteamVR set as the OpenXR runtime through the Menu icon in the top-left corner of the SteamVR app -> Settings -> Advanced Settings: Show -> Developer -> Set SteamVR as OpenXR Runtime.

I do see a lot of similar warnings and errors that you are getting, here are the first handful of messages:


[Vulkan] Validation Error: [ VUID-VkImageCreateInfo-pNext-01443 ] Object 0: handle = 0x29ade0a5300, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x18d987f6 | vkCreateImage: VkImageCreateInfo pNext chain includes VkExternalMemoryImageCreateInfo with handleTypes 16 but pCreateInfo->initialLayout is VK_IMAGE_LAYOUT_PREINITIALIZED. The Vulkan spec states: If the pNext chain includes a VkExternalMemoryImageCreateInfo or VkExternalMemoryImageCreateInfoNV structure whose handleTypes member is not 0, initialLayout must be VK_IMAGE_LAYOUT_UNDEFINED (https://vulkan.lunarg.com/doc/view/1.3.224.1/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-01443)

[Vulkan] Validation Error: [ VUID-VkImageCreateInfo-pNext-01443 ] Object 0: handle = 0x29ade0a5300, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x18d987f6 | vkCreateImage: VkImageCreateInfo pNext chain includes VkExternalMemoryImageCreateInfo with handleTypes 16 but pCreateInfo->initialLayout is VK_IMAGE_LAYOUT_PREINITIALIZED. The Vulkan spec states: If the pNext chain includes a VkExternalMemoryImageCreateInfo or VkExternalMemoryImageCreateInfoNV structure whose handleTypes member is not 0, initialLayout must be VK_IMAGE_LAYOUT_UNDEFINED (https://vulkan.lunarg.com/doc/view/1.3.224.1/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-01443)

[Vulkan] Validation Error: [ VUID-VkImageCreateInfo-pNext-01443 ] Object 0: handle = 0x29ade0a5300, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x18d987f6 | vkCreateImage: VkImageCreateInfo pNext chain includes VkExternalMemoryImageCreateInfo with handleTypes 16 but pCreateInfo->initialLayout is VK_IMAGE_LAYOUT_PREINITIALIZED. The Vulkan spec states: If the pNext chain includes a VkExternalMemoryImageCreateInfo or VkExternalMemoryImageCreateInfoNV structure whose handleTypes member is not 0, initialLayout must be VK_IMAGE_LAYOUT_UNDEFINED (https://vulkan.lunarg.com/doc/view/1.3.224.1/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-01443)

[Vulkan] Validation Performance Warning: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x29af1671310, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkCmdBlitImage: For optimal performance VkImage 0xa56ac00000000d4[] layout should be VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL instead of GENERAL.

[Vulkan] Validation Error: [ VUID-VkImageCreateInfo-pNext-01443 ] Object 0: handle = 0x29ade0a5300, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x18d987f6 | vkCreateImage: VkImageCreateInfo pNext chain includes VkExternalMemoryImageCreateInfo with handleTypes 16 but pCreateInfo->initialLayout is VK_IMAGE_LAYOUT_PREINITIALIZED. The Vulkan spec states: If the pNext chain includes a VkExternalMemoryImageCreateInfo or VkExternalMemoryImageCreateInfoNV structure whose handleTypes member is not 0, initialLayout must be VK_IMAGE_LAYOUT_UNDEFINED (https://vulkan.lunarg.com/doc/view/1.3.224.1/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-01443)

[Vulkan] Validation Error: [ VUID-VkImageCreateInfo-pNext-01443 ] Object 0: handle = 0x29ade0a5300, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x18d987f6 | vkCreateImage: VkImageCreateInfo pNext chain includes VkExternalMemoryImageCreateInfo with handleTypes 16 but pCreateInfo->initialLayout is VK_IMAGE_LAYOUT_PREINITIALIZED. The Vulkan spec states: If the pNext chain includes a VkExternalMemoryImageCreateInfo or VkExternalMemoryImageCreateInfoNV structure whose handleTypes member is not 0, initialLayout must be VK_IMAGE_LAYOUT_UNDEFINED (https://vulkan.lunarg.com/doc/view/1.3.224.1/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-01443)

[Vulkan] Validation Error: [ VUID-VkImageCreateInfo-pNext-01443 ] Object 0: handle = 0x29ade0a5300, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0x18d987f6 | vkCreateImage: VkImageCreateInfo pNext chain includes VkExternalMemoryImageCreateInfo with handleTypes 16 but pCreateInfo->initialLayout is VK_IMAGE_LAYOUT_PREINITIALIZED. The Vulkan spec states: If the pNext chain includes a VkExternalMemoryImageCreateInfo or VkExternalMemoryImageCreateInfoNV structure whose handleTypes member is not 0, initialLayout must be VK_IMAGE_LAYOUT_UNDEFINED (https://vulkan.lunarg.com/doc/view/1.3.224.1/windows/1.3-extensions/vkspec.html#VUID-VkImageCreateInfo-pNext-01443)

[Vulkan] Validation Performance Warning: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f1e35fe0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkCmdBlitImage: For optimal performance VkImage 0x5fc58300000000db[] layout should be VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL instead of GENERAL.

[Vulkan] Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f1e35fe0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] command buffer VkCommandBuffer 0x2d7f1e35fe0[] expects VkImage 0xe2478b00000000a1[] (subresource: aspectMask 0x1 array layer 1, mip level 0) to be in layout VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.

[Vulkan] Validation Performance Warning: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f1e35fe0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkCmdBlitImage: For optimal performance VkImage 0xa56ac00000000d4[] layout should be VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL instead of GENERAL.

[Vulkan] Validation Performance Warning: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f152caf0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkCmdBlitImage: For optimal performance VkImage 0x5fc58300000000db[] layout should be VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL instead of GENERAL.

[Vulkan] Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f152caf0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] command buffer VkCommandBuffer 0x2d7f152caf0[] expects VkImage 0xe4607e00000000a4[] (subresource: aspectMask 0x1 array layer 1, mip level 0) to be in layout VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.

[Vulkan] Validation Performance Warning: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f152caf0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkCmdBlitImage: For optimal performance VkImage 0xa56ac00000000d4[] layout should be VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL instead of GENERAL.

[Vulkan] Validation Performance Warning: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f1e35fe0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkCmdBlitImage: For optimal performance VkImage 0x5fc58300000000db[] layout should be VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL instead of GENERAL.

[Vulkan] Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f1e35fe0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] command buffer VkCommandBuffer 0x2d7f1e35fe0[] expects VkImage 0xea035600000000a7[] (subresource: aspectMask 0x1 array layer 1, mip level 0) to be in layout VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.

[Vulkan] Validation Performance Warning: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f1e35fe0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkCmdBlitImage: For optimal performance VkImage 0xa56ac00000000d4[] layout should be VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL instead of GENERAL.

[Vulkan] Validation Performance Warning: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f152caf0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkCmdBlitImage: For optimal performance VkImage 0x5fc58300000000db[] layout should be VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL instead of GENERAL.

[Vulkan] Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x2d7f152caf0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x4dae5635 | vkQueueSubmit(): pSubmits[0].pCommandBuffers[0] command buffer VkCommandBuffer 0x2d7f152caf0[] expects VkImage 0xe2478b00000000a1[] (subresource: aspectMask 0x1 array layer 1, mip level 0) to be in layout VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL--instead, current layout is VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL.

from openxr-vulkan-example.

madwax commented on August 12, 2024

I'm on the same versions and Streams OpenXR is active not Windows. I've got 2 HP Reverb G1 (on different Windows boxes) I could try digging out my CV1 and see what that does.

Yer it's looking like similar messages :( I'll have time later today so I'll do some digging.

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

Sounds good, I do also have an HP Reverb G2 that I can test on later.

I am working on some uniform buffer changes and actually was able to reproduce your problem that way on my main device, a Quest 2, so maybe my next commit will fix it for you as well. Hopefully in a couple hours... :)

from openxr-vulkan-example.

madwax commented on August 12, 2024

Just gone through git history doing builds and found something interesting. Everything works up to 3a54316 when you added multiview. The interesting thing is I see the right eye not the left. The next checking the eye swaps

Also the latest version starts but then locks up. Looking at it now

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

Good idea with going through the history, and very strange that multiview is giving you the problem. What GPU are you on? I've been running this on an RTX 3080 and 3090. I assume the check for physicalDeviceMultiviewFeatures.multiview support in the Context class passes fine on your end?

I am also wondering if one eye not rendering has to do with the eye that is getting blitted to the mirror view? Can you try changing it from the right to the left eye and see if that makes a difference? The eye to mirror is defined as mirrorEyeIndex at the top of MirrorView.cpp.

from openxr-vulkan-example.

madwax commented on August 12, 2024

Well I disabled the MirrorView rendering (returning MirrorView::RenderResult::Invisible; at the start of render() and I get both eyes. Should have done that first lol. Also switching mirrorEyeIndex changed which eye was displayed in the headset

I had to fix the alignment of the uniform buffers as well (pushing my changes up to a fork) and are you seeing lots of warmings about image CommandBuffers and images being destroyed when the app terminates?

1080ti's in both machines

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

Okay, so then it is definitely the mirror view rendering that is causing whatever eye image it is blitting from to get 'invalidated' somehow. I do have a laptop with a 1080 that I will try running this on as well to see if it's a GPU-related issue maybe.

Interesting with the uniform buffer alignment. I thought it was fine to not align 3x mat4s and 1x float in that order, but I guess that's wrong on some systems. I will definitely fix that in my repo as well, good catch!

I do not get any warnings or errors whatsoever during the entire program lifecycle on my end. I would assume that these are once again coming from the SteamVR OpenXR implementation.

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

Good news! I think I finally found the issue. Please do me a huge favor and confirm that this fixes the problem for you! It had to do with providing some more accurate layout, access mask and pipeline stage descriptions when blitting from the headset eye to the mirror view.

Try to replace the code block in MirrorView::render() from line 151 to 247 with:

  // Convert the source image layout from undefined to transfer source
  VkImageMemoryBarrier imageMemoryBarrier{ VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER };
  imageMemoryBarrier.image = sourceImage;
  imageMemoryBarrier.oldLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
  imageMemoryBarrier.newLayout = VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL;
  imageMemoryBarrier.srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
  imageMemoryBarrier.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT;
  imageMemoryBarrier.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
  imageMemoryBarrier.subresourceRange.layerCount = 1u;
  imageMemoryBarrier.subresourceRange.baseArrayLayer = mirrorEyeIndex;
  imageMemoryBarrier.subresourceRange.levelCount = 1u;
  imageMemoryBarrier.subresourceRange.baseMipLevel = 0u;
  vkCmdPipelineBarrier(commandBuffer, VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT,
                       VK_DEPENDENCY_BY_REGION_BIT, 0u, nullptr, 0u, nullptr, 1u, &imageMemoryBarrier);

  // Convert the destination image layout from undefined to transfer destination
  imageMemoryBarrier.image = destinationImage;
  imageMemoryBarrier.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED;
  imageMemoryBarrier.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL;
  imageMemoryBarrier.srcAccessMask = 0u;
  imageMemoryBarrier.dstAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
  imageMemoryBarrier.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
  imageMemoryBarrier.subresourceRange.layerCount = 1u;
  imageMemoryBarrier.subresourceRange.baseArrayLayer = 0u;
  imageMemoryBarrier.subresourceRange.levelCount = 1u;
  imageMemoryBarrier.subresourceRange.baseMipLevel = 0u;
  vkCmdPipelineBarrier(commandBuffer, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT,
                       VK_DEPENDENCY_BY_REGION_BIT, 0u, nullptr, 0u, nullptr, 1u, &imageMemoryBarrier);

  // We need to crop the source image region to preserve the aspect ratio of the mirror view window
  const glm::vec2 sourceResolution = { static_cast<float>(eyeResolution.width),
                                       static_cast<float>(eyeResolution.height) };
  const float sourceAspectRatio = sourceResolution.x / sourceResolution.y;
  const glm::vec2 destinationResolution = { static_cast<float>(swapchainResolution.width),
                                            static_cast<float>(swapchainResolution.height) };
  const float destinationAspectRatio = destinationResolution.x / destinationResolution.y;
  glm::vec2 cropResolution = sourceResolution, cropOffset = { 0.0f, 0.0f };

  if (sourceAspectRatio < destinationAspectRatio)
  {
    cropResolution.y = sourceResolution.x / destinationAspectRatio;
    cropOffset.y = (sourceResolution.y - cropResolution.y) / 2.0f;
  }
  else if (sourceAspectRatio > destinationAspectRatio)
  {
    cropResolution.x = sourceResolution.y * destinationAspectRatio;
    cropOffset.x = (sourceResolution.x - cropResolution.x) / 2.0f;
  }

  // Blit the source to the destination image
  VkImageBlit imageBlit{};
  imageBlit.srcOffsets[0] = { static_cast<int32_t>(cropOffset.x), static_cast<int32_t>(cropOffset.y), 0 };
  imageBlit.srcOffsets[1] = { static_cast<int32_t>(cropOffset.x + cropResolution.x),
                              static_cast<int32_t>(cropOffset.y + cropResolution.y), 1 };
  imageBlit.srcSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
  imageBlit.srcSubresource.mipLevel = 0u;
  imageBlit.srcSubresource.baseArrayLayer = mirrorEyeIndex;
  imageBlit.srcSubresource.layerCount = 1u;

  imageBlit.dstOffsets[0] = { 0, 0, 0 };
  imageBlit.dstOffsets[1] = { static_cast<int32_t>(destinationResolution.x),
                              static_cast<int32_t>(destinationResolution.y), 1 };
  imageBlit.dstSubresource.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
  imageBlit.dstSubresource.layerCount = 1u;
  imageBlit.dstSubresource.baseArrayLayer = 0u;
  imageBlit.dstSubresource.mipLevel = 0u;

  vkCmdBlitImage(commandBuffer, sourceImage, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, destinationImage,
                 VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1u, &imageBlit, VK_FILTER_NEAREST);

  // Convert the source image layout from transfer source to color attachment
  imageMemoryBarrier.image = sourceImage;
  imageMemoryBarrier.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL;
  imageMemoryBarrier.newLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
  imageMemoryBarrier.srcAccessMask = VK_ACCESS_TRANSFER_READ_BIT;
  imageMemoryBarrier.dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
  imageMemoryBarrier.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
  imageMemoryBarrier.subresourceRange.layerCount = 1u;
  imageMemoryBarrier.subresourceRange.baseArrayLayer = mirrorEyeIndex;
  imageMemoryBarrier.subresourceRange.levelCount = 1u;
  imageMemoryBarrier.subresourceRange.baseMipLevel = 0u;
  vkCmdPipelineBarrier(commandBuffer, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
                       VK_DEPENDENCY_BY_REGION_BIT, 0u, nullptr, 0u, nullptr, 1u, &imageMemoryBarrier);

  // Convert the destination image layout from transfer destination to present
  imageMemoryBarrier.image = destinationImage;
  imageMemoryBarrier.oldLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL;
  imageMemoryBarrier.newLayout = VK_IMAGE_LAYOUT_PRESENT_SRC_KHR;
  imageMemoryBarrier.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT;
  imageMemoryBarrier.dstAccessMask = 0u;
  imageMemoryBarrier.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
  imageMemoryBarrier.subresourceRange.layerCount = 1u;
  imageMemoryBarrier.subresourceRange.baseArrayLayer = 0u;
  imageMemoryBarrier.subresourceRange.levelCount = 1u;
  imageMemoryBarrier.subresourceRange.baseMipLevel = 0u;
  vkCmdPipelineBarrier(commandBuffer, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT,
                       VK_DEPENDENCY_BY_REGION_BIT, 0u, nullptr, 0u, nullptr, 1u, &imageMemoryBarrier);

Either try this on the experimental branch in your fork, where you already fixed the other, unrelated uniform buffer alignment issue, or make the change in this repo in the main branch, but then you will have to fix the uniform buffer alignment that will otherwise cause a fatal error. The easiest way to do so in my repo is to temporarily change the UniformBufferData struct in RenderProcess.h to include an extra glm::mat4 for padding:

  struct UniformBufferData final
  {
    glm::mat4 world;
    glm::mat4 viewProjection[2]; // View projection matrices, 0 = left eye, 1 = right eye
    glm::mat4 padding;           // Pad to 256 bytes
    float time;                  // For animation
  } uniformBufferData;

This ensures that the float time variable is aligned to 256 bytes, which is required on my GTX 1080. I am assuming that this is the same requirement on your 1080 Ti. You can print out the minUniformBufferOffsetAlignment limit of the vkPhysicalDevice if you want to verify this.

I am not currently sure about the freeze at startup when using Windows Mixed Reality - maybe it's caused by the uniform buffer alignment as well? I will test this today.

As a sidenote: I was able to reproduce it on my GTX 1080 laptop even on my Quest 2, where the Oculus OpenXR runtime shows no warnings or errors, meaning that all the console output you are getting is most likely from the naughty SteamVR OpenXR runtime.

from openxr-vulkan-example.

madwax commented on August 12, 2024

@janhsimon That fixed it. Both eyes and mirror working fine :) Super and big thanks for finding the fix.

Good to know that the errors are coming from StreamVR

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

Awesome, thanks for sticking around for so long and helping with troubleshooting this issue. I will add a new commit with the above fix, and then clean up the uniform buffer problem as well.

from openxr-vulkan-example.

madwax commented on August 12, 2024

No problem, I keep getting pulled away onto other things so.

from openxr-vulkan-example.

janhsimon commented on August 12, 2024

I have just pushed the commits that fix the problems you found (5dc0f53 and 13a6810). I will close this issue since you were able to confirm the fixes on your end.

from openxr-vulkan-example.

Did a clone/build, only got left eye rendering about openxr-vulkan-example HOT 17 CLOSED

Comments (17)

Related Issues (5)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent