Framework vulkan RenderGraph misses layout transition from attachment to present_khr if no renderpass callback is registered about rafx HOT 2 CLOSED

#242 should address all three issues I mentioned in my previous comment

from rafx.

I'm going to look at and think about this more, but what's happening is that we have two kinds of render graph nodes - "render" and "callback". Callback is usually compute operations but there's no reason one couldn't do other GPU operations, so I use the more general naming of a "callback" node because it's literally just giving you a callback. The way the render graph code knows which type of node you want is based on if you called set_renderpass_callback or set_callback. If you don't call either, it defaults to just a callback (i.e. no renderpass is created.)

At a code level, we're checking image states as we progress through nodes to determine where barriers need to be placed. Most of the time, we visit a node and look at all the transitions needed to do the work before the node performs work. The exception is output images - their transition is after all nodes are visited. So that case is a different (older) codepath. Right now it only handles image attachments, which should only ever be placed on renderpasses. So the code is reasonably skipped for non-renderpasses.

So some things to resolve:

• A node with an image attachment only makes sense as a renderpass, but the API design where no callback set = no renderpass unintuitively opts out of creating the renderpass. So may need to rethink this. I don't think we should implicitly do a renderpass based on there being an attachment or not. Other options:
  • Change add_node to be add_renderpass_node and add_callback_node
  • Change add_node to take an enum that specifies what kind of node to create
• Should probably have better error messaging if an image attachment is used on a non-renderpass node as it's always wrong.
• Handling barriers for output images/buffers seems broken for anything but attachment images. The current form of this code is relatively unchanged since the first implementation of the render graph which was extremely limited, basically only supporting renderpasses. So the end of build_pass_barriers needs a proper implementation, and it probably needs to be done as a final pass on the graph as a whole rather than tacked onto the end of whatever node last touched the resource.

A potential bandaid would be to fail loudly if no callback is provided, and probably also fail loudly if a color attachment is used with a node that's not a renderpass. This would make it more obvious that the graph in this repro is not legal (because under current API design it describes using a color attachment without a renderpass.)

from rafx.