Developement has moved inside the wgpu respository! Despite it being moved, it is still being developed as a stand alone library.
This respository is now archived.
Universal shader translation in Rust
Home Page: https://github.com/gfx-rs/wgpu/tree/trunk/naga
License: Other
Developement has moved inside the wgpu respository! Despite it being moved, it is still being developed as a stand alone library.
This respository is now archived.
Currently, we have both the image and the sampler specified as expressions:
ImageSample {
image: Handle<Expression>,
sampler: Handle<Expression>,
coordinate: Handle<Expression>,
level: SampleLevel,
depth_ref: Option<Handle<Expression>>,
},
WGSL today requires them to be statically known, so maybe it would make more sense to have this expression as:
ImageSample {
image: Handle<GlobalVariable>,
sampler: Handle<GlobalVariable>,
coordinate: Handle<Expression>,
level: SampleLevel,
depth_ref: Option<Handle<Expression>>,
},
The main concern is: whether or not we'll want to support the variable pointers at some point. In this case, the exact image or sampler will not be statically known to IR.
Given the following piece of wgsl code:
var index : u32 = gl_GlobalInvocationID.x;
if (index >= 5) {
return;
}
The wgsl frontend will generate ConstantInner::Sint(5)
instead of ConstantInner::Uint(5)
this causes issues in backends that don't have implicit conversions, for example the glsl es backend will generate
uint index = gl_GlobalInvocationID[0];
if((index >= 5)) {
return;
}
which is wrong and glslangValidator
complains about the correct code would be
uint index = gl_GlobalInvocationID[0];
if((index >= 5u)) {
return;
}
hence why generating the right variant is important
Recently the Expression::Call
has been implemented for the WGSL front-end. Using this at the right handed side of an expression will add it to the statement body in the IR.
WGSL:
fn test_function(test: f32) -> f32 {
return test;
}
fn main_vert() -> void {
var test: f32 = test_function(1.0);
return;
}
entry_point vertex as "main" = main_vert;
IR:
expressions: Arena {
data: [
Constant(
Handle(1),
),
Call {
origin: Local(
Handle(1),
),
arguments: [
Handle(1),
],
},
LocalVariable(
Handle(1),
),
],
},
body: [
Store {
pointer: Handle(3),
value: Handle(2),
},
Return {
value: None,
},
],
When not calling the function at the right handed side of an expression, but rather just call it in the body of a function, this will not be added to the IR.
WGSL:
fn test_function(test: f32) -> f32 {
return test;
}
fn main_vert() -> void {
test_function(1.0);
return;
}
entry_point vertex as "main" = main_vert;
IR:
expressions: Arena {
data: [
Constant(
Handle(1),
),
Call {
origin: Local(
Handle(1),
),
arguments: [
Handle(1),
],
},
],
},
body: [
Empty,
Return {
value: None,
},
],
The most probable reason that this happens is because the IR technically thinks it is always an expression, and cannot be a statement, hence there is only Expression::Call
.
What I propose to do is adding a Statement::Call
, and to detect if it is an expression or an statement will be done based on if the function is called at the right side of an expression (e.g. variable declaration), or not.
Current parser API requires the execution mode to be specified for the whole module:
naga::front::glsl::parse_str(&input, "main".to_string(), spirv::ExecutionModel::Vertex)
This isn't ideal, and this doesn't match other front-ends. Ideally, a GLSL file could be loaded containing multiple different entry points.
Here is what I suggest we do instead:
struct Options<'a> {
entry_points: &'a FastHashMap<String, spirv::ExecutionModel>,
}
fn parse_str(source: &str, options: Options) -> crate::Module {..}
So basically we could externally specify the entry points, and support any number of them in a single GLSL file. Having the "external" data is required in some of the other end-points too: Metal backend needs to have the mapping of resources specified, for example.
Currently it's used in the frontends only, and it mutates the type arena.
GLSL and MSL backends would benefit from having this automated too, but we need to do something about the mutable arena.
Since SPIR-V requires float
to be declared before vec4<float>
, for example, it can't possibly express a case where vec4
comes first in the source (e.g. WGSL).
What can we do?
Minimal reproduction: feed the WGSL parser the following string:
"\"\u{2}ПЀ\u{0}\""
Leads to the following panic in wgsl::lex::consume_token
:
thread '<unnamed>' panicked at 'byte index 4 is not a char boundary; it is inside 'Ѐ' (bytes 3..5) of `ПЀ"`', /home/gabriel/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/str/mod.rs:1987:47
stack backtrace:
0: backtrace::backtrace::libunwind::trace
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
1: backtrace::backtrace::trace_unsynchronized
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
2: std::sys_common::backtrace::_print_fmt
at src/libstd/sys_common/backtrace.rs:78
3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
at src/libstd/sys_common/backtrace.rs:59
4: core::fmt::write
at src/libcore/fmt/mod.rs:1076
5: std::io::Write::write_fmt
at src/libstd/io/mod.rs:1537
6: std::sys_common::backtrace::_print
at src/libstd/sys_common/backtrace.rs:62
7: std::sys_common::backtrace::print
at src/libstd/sys_common/backtrace.rs:49
8: std::panicking::default_hook::{{closure}}
at src/libstd/panicking.rs:198
9: std::panicking::default_hook
at src/libstd/panicking.rs:217
10: libfuzzer_sys::initialize::{{closure}}
11: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:524
12: rust_begin_unwind
at src/libstd/panicking.rs:431
13: core::panicking::panic_fmt
at src/libcore/panicking.rs:85
14: core::str::slice_error_fail
at src/libcore/str/mod.rs:0
15: core::str::traits::<impl core::slice::SliceIndex<str> for core::ops::range::RangeTo<usize>>::index::{{closure}}
16: naga::front::wgsl::lex::consume_token
17: naga::front::wgsl::Parser::parse_global_decl
18: naga::front::wgsl::Parser::parse
19: rust_fuzzer_test_input
20: __rust_try
21: LLVMFuzzerTestOneInput
22: _ZN6fuzzer6Fuzzer15ExecuteCallbackEPKhm
23: _ZN6fuzzer10RunOneTestEPNS_6FuzzerEPKcm
24: _ZN6fuzzer12FuzzerDriverEPiPPPcPFiPKhmE
25: main
26: __libc_start_main
27: _start
The panic is likely from this indexing here:
Line 101 in 1f5008f
Travis signalling stopped working. Let's move to Github Actions?
This is a list of generic shader transformations that we need to do for WebGPU:
It's not an opcode in SPIR-V, it's an extension method.
Arguably, we can even move the Dot
variant as well, but that should be another issue.
In WebGPU API, the comparison sampler is now a binding type. We want to validate it against the shader, so it would make sense if the IR here had a separation between comparison and non-comparison samplers.
The TypeInner
variant shall be Sampler { comparison: bool }
If our IR is similar to Tint AST, we'll have structured control flow there. That means, when parsing SPIR-V, we'll have to convert unstructured control flow into structured.
Relevant links:
SPIR-V lists all the inputs and outputs for each entry point. The question is whether it's a good idea to represent this somehow in the IR. See gpuweb/gpuweb#592 for WGSL discussion.
Pros for specifying:
Cons for specifying:
Another thought I had is that it doesn't make sense to specify that interface per entry point. It seems like it would make more sense to have for the function.
I'm getting a panic when trying to use spirv-out for the simple Rosetta test:
$ RUST_BACKTRACE=1 cargo run --all-features --example convert test-data/simp
le/simple.wgsl target/out.spv true
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Running `target/debug/examples/convert test-data/simple/simple.wgsl target/out.spv true`
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/back/spv/writer.rs:62:40
stack backtrace:
0: backtrace::backtrace::libunwind::trace
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs
:86
1: backtrace::backtrace::trace_unsynchronized
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
2: std::sys_common::backtrace::_print_fmt
at src/libstd/sys_common/backtrace.rs:78
3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
at src/libstd/sys_common/backtrace.rs:59
4: core::fmt::write
at src/libcore/fmt/mod.rs:1076
5: std::io::Write::write_fmt
at src/libstd/io/mod.rs:1537
6: std::sys_common::backtrace::_print
at src/libstd/sys_common/backtrace.rs:62
7: std::sys_common::backtrace::print
at src/libstd/sys_common/backtrace.rs:49
8: std::panicking::default_hook::{{closure}}
at src/libstd/panicking.rs:198
9: std::panicking::default_hook
at src/libstd/panicking.rs:217
10: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:526
11: rust_begin_unwind
at src/libstd/panicking.rs:437
12: core::panicking::panic_fmt
at src/libcore/panicking.rs:85
13: core::panicking::panic
at src/libcore/panicking.rs:50
14: core::option::Option<T>::unwrap
at /home/code/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/macros/mod.rs:10
15: naga::back::spv::writer::Function::to_words
at src/back/spv/writer.rs:62
16: naga::back::spv::writer::Writer::write_logical_layout
at src/back/spv/writer.rs:1547
17: naga::back::spv::writer::Writer::write
at src/back/spv/writer.rs:1575
18: convert::main
at examples/convert.rs:191
19: std::rt::lang_start::{{closure}}
at /home/code/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:67
20: std::rt::lang_start_internal::{{closure}}
at src/libstd/rt.rs:52
21: std::panicking::try::do_call
at src/libstd/panicking.rs:348
22: std::panicking::try
at src/libstd/panicking.rs:325
23: std::panic::catch_unwind
at src/libstd/panic.rs:394
24: std::rt::lang_start_internal
at src/libstd/rt.rs:51
25: std::rt::lang_start
at /home/code/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:67
26: main
27: __libc_start_main
28: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
We should allow constant expressions for array sizes. This needs IR changes:
enum ConstExpression
that takes a subset of Expression
that's valid for constant propagationExpression::Const(Handle<ConstExpression>)
to transition from regular expressions (instead of the current Expression::Constant
)pub const_expressions: Arena<ConstExpression>
in a functionThe Module
struct that our front-ends produce generally has some level of type-level encoding of the constraints, however many properties can only be checked at run-time. One big area of validation is type checking:
Another area is access checking:
Finally we need to enforce some of the conventions. For example, variables can either be stored as pointers (that are loaded/stored), or directly. There is currently no consistency, and whatever we choose would need to be enforced.
We have a proc module for end-point-independent processing. I think the validator
sub-module could live there. The main task of it would be getting a module and making sure that it's valid. We'd be running it in CI tests to make sure that the module contents produced by the front-ends, or by different processors/transformers, are all valid.
Naga is a pure data processing library. There are two kinds of unexpected here:
Err
without panics!The problem is - unlike with unsafe {}
- Rust doesn't have a way to capture/detect the places we panic, easily. For example, just accessing some slice by index has an implicit panic in it.
It would be very useful to have some mechanism where we'd explicitly mark panic places where we are allowed to panic, and warn/error in other places. Perhaps, something like a compiler plugin could be crafted?
Since #148 cargo test
fails to compile.
$ cargo test
Compiling naga v0.2.0 (git/naga)
error[E0277]: the trait bound `Module: serde::ser::Serialize` is not satisfied
--> src/front/wgsl_rosetta_tests.rs:13:18
|
13 | let output = ron::ser::to_string_pretty(&module, Default::default()).unwrap();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `serde::ser::Serialize` is not implemented for `Module`
|
::: .cargo/registry/src/github.com-1ecc6299db9ec823/ron-0.5.1/src/ser/mod.rs:30:8
|
30 | T: Serialize,
| --------- required by this bound in `ron::ser::to_string_pretty`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0277`.
error: could not compile `naga`.
To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
The problem is that the serialize
feature needs to be enabled. That's also the reason this wasn't caught in CI since --all-features
is enabled there.
cc @pjoe
I faced with the following issue: ParseError { kind: UnknownVariable(TokenMetadata { line: 19, chars: 16..21 }, "Model") }
when tried to parse .vert
file.
I noticed this commented test and now curious - what happened? It seems like it worked before but now - it's not.
How to fix it?
Do we really need a parser? Especially one with compile-time code generation from an external grammar, like Pest (that we are currently using for WGSL). Technically, all we care about is working with tokens. And we also need recursive-descent parsing.
Since we are trying to gracefully handle errors, we could get the recursive descent by just trying to parse things that make sense in a context.
Problems:
unreachable!
. We might as well do it on tokens then.After speaking with @jrmuizel, I'm thinking that we could just write a simple tokenizer. Possibly using the perfect hash tables of phf.
When shader writes into the gl_Depth
variable SPIR-V backend must output ExectutionMode DepthReplacing instruction for SPIR-V bytecode to be valid.
In spir-v any access to variables is done via load/stores in pointers. In WGSL that pointer step is skipped.
I think we should change the SPIR-V front-end to ignore the indirection, and write the IR validation accordingly (#59).
Converting this WGSL:
[[location 0]] var<in> a_pos : vec2<f32>;
[[location 0]] var<out> v_uv : vec2<f32>;
fn main_vert() -> void {
v_uv = a_pos * a_pos;
return;
}
entry_point vertex as "main" = main_vert;
To the following IR module (omitted most things):
types: Arena {
data: [
Type {
name: None,
inner: Vector {
size: Bi,
kind: Float,
width: 32,
},
},
],
},
As you can see, the Float scalar is not created, the SPIR-V back-end has to check if the scalar type already exists, if not, create it.
The reason the quad.wgsl works, is because at the top of the file is the following line declared:
const c_scale: f32 = 1.2;
This causes the front-end to create the Float scalar for the Vectors that need this.
Writing what is, essentially a SPIRV-to-whatever compiler sounds like fun! What are our specific targets for "whatever"?
Currently, we are trying to represent types by value. For nested types, such as pointers/structures/arrays, we have a redirection using Storage
s. We have two problems with this:
Type::Pointer { class, .. }
or in the struct PointerDeclaraion
field, is arbitrary. If we move the pointer class and array length attributes to the respective enum Type
variants, we basically end up with Storage<Type>
for dependent types. And if we have Storage<Type>
, we might as well store all types in it, not just the dependent ones.One interesting idea that I thought worth considering is changing the representation a bit based on whether the name or decorations are present. I.e. these become the signs for the IR to make it a separate entry somewhere, which leads to a separate typedef when generating MSL and such.
Shipping glsl_to_spirv has been a major pain for all gfx-rs related projects, see gfx-rs/gfx#2546
Building and deploying it is a portability nightmare.
It would be great to have a front-end for GLSL here, in the lines of https://github.com/jrmuizel/glsl-to-spirv , which uses https://crates.io/crates/glsl parser.
As I was testing to run this following GLSL code in SPIR-V format using the SPIR-V front-end I got an UnsupportedInstruction error for OpConstantComposite.
This is the fragment part of the quad.wgsl part in GLSL
#version 450
layout(location = 0) in vec2 a_uv;
layout(location = 0) out vec4 o_color;
void main()
{
o_color = vec4(1, 0, 0, 1);
return;
}
Eventually the o_color OpStore instruction is OpStore %o_color %12
where %12 is the OpConstComposite: %12 = OpConstantComposite %v4float %float_1 %float_0 %float_0 %float_1
.
Backend for SPIR-V (based on SPIR-V 1.0 for now). This will be done in many small iterations, so we can get a tight feedback loop from the early adaptors that will start using the WGSL -> SPIR-V path.
✔️ = Done - 🚧 = In Progress - 📃 = Planned
Word | Status |
---|---|
Magic Number | ✔️ |
Version Number | ✔️ |
Generator's Magic Number | ✔️ |
Bound | ✔️ |
Instruction Schema | ✔️ |
OpCode | Status |
---|---|
OpNop | |
OpUndef |
OpCode | Status |
---|---|
OpSourceContinued | |
OpSource | ✔️ |
OpSourceExtension | |
OpName | ✔️ |
OpMemberName | ✔️ |
OpString | |
OpLine | |
OpNoLine |
OpCode | Status |
---|---|
OpDecorate | ✔️ |
OpMemberDecorate | ✔️ |
OpDecorationGroup | |
OpGroupDecorate | |
OpGroupMemberDecorate |
OpCode | Status |
---|---|
OpExtension | |
OpExtInstImport | ✔️ |
OpExtInst |
OpCode | Status |
---|---|
OpMemoryModel | ✔️ |
OpEntryPoint | ✔️ |
OpExecutionMode | ✔️ |
OpCapability | ✔️ |
OpCode | Status |
---|---|
OpTypeVoid | ✔️ |
OpTypeBool | ✔️ |
OpTypeInt | ✔️ |
OpTypeFloat | ✔️ |
OpTypeVector | ✔️ |
OpTypeMatrix | ✔️ |
OpTypeImage | ✔️ |
OpTypeSampler | ✔️ |
OpTypeSampledImage | ✔️ |
OpTypeArray | ✔️ |
OpTypeRuntimeArray | ✔️ |
OpTypeStruct | ✔️ |
OpTypeOpaque | |
OpTypePointer | ✔️ |
OpTypeFunction | ✔️ |
OpTypeEvent | |
OpTypeDeviceEvent | |
OpTypeReserveId | |
OpTypeQueue | |
OpTypePipe | |
OpTypeForwardPointer |
OpCode | Status |
---|---|
OpConstantTrue | ✔️ |
OpConstantFalse | ✔️ |
OpConstant | ✔️ |
OpConstantComposite | ✔️ |
OpConstantSampler | |
OpConstantNull | |
OpSpecConstantTrue | |
OpSpecConstantFalse | |
OpSpecConstant | |
OpSpecConstantComposite | |
OpSpecConstantOp |
OpCode | Status |
---|---|
OpVariable | ✔️ |
OpImageTexelPointer | |
OpLoad | ✔️ |
OpStore | ✔️ |
OpCopyMemory | |
OpCopyMemorySized | |
OpAccessChain | ✔️ |
OpInBoundsAccessChain | |
OpPtrAccessChain | |
OpArrayLength | |
OpGenericPtrMemSemantics | |
OpInBoundsPtrAccessChain |
OpCode | Status |
---|---|
OpFunction | ✔️ |
OpFunctionParameter | |
OpFunctionEnd | |
OpFunctionCall | ✔️ |
OpCode | Status |
---|---|
OpSampledImage | ✔️ |
OpImageSampleImplicitLod | ✔️ |
OpImageSampleExplicitLod | |
OpImageSampleDrefImplicitLod | |
OpImageSampleDrefExplicitLod | |
OpImageSampleProjImplicitLod | |
OpImageSampleProjExplicitLod | |
OpImageSampleProjDrefImplicitLod | |
OpImageSampleProjDrefExplicitLod | |
OpImageFetch | |
OpImageGather | |
OpImageDrefGather | |
OpImageRead | |
OpImageWrite | |
OpImage | |
OpImageQueryFormat | |
OpImageQueryOrder | |
OpImageQuerySizeLod | |
OpImageQuerySize | |
OpImageQueryLod | |
OpImageQueryLevels | |
OpImageQuerySamples | |
OpImageSparseSampleImplicitLod | |
OpImageSparseSampleExplicitLod | |
OpImageSparseSampleDrefImplicitLod | |
OpImageSparseSampleDrefExplicitLod | |
OpImageSparseSampleProjImplicitLod | |
OpImageSparseSampleProjExplicitLod | |
OpImageSparseSampleProjDrefImplicitLod | |
OpImageSparseSampleProjDrefExplicitLod | |
OpImageSparseFetch | |
OpImageSparseGather | |
OpImageSparseDrefGather | |
OpImageSparseTexelsResident | |
OpImageSparseRead |
OpCode | Status |
---|---|
OpConvertFToU | ✔️ |
OpConvertFToS | ✔️ |
OpConvertSToF | ✔️ |
OpConvertUToF | ✔️ |
OpUConvert | |
OpSConvert | |
OpFConvert | |
OpQuantizeToF16 | |
OpConvertPtrToU | |
OpSatConvertSToU | |
OpSatConvertUToS | |
OpConvertUToPtr | |
OpPtrCastToGeneric | |
OpGenericCastToPtr | |
OpGenericCastToPtrExplicit | |
OpBitcast |
OpCode | Status |
---|---|
OpVectorExtractDynamic | ✔️ |
OpVectorInsertDynamic | |
OpVectorShuffle | |
OpCompositeConstruct | ✔️ |
OpCompositeExtract | ✔️ |
OpCompositeInsert | |
OpCopyObject | ✔️ |
OpTranspose |
OpCode | Status |
---|---|
OpSNegate | ✔️ |
OpFNegate | ✔️ |
OpIAdd | ✔️ |
OpFAdd | ✔️ |
OpISub | ✔️ |
OpFSub | ✔️ |
OpIMul | ✔️ |
OpFMul | ✔️ |
OpUDiv | ✔️ |
OpSDiv | ✔️ |
OpFDiv | ✔️ |
OpUMod | ✔️ |
OpSRem | |
OpSMod | ✔️ |
OpFRem | |
OpFMod | ✔️ |
OpVectorTimesScalar | ✔️ |
OpMatrixTimesScalar | ✔️ |
OpVectorTimesMatrix | ✔️ |
OpMatrixTimesVector | ✔️ |
OpMatrixTimesMatrix | ✔️ |
OpOuterProduct | |
OpDot | ✔️ |
OpIAddCarry | |
OpISubBorrow | |
OpUMulExtended | |
OpSMulExtended |
OpCode | Status |
---|---|
OpShiftRightLogical | ✔️ |
OpShiftRightArithmetic | ✔️ |
OpShiftLeftLogical | ✔️ |
OpBitwiseOr | ✔️ |
OpBitwiseXor | ✔️ |
OpBitwiseAnd | ✔️ |
OpNot | ✔️ |
OpBitFieldInsert | |
OpBitFieldSExtract | |
OpBitFieldUExtract | |
OpBitReverse | |
OpBitCount |
OpCode | Status |
---|---|
OpAny | |
OpAll | |
OpIsNan | |
OpIsInf | |
OpIsFinite | |
OpIsNormal | |
OpSignBitSet | |
OpLessOrGreater | |
OpOrdered | |
OpUnordered | |
OpLogicalEqual | ✔️ |
OpLogicalNotEqual | ✔️ |
OpLogicalOr | ✔️ |
OpLogicalAnd | ✔️ |
OpLogicalNot | ✔️ |
OpSelect | |
OpIEqual | ✔️ |
OpINotEqual | ✔️ |
OpUGreaterThan | ✔️ |
OpSGreaterThan | ✔️ |
OpUGreaterThanEqual | ✔️ |
OpSGreaterThanEqual | ✔️ |
OpULessThan | ✔️ |
OpSLessThan | ✔️ |
OpULessThanEqual | ✔️ |
OpSLessThanEqual | ✔️ |
OpFOrdEqual | ✔️ |
OpFUnordEqual | |
OpFOrdNotEqual | ✔️ |
OpFUnordNotEqual | |
OpFOrdLessThan | ✔️ |
OpFUnordLessThan | |
OpFOrdGreaterThan | ✔️ |
OpFUnordGreaterThan | |
OpFOrdLessThanEqual | ✔️ |
OpFUnordLessThanEqual | |
OpFOrdGreaterThanEqual | ✔️ |
OpFUnordGreaterThanEqual |
OpCode | Status |
---|---|
OpDPdx | |
OpDPdy | |
OpFwidth | |
OpDPdxFine | |
OpDPdyFine | |
OpFwidthFine | |
OpDPdxCoarse | |
OpDPdyCoarse | |
OpFwidthCoarse |
OpCode | Status |
---|---|
OpPhi | |
OpLoopMerge | ✔️ |
OpSelectionMerge | ✔️ |
OpLabel | ✔️ |
OpBranch | ✔️ |
OpBranchConditional | ✔️ |
OpSwitch | |
OpKill | ✔️ |
OpReturn | ✔️ |
OpReturnValue | ✔️ |
OpUnreachable | |
OpLifetimeStart | |
OpLifetimeStop |
OpCode | Status |
---|---|
OpAtomicLoad | |
OpAtomicStore | |
OpAtomicExchange | |
OpAtomicCompareExchange | |
OpAtomicCompareExchangeWeak | |
OpAtomicIIncrement | |
OpAtomicIDecrement | |
OpAtomicIAdd | |
OpAtomicISub | |
OpAtomicSMin | |
OpAtomicUMin | |
OpAtomicSMax | |
OpAtomicUMax | |
OpAtomicAnd | |
OpAtomicOr | |
OpAtomicXor | |
OpAtomicFlagTestAndSet | |
OpAtomicFlagClear |
OpCode | Status |
---|---|
OpEmitVertex | |
OpEndPrimitive | |
OpEmitStreamVertex | |
OpEndStreamPrimitive |
OpCode | Status |
---|---|
OpControlBarrier | |
OpMemoryBarrier |
OpCode | Status |
---|---|
OpGroupAsyncCopy | |
OpGroupWaitEvents | |
OpGroupAll | |
OpGroupAny | |
OpGroupBroadcast | |
OpGroupIAdd | |
OpGroupFAdd | |
OpGroupFMin | |
OpGroupUMin | |
OpGroupSMin | |
OpGroupFMax | |
OpGroupUMax | |
OpGroupSMax | |
OpSubgroupBallotKHR | |
OpSubgroupFirstInvocationKHR | |
OpSubgroupReadInvocationKHR | |
OpGroupIAddNonUniformAMD | |
OpGroupFAddNonUniformAMD | |
OpGroupFMinNonUniformAMD | |
OpGroupUMinNonUniformAMD | |
OpGroupSMinNonUniformAMD | |
OpGroupFMaxNonUniformAMD | |
OpGroupUMaxNonUniformAMD | |
OpGroupSMaxNonUniformAMD |
OpCode | Status |
---|---|
OpEnqueueMarker | |
OpEnqueueKernel | |
OpGetKernelNDrangeSubGroupCount | |
OpGetKernelNDrangeMaxSubGroupSize | |
OpGetKernelWorkGroupSize | |
OpGetKernelPreferredWorkGroupSizeMultiple | |
OpRetainEvent | |
OpReleaseEvent | |
OpCreateUserEvent | |
OpIsValidEvent | |
OpSetUserEventStatus | |
OpCaptureEventProfilingInfo | |
OpGetDefaultQueue | |
OpBuildNDRange |
OpCode | Status |
---|---|
OpReadPipe | |
OpWritePipe | |
OpReservedReadPipe | |
OpReservedWritePipe | |
OpReserveReadPipePackets | |
OpReserveWritePipePackets | |
OpCommitReadPipe | |
OpCommitWritePipe | |
OpIsValidReserveId | |
OpGetNumPipePackets | |
OpGetMaxPipePackets | |
OpGroupReserveReadPipePackets | |
OpGroupReserveWritePipePackets | |
OpGroupCommitReadPipe | |
OpGroupCommitWritePipe |
Function | Status |
---|---|
Round | ✔️ |
RoundEven | |
Trunc | ✔️ |
FAbs | ✔️ |
SAbs | ✔️ |
FSign | ✔️ |
SSign | ✔️ |
Floor | ✔️ |
Ceil | ✔️ |
Fract | ✔️ |
Radians | |
Degrees | |
Sin | ✔️ |
Cos | ✔️ |
Tan | ✔️ |
Asin | ✔️ |
Acos | ✔️ |
Atan | ✔️ |
Sinh | ✔️ |
Cosh | ✔️ |
Tanh | ✔️ |
Asinh | |
Acosh | |
Atanh | |
Atan2 | ✔️ |
Pow | ✔️ |
Exp | ✔️ |
Log | ✔️ |
Exp2 | ✔️ |
Log2 | ✔️ |
Sqrt | ✔️ |
InverseSqrt | ✔️ |
Determinant | ✔️ |
MatrixInverse | |
Modf | ✔️ |
ModfStruct | |
FMin | ✔️ |
UMin | ✔️ |
SMin | ✔️ |
FMax | ✔️ |
UMax | ✔️ |
SMax | ✔️ |
FClamp | ✔️ |
UClamp | ✔️ |
SClamp | ✔️ |
FMix | ✔️ |
Step | ✔️ |
SmoothStep | ✔️ |
Fma | ✔️ |
Frexp | ✔️ |
FrexpStruct | |
Ldexp | ✔️ |
PackSnorm4x8 | |
PackUnorm4x8 | |
PackSnorm2x16 | |
PackUnorm2x16 | |
PackHalf2x16 | |
PackDouble2x32 | |
UnpackSnorm2x16 | |
UnpackUnorm2x16 | |
UnpackHalf2x16 | |
UnpackSnorm4x8 | |
UnpackUnorm4x8 | |
UnpackDouble2x32 | |
Length | ✔️ |
Distance | ✔️ |
Cross | ✔️ |
Normalize | ✔️ |
FaceForward | ✔️ |
Reflect | ✔️ |
Refract | |
FindILsb | |
FindSMsb | |
FindUMsb | |
InterpolateAtCentroid | |
InterpolateAtSample | |
InterpolateAtOffset | |
NMin | |
NMax | |
NClamp |
I was thinking that it would be great to be able to save (and load?) RON files by the converter. The joys of RON is that it's very nice to inspect visually (by humans):
Handle(xxx)
values through the rest of the moduleDebug
implementationTo get this, we need a serde
optional feature, implementation of Serialize
+Deserialize
for all IR objects, and also gfx-rs/rspirv#123 resolved.
I threw together a quick, partial implementation of the glsl preprocessor (notably doesn't support #if
or #elif
because I didn't want to implement constant expression evaluation) for bevy (here) because the glsl-new
frontend doesn't have one. I'm happy to donate this to naga if it's wanted as a starting point.
Having the following WGSL:
[[location 0]] var<in> a_uv : vec2<f32>;
[[location 0]] var<out> o_color : vec4<f32>;
fn main_frag() -> void {
o_color = vec4<f32>(1, 0, 0, 1);
return;
}
entry_point fragment as "main" = main_frag;
The values provided into the vector float 4 with the 4 integer values will be correctly parsed to the IR model:
Constant {
name: None,
specialization: None,
inner: Sint(
1,
),
ty: Handle(4),
},
Constant {
name: None,
specialization: None,
inner: Sint(
0,
),
ty: Handle(4),
},
The back-ends would now have to check if the vector where these constants are provided for is able to use them. I think it would be better to let the WGSL front-end check if the values in a vector are of the same Type as the composite they are in.
This is not something only applicable to vectors, and should have a general type check to ensure a correct IR model is delivered to the back-end.
We have the image type variant define as follows:
Image {
kind: ScalarKind,
dim: ImageDimension,
arrayed: bool,
class: ImageClass,
},
However, the kind
member is redundant for ImageClass::Depth
(always Float
) and ImageClass:Storage(format)
(always computed as ScalarKind::from(format)
).
Therefore, we may consider to make this less redundant:
Image {
dim: ImageDimension,
arrayed: bool,
class: ImageClass,
},
pub enum ImageClass {
Sampled { kind: ScalarKind, multi: bool },
Depth,
Storage(StorageFormat),
}
I think getting less redundancy in IR is always welcome!
The possible downside is that all the front-ends already know this kind
, and all the backends need it. But that is covered by the typifier anyway, which the backends use.
This is what we are missing to load SPIR-Vs of wgpu-rs examples for introspection
WorkgroupSize
builtinUnsupportedInstruction(Function, Bitcast)
UnsupportedExtInst(25)
InvalidWordCount
after Switch
instructionUnsupportedInstruction(Function, MatrixTimesMatrix)
UnsupportedInstruction(Function, ConvertSToF)
MissingDecoration(Offset)
UnsupportedInstruction(Function, Bitcast)
InvalidImageExpression(Handle(10))
UnsupportedExtInst(69)
UnsupportedExtInst(49)
UnsupportedInstruction(Function, Kill)
UnsupportedInstruction(Function, ConvertSToF)
For the back-end there is now no information about where a function in the call expression is located. It could be a intrinsic/built-in, imported or user declared function.
In case for SPIR-V there are other opcode instructions used for all three cases. So it would be very hard if not impossible at the moment to determine where to look for the function and what opcode to use.
We have many front-ends, many back-ends, and we need to test them all using the a limited number of shaders in the tree (in test-data
), and we need to do it in a way that is straightforward and extendable. Here is an idea on how to do it, inspired by Rosetta stone.
Steps:
The main idea is to bring structure to the tested material, so that we know how to write the tests and configure them. Basically, the test suite is a collection of those base IR shaders. For each we can define a list of front-ends to test (with the source files) and a list of back-ends to test (with the source files), plus possibly some configuration tables.
One way to structure this is test-data/boids/
folder having all of shader.wgsl
, shader.spv
, shader.glsl
, config.ron
(testing configuration), and shader.ron
(the IR).
Caveats:
Implement all the necessary changes needed to create a working Boids example.
Todo for working Boids:
For WGSL, it is possible to use the "GLSL.std.450" import, as can be seen here in the spec. WGSL constraints this to only this import.
For instance, the SPIR-V backend now always includes this import statically, even though it isn't present in the WGSL. This causes an extra instruction, but it would be nice to keep it clean.
To continue on the previous point: as the import is always statically included, other front-ends are also limited by the WGSL constraint of only having the "GLSL.std.450" import available.
I think it would be a good idea to have a representation of "imports" in the IR so all the back-ends only have to loop through it and include it.
Incoming SPIR-Vs have Bitcast
. That's the last major blocker in validating wgpu-rs examples.
We need to do a research about how this looks like in WGSL, what the previous investigations are, and find a way to process this instruction.
The following shaders are not validated:
#version 450
layout(location = 0) in vec2 a_position;
layout(location = 0) out vec2 out_tile_index;
void main() {
const vec2 position = a_position * 0.8;
gl_Position = vec4(position.x, position.y, 0.8, 1.0);
out_tile_index = (a_position + 1.0) / 2.0 * 2.0;
}
#version 450
layout(location = 0) in vec2 in_tile_index;
layout(location = 0) out vec4 f_colour;
layout(set = 0, binding = 0) buffer readonly Map { uint a_test[]; };
void main() {
uint tile_index = a_test[uint(in_tile_index.y) * 2 + uint(in_tile_index.x)];
const vec3 colours[] = vec3[](vec3(1.0, 0.0, 0.0), vec3(0.0, 1.0, 0.0),
vec3(0.0, 0.0, 1.0), vec3(1.0, 1.0, 0.0));
vec3 colour =
tile_index < colours.length() ? colours[tile_index] : vec3(0.0, 0.0, 0.0);
f_colour = vec4(colour, 1.0);
}
[0.040029 WARN]()(no module): Unable to find layer: VK_LAYER_KHRONOS_validation
[0.055584 WARN]()(wgpu_core::device): Failed to parse shader SPIR-V code: UnsupportedBuiltIn(4)
[0.055656 WARN]()(wgpu_core::device): Shader module will not be validated
[0.055858 WARN]()(no module): Unknown decoration NonWritable
[0.056285 WARN]()(wgpu_core::device): Failed to parse shader SPIR-V code: UnsupportedInstruction(Function, ConvertFToU)
[0.056316 WARN]()(wgpu_core::device): Shader module will not be validated
Because I am apparently subscribed to issues here but I can't even remember what it's for anymore.
The GLSL front-end is lacking a Module file structure at the moment. Because of this, in parser.rs the Path attribute is used to refer to a file.
I found out this does not work on Windows, and read about the Path attribute being an anti-pattern
I suggest using the same structure as the SPIR-V back-end, for all future new front- and back-ends.
In spir-v front-end, constants are always declared up-front, and then come first in the expression list.
In other front-ends constants can be inlined, and can come later...
A typical task is figuring out the component type of something (see #24). Vectors and Matrices pose a problem because they are fully specified, so while we can always construct a base type, it's not registered in the type arena.
I'm thinking that instead we could have just a Vector
type with a size and a base type.
It was introduced when the types weren't copyable. Now they are. Maybe we can simplify this better.
That would make all the Option<Handle<T>>
to be free and not pollute the data cache.
https://github.com/mitsuhiko/insta
Depends on #32
It's likely a small dependency, but still - Naga's IR doesn't have to include any SPIR-V types directly. We could detach the IR from spir-v and only enabled the latter when either front-end or back-end is requested as a feature.
Having the following WGSL input:
[[location 0]] var<in> a_uv : vec2<f32>;
[[location 0]] var<out> o_color : vec4<f32>;
fn main() -> void {
o_color = vec4<f32>(1.0, 0.0, 0.0, 1.0);
return;
}
entry_point fragment as "main" = main;
The vector float 4 is parsing this to the following expression:
Constant(
Handle(1),
),
Constant(
Handle(2),
),
Constant(
Handle(3),
),
Constant(
Handle(4),
),
Compose {
ty: Handle(2),
components: [
Handle(3),
Handle(4),
Handle(5),
Handle(6),
],
},
You would expect that two constant handles would point to the value "1.0" constant, and two constant handles point to the value "0.0" constant. Each float is parsed as a separate float constant, despite having the same value.
Function calling is not yet supported in the WGSL front-end. The following WGSL throws an error:
fn main_vert() -> void {
test_function(1.0);
return;
}
fn test_function(test: f32) -> void {
return;
}
entry_point vertex as "main" = main_vert;
Error: ParseError { error: UnknownIdent("test_function"), scopes: [FunctionDecl, Block, Statement], pos: (6, 15) }
We have more panics than needed. The corners were cut when writing the backend as a PoC. However, for any serious use, it should return an error instead of panicing. This applies to all the front-ends and backends. Panic is only allowed where our internal consistency is shattered, and even then it would work best if we just error out.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.