Giter Club home page Giter Club logo

shaderconductor's Introduction

ShaderConductor

Build Status License

ShaderConductor is a tool designed for cross-compiling HLSL to other shading languages.

Features

  • Converts HLSL to readable, usable and efficient GLSL
  • Converts HLSL to readable, usable and efficient ESSL
  • Converts HLSL to readable, usable and efficient Metal Shading Language (MSL)
  • Converts HLSL to readable, usable and efficient old shader model HLSL
  • Supports all stages of shaders, vertex, pixel, hull, domain, geometry, and compute.

Note that this project is still in an early stage, and it is under active development.

Architecture

ShaderConductor is not a real compiler. Instead, it glues existing open source components to do the cross-compiling.

  1. DirectX Shader Compiler to compile HLSL to DXIL or SPIR-V,
  2. SPIRV-Cross to convert SPIR-V to target shading languages.

Architecture

Prerequisites

  • Git. Put git into the PATH is recommended.
  • Visual Studio 2017. Select the following workloads: Universal Windows Platform Development and Desktop Development with C++.
  • CMake. Version 3.9 or up. It's highly recommended to choose "Add CMake to the system PATH for all users" during installation.
  • Python. Version 2.7 or up. You need not change your PATH variable during installation.

Building

ShaderConductor has been tested on Windows, Linux, and macOS.

The script way:

  BuildAll.py <BuildSystem> <Compiler> <Architecture> <Configuration>

where,

  • <BuildSystem> can be ninja or vs2017. Default is vs2017.
  • <Compiler> can be vc141 on Windows, gcc or clang on Linux, clang on macOS.
  • <Architecture> must be x64 (for now).
  • <Configuration> can be Debug, Release, RelWithDebInfo, or MinSizeRel. Default is Release.

This script automatically grabs external dependencies to External folder, generates project file in Build/<BuildSystem>-<Compiler>-<Platform>-<Architecture>[-<Configuration>], and builds it.

The manual way:

  mkdir Build
  cd Build
  cmake -G "Visual Studio 15" -T host=x64 -A x64 ../
  cmake --build .

After building, the output file ShaderConductor.dll can be located in <YourCMakeTargetFolder>/Bin/<Configuration>/. It depends on dxcompiler.dll in the same folder.

Artifacts

You can download the prebuilt binaries generated by CI system. Currently, artifacts for Windows, Linux, macOS are published every commit.

License

ShaderConductor is distributed under the terms of MIT License. See LICENSE for details.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

shaderconductor's People

Contributors

amzeratul avatar gongminmin avatar jorgemagic avatar microsoft-github-policy-service[bot] avatar microsoftopensource avatar msftgits avatar qartar avatar slin avatar sunnypchen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

shaderconductor's Issues

Remove the duplicate external libraries

GTest is depended by DirectXShaderCompiler and ShaderConductor. It should be moved to External and shared by them.

Also, SPIRV-Tools and SPIRV-Headers are potentially depended by DirectXShaderCompiler and SPIRV-Cross. Need to be handled as well.

Load ShaderConductor dependencies on Mac and Linux

The dynamic libraries on Mac using rpath mechanisms to load dependency libraries, so is important to be in mind when you create a dylib, you must specify correct ID, dependencies and rpaths where linker will find dependency libraries.

If you download ShaderConductor Mac dylib from AzurePipeline artifacts and you use for example MacDependencies app, you can see:

File: libShaderConductor.dylib
ID: @rpath/libShaderConductor.dylib
Dependency with: @rpath/libdxcompiler.3.7.dylib
Rpaths: /Users/vts/agent/2.155.1/work/1/s/Build/ninja-osx-clang9-x64-Release/External/DirectXShaderCompiler/lib.

I created a netcore 3.0 app and make a NativeLibrary.Load to load libShaderConductor.dylib but it produces a fail when load dxcompiler library dependency.

Issues:
Dependency filename is wrong, @rpath/libdxcompiler.3.7.dylib must be @rpath/libdxcompiler.dylib
RPaths is wrong, RPaths could be @loader_path; @executable_path

Is possible fix this references using install_name_tool on Mac but maybe is possible fix it from ninja script.

This is an interesting article about this issue on Mac.

Edit. In Linux exists the same issue, it is possible fix with patchelf command but would be fixed in ninja script.

patchelf --set-sorname libdxcompiler.so libdxcompiler.so
patchelf --set-rpath '$ORIGIN' libShaderConductor.so

Half data type support

DXC supports half type in shader model 6.2, but there are some problems. Need to add tests to capture them.

CMake project generation requires git.exe to be in PATH

Describe the bug
CMake project generation requires git.exe to be in PATH in order to successfully run the UpdateExternalLib function defined in External/CMakeLists.txt. This requirement is not explicitly stated in README.md even though the instructions for CMake and Python both mention the PATH variable.

To Reproduce

  1. Clone the repository
  2. In a context where git is not in PATH run Build.py or cmake -G ... as instructed in README.md

Expected behavior
Expected CMake project generation to complete successfully. Instead CMake silently fails when running the git commands in UpdateExternalLib and generates errors later when attempting to call add_subdirectory on the cloned external repositories and subsequent commands on targets in those repositories.

Additional context
I hit this issue because I have git installed into the PATH of Git Bash for Windows only and tried to generate the project files in the developer command prompt because of an issue with CMake and Visual Studio 2019.

Not sure why git submodules aren't being used for this step.

Error building on Mac M1 Monterrey

Describe the bug

[1176/1181] Building CXX object Source/Core/CMakeFiles/ShaderConductor.dir/ShaderConductor.cpp.o
FAILED: Source/Core/CMakeFiles/ShaderConductor.dir/ShaderConductor.cpp.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DSHADER_CONDUCTOR_SOURCE -DShaderConductor_EXPORTS -I/Users/douglaslassance/Desktop/WIP/ShaderConductor/Include -I/Users/douglaslassance/Desktop/WIP/ShaderConductor/Build/Ninja-osx-clang-x64-Release/External/DirectXShaderCompiler/include -I/Users/douglaslassance/Desktop/WIP/ShaderConductor/External/DirectXShaderCompiler/include -I/Users/douglaslassance/Desktop/WIP/ShaderConductor/External/SPIRV-Cross -I/Users/douglaslassance/Desktop/WIP/ShaderConductor/External/SPIRV-Tools/include -std=c++1z -fms-extensions -Wno-language-extension-token -W -Wall -Werror -march=core2 -msse2 -Wno-missing-field-initializers -m64 -DNDEBUG -O2 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -fPIC -MD -MT Source/Core/CMakeFiles/ShaderConductor.dir/ShaderConductor.cpp.o -MF Source/Core/CMakeFiles/ShaderConductor.dir/ShaderConductor.cpp.o.d -o Source/Core/CMakeFiles/ShaderConductor.dir/ShaderConductor.cpp.o -c /Users/douglaslassance/Desktop/WIP/ShaderConductor/Source/Core/ShaderConductor.cpp
clang: error: the clang compiler does not support '-march=core2'
[1177/1181] Building CXX object Source/Tests/CMakeFiles/ShaderConductorTest.dir/ShaderConductorTest.cpp.o
FAILED: Source/Tests/CMakeFiles/ShaderConductorTest.dir/ShaderConductorTest.cpp.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DTEST_DATA_DIR=\"/Users/douglaslassance/Desktop/WIP/ShaderConductor/Source/Tests/Data/\" -I/Users/douglaslassance/Desktop/WIP/ShaderConductor/Include -isystem /Users/douglaslassance/Desktop/WIP/ShaderConductor/External/googletest/googletest/include -isystem /Users/douglaslassance/Desktop/WIP/ShaderConductor/External/googletest/googletest -std=c++1z -fms-extensions -Wno-language-extension-token -W -Wall -Werror -march=core2 -msse2 -Wno-missing-field-initializers -m64 -DNDEBUG -O2 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -MD -MT Source/Tests/CMakeFiles/ShaderConductorTest.dir/ShaderConductorTest.cpp.o -MF Source/Tests/CMakeFiles/ShaderConductorTest.dir/ShaderConductorTest.cpp.o.d -o Source/Tests/CMakeFiles/ShaderConductorTest.dir/ShaderConductorTest.cpp.o -c /Users/douglaslassance/Desktop/WIP/ShaderConductor/Source/Tests/ShaderConductorTest.cpp
clang: error: the clang compiler does not support '-march=core2'
[1178/1181] Building CXX object Source/Tools/CMakeFiles/ShaderConductorCmd.dir/ShaderConductorCmd.cpp.o
FAILED: Source/Tools/CMakeFiles/ShaderConductorCmd.dir/ShaderConductorCmd.cpp.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++  -I/Users/douglaslassance/Desktop/WIP/ShaderConductor/Include -I/Users/douglaslassance/Desktop/WIP/ShaderConductor/External/cxxopts/include -std=c++1z -fms-extensions -Wno-language-extension-token -W -Wall -Werror -march=core2 -msse2 -Wno-missing-field-initializers -m64 -DNDEBUG -O2 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk -MD -MT Source/Tools/CMakeFiles/ShaderConductorCmd.dir/ShaderConductorCmd.cpp.o -MF Source/Tools/CMakeFiles/ShaderConductorCmd.dir/ShaderConductorCmd.cpp.o.d -o Source/Tools/CMakeFiles/ShaderConductorCmd.dir/ShaderConductorCmd.cpp.o -c /Users/douglaslassance/Desktop/WIP/ShaderConductor/Source/Tools/ShaderConductorCmd.cpp
clang: error: the clang compiler does not support '-march=core2'
ninja: build stopped: subcommand failed.
[E] Build failed.

To reproduce

brew install ninja
git clone https://github.com/microsoft/ShaderConductor.git
cd ShaderConductor
python ./BuildAll.py Ninja clang

Additional context

❯ sw_vers
ProductName:	macOS
ProductVersion:	12.0.1
BuildVersion:	21A559
❯ clang --version
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin21.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
❯ python --version
Python 3.7.12

Also had to change BuildAll.py as such:

diff --git a/BuildAll.py b/BuildAll.py
index 9af0521..297e1bc 100644
--- a/BuildAll.py
+++ b/BuildAll.py
@@ -224,7 +224,7 @@ if __name__ == "__main__":
 		hostArch = "x64"
 	elif (hostArch == "i386"):
 		hostArch = "x86"
-	elif (hostArch == "ARM64"):
+	elif (hostArch.lower() == "arm64"):
 		hostArch = "arm64"
 	else:
 		LogError("Unknown host architecture %s.\n" % hostArch)

Metal macosx- Shift cbuffer bindings broken

If this is supposed to shift the binding number for buffers in metal then it's not working, otherwise a comment mentioning specifically which shading languages it applies to would be nice.

To reproduce- I expect this will be the case with any HLSL shader to the msl_macos target, likely ios too but I've not checked.

Expected behavior- I expected that the constant buffers binding #'s would come out in MSL as either (the source binding number, depending on the input language, or 0) + the shift cbuffers int parameter in Compiler::Options

I should perhaps mention this is only significant as an issue because I'm trying to automate the process of converting shaders from HLSL to MSL as part of the build process for a project.

LLVM error in ::dlopen running on Mac, when CMake project loads another CMake project

Describe the bug
Crash trying to load dynamic library on Mac, if the library is loaded as part of another library using CMake.

To Reproduce
Steps to reproduce the behavior:

  1. Input Shader:
// Example.hlsl

cbuffer Settings : register(b0) {
	float4x4 wvpMatrix;
};

struct InputVS
{
	float3 position : POSITION;
	float3 color : COLOR;
};

struct OutputVS
{
	float4 position : SV_Position;
	float3 color : COLOR;
};

// Vertex shader main function
OutputVS VS(InputVS inp)
{
	OutputVS outp;
	outp.position = mul(wvpMatrix,float4(inp.position, 1));
	outp.color = inp.color;
	return outp;
}

// Pixel shader main function
float4 PS(OutputVS inp) : SV_Target
{
	return float4(inp.color, 1);
};
  1. Full parameters
    • ShaderSrc is C string of the input shader
    • Options left at defaults
    • TargetDesc::language = ShadingLanguage::Msl_macOS
    • TargetDesc::version = "1.0"
    • TargetDesc::asModule = true
  2. File structure:
bug-repro
|
+ -- CMakeLists.txt
+ -- shader-transpiler
   |
   + -- CMakeLists.txt
   + -- main.cpp
   + -- ShaderConductor

Create a CMakeLists.txt in a folder named shader-transpiler with the following contents:

cmake_minimum_required(VERSION 3.17)
PROJECT(shader-transpiler)

add_subdirectory("${CMAKE_CURRENT_SOURCE_DIR}/ShaderConductor")

add_executable("${PROJECT_NAME}" "main.cpp")
target_include_directories("${PROJECT_NAME}" PRIVATE 
    "${ST_DEPS_DIR}/ShaderConductor/External/cxxopts/include"
    )

target_link_libraries("${PROJECT_NAME}" PRIVATE "ShaderConductor")

Example main.cpp:

#include <ShaderConductor/ShaderConductor.hpp>
#include <fstream>
#include <streambuf>
#include <cassert>
#include <iostream>

using namespace std;
using namespace ShaderConductor;

int main(){
	
	//read in source file
	auto name = "Example.hlsl";
	
	ifstream in(name);
	assert(in.good());
	string shader_src((std::istreambuf_iterator<char>(in)),
					std::istreambuf_iterator<char>());
	
	//describe the input
	Compiler::SourceDesc source;
	source.source = shader_src.c_str();
	source.entryPoint = "VS";
	//source.fileName = name;
	source.defines = nullptr;
	source.numDefines = 0;
	source.stage = ShaderStage::VertexShader;
	
	//default options
	Compiler::Options options;
	
	//describe the output
	Compiler::TargetDesc target;
	target.language = ShadingLanguage::Msl_macOS;
	target.version = "2.0";
	target.asModule = true;
	
	auto result = Compiler::Compile(source, options, target);
	
	ofstream out("Example_DS.metal", ios::out | (!result.isText ? ios::binary : 0 ));
	if(result.hasError){
		char* errormsg = (char*)result.errorWarningMsg->Data();
		cerr << "Compile Failed: " << errormsg << endl;
		return -1;
	}
	else{
		char* generatedsrc = (char*)result.target->Data();
		auto size = result.target->Size();
		
		for(int i = 0; i < size; i++){
			auto byte = (char)*(generatedsrc + i);
			out << byte;
		}
		
		cout << "Wrote result" << endl;
	}
	
	return 0;
}
  1. Create a second CMakeLists.txt, one directory up, with the following contents:
project(bug_demo)
add_subdirectory("shader-transpiler")
  1. Within the directory of the upper-level CMakeLists.txt, run mkdir build && cd build && cmake -G "Xcode" ..
  2. Place Example.hlsl in the build directory so the program can open it.
  3. Open the generated Xcode project, select the shader-transpiler target, and attempt to run it. The program will crash with the following error:
LLVM ERROR: inconsistency in registered CommandLine options
LLVM ERROR: IO failure on output stream.
Program ended with exit code: 1

The faulty line is: ShaderConductor.cpp:145

#ifdef _WIN32
            m_dxcompilerDll = ::LoadLibraryA(dllName);
#else
            m_dxcompilerDll = ::dlopen(dllName, RTLD_LAZY);    // <------ HERE
#endif

            if (m_dxcompilerDll != nullptr)
            {

Expected behavior
Compiler::Compile should succeed.

Additional context
The error does not occur if ShaderConductor is loaded from the top-level CMakeLists.txt. In the context of my project, this is not possible for me.

Add options to Compiler()

Enable to have options for compiling shader.

Consider adding some options like these:

  • Zpc: Packs matrices in column-major order by deafult
  • Zpr: Packs matrices in row-major order by deafult
  • enable-16bit-types: Half, int16_t, etc
  • O{|0|1|2|3}: Optmization levels
  • Zi: Debug info
  • ShaderModel: Assign a shader model

Legacy HLSL and bytecode support

Is your feature request related to a problem? Please describe.
Have support to compiler hlsl 5.0 and 5.1 shaders to dxc bytecode using D3DShaderCompiler, this will remove the barrier of not having geometry, hull and domain shader support, user can write macro for both 6.0+ and 5.0 targets and support both of the,

Describe the solution you'd like
Extend the ShadingLanguage with new enums and support for cso bytecode.

Describe alternatives you've considered
Now the user need to use the legacy fxc compiler for compiling geometry, hull and domain shaders to hlsl (DirectX11) targets.

Check code format in CI

Add a step to CI to run clang-format through code

  • Make sure current code is formatted
  • Run clang-format in CI

Better legacy GLSL compatibility

Is your feature request related to a problem? Please describe.
My engine supports GLES2 / WebGL1 in addition to newer OpenGL versions and Vulkan/DX12/Metal. By default, the shaders generated don't play well with the legacy GLSL versions. Since it's still a major target, I suggest some improvements & conventions to make it work out of the box.

Describe the solution you'd like
I suggest two improvements using SPRIVCross's shader reflection API:

  • Make the vertex output names match the fragment input names (renaming out_var_X and in_var_X to varying_X)
  • Append descriptor binding info into uniform names so it can be easily extracted when uniforms are enumerated after compilation/linking

Describe alternatives you've considered
The minor changes I propose should cover most of the use cases, but there might be a way to do this more generically with callbacks etc, although I feel it would go against the super streamlined design of ShaderConductor and would leak implementation details into user code.

Additional context
This is the code I'm using currently in ConvertBinary():

case ShadingLanguage::Glsl:
case ShadingLanguage::Essl:
    compiler = std::make_unique<spirv_cross::CompilerGLSL>(spirvIr, spirvSize);
    combinedImageSamplers = true;
    buildDummySampler = true;

    // Legacy GLSL fixups
    if (intVersion <= 300)
    {
        auto&& vars = compiler->get_active_interface_variables();
        for (auto& var : vars)
        {
            auto varClass = compiler->get_storage_class(var);
            
            // Make VS out and PS in variable names match
            if (source.stage == ShaderStage::VertexShader && varClass == spv::StorageClass::StorageClassOutput)
            {
                auto name = compiler->get_name(var);
                if (name.find("out_var") == 0)
                {
                    name.replace(0, 7, "varying");
                    compiler->set_name(var, name);
                }
            }
            else if (source.stage == ShaderStage::PixelShader && varClass == spv::StorageClass::StorageClassInput)
            {
                auto name = compiler->get_name(var);
                if (name.find("in_var") == 0)
                {
                    name.replace(0, 6, "varying");
                    compiler->set_name(var, name);
                }
            }
            
            // Encode binding info into variable name for uniform buffers, textures, samplers
            if (varClass == spv::StorageClass::StorageClassUniform || varClass == spv::StorageClass::StorageClassUniformConstant)
            {
                auto space = compiler->get_decoration(var, spv::Decoration::DecorationDescriptorSet);
				auto reg = compiler->get_decoration(var, spv::Decoration::DecorationBinding);
            
                char buf[128];
                sprintf(buf, "_reg_%d_space_%d", reg, space);
            
				auto type = compiler->get_type_from_variable(var);
 				auto typeName = compiler->get_name(type.self);
 				typeName.append(buf);
 				compiler->set_name(type.self, typeName);
            
                auto name = compiler->get_name(var);
                name.append(buf);
                compiler->set_name(var, name);
            }
        }
    }

    break;

HLSL转 MSL的一些疑问?

比如这样一个HLSL文件

uniform float4 data = 0;

struct VS_OUT {
float4 pos: SV_POSITION;
float4 color:COLOR;

};

VS_OUT vs_main(float4 pos: POSITION) {
VS_OUT ret = (VS_OUT)0;
ret.pos = pos + data;
return ret;
}

转化成MSL的文件

#include <metal_stdlib>
#include <simd/simd.h>

using namespace metal;

struct type_Globals
{
float4 data;
};

struct vs_main_out
{
float4 out_var_COLOR [[user(locn0)]];
float4 gl_Position [[position]];
};

struct vs_main_in
{
float4 in_var_POSITION [[attribute(0)]];
};

vertex vs_main_out vs_main(vs_main_in in [[stage_in]], constant type_Globals& _Globals [[buffer(0)]])
{
vs_main_out out = {};
out.gl_Position = in.in_var_POSITION + _Globals.data;
out.out_var_COLOR = float4(0.0);
return out;
}

这个文件中 定义了type_Globals 并把data放在了里面,问题是我如何拿到这个data的index?如果有2个uniform的话应该如何处理?不可能通过源码的方式来约定吧。 Metal本身好像没有提供这个API

Linux build failure due to -WError when building SPIRV-Tools

Describe the bug
SPIRV-Tools is compiled with the -WError flag. Building a clean clone of the repo with either clang or gcc fails because of the warning reported in KhronosGroup/SPIRV-Tools#1541. Not sure why CI builds don't run into this.

To Reproduce
Run python BuildAll.py ninja gcc or python BuildAll.py ninja clang.
Clang version: 7.0.0 (tags/RELEASE_700/final)
Gcc version: 8.2.1 20181127 (GCC)

Expected behavior
Cloning a build and running the python build script Just Works.

Additional context
I don't think ShaderConductor needs to set -WError. This can be disabled by setting SPIRV_WERROR=OFF.

Options::packMatricesInRowMajor is inverted

Describe the bug
Options::packMatricesInRowMajor is inverted. Setting it to true, passes in '-Zpc' which makes the compiler generate column major matrices.

To Reproduce
Inspect the source code:

        // HLSL matrices are translated into SPIR-V OpTypeMatrixs in a transposed manner,
        // See also https://antiagainst.github.io/post/hlsl-for-vulkan-matrices/
        if (options.packMatricesInRowMajor)
        {
            dxcArgStrings.push_back(L"-Zpc");
        }
        else
        {
            dxcArgStrings.push_back(L"-Zpr");
        }

Expected behavior
The values need to be inverted. Reading the comment it seems that this might've been intentionally inverted to fix SPIRV codegen, but my DX12/Vulkan/GLES engine works correctly for all three APIs when I pass in -Zpr and my matrices are row major in memory.

libtinfo.so.5 dependency

Describe the bug

The Linux CI builds produce a libdxcompiler.so build that dynamically links to libtinfo.so.5. That's not something that is always installed on a user machine. I have libtinfo.so.6 installed as an ncurses dependency on Arch Linux). I managed to run the CI binaries after installing ncurses5-compat-libs from the AUR. That provides libtinfo.so.5, though when running I get the following message /usr/lib/libtinfo.so.5: no version information available.

Expected behavior

There is no libtinfo dependeny.

Additional context

libtinfo is an optional dependency of LLVM, though I don't see where it is enabled exactly. It looks like it's only compiled as a dependency if the LLVM_ENABLE_TERMINFO flag is set. A cmake file in DirectXShaderCompiler checks that flag and finds a lib providing terminfo (e.g. libtinfo) here. The flag is enabled by default here. But it also looks like terminfo is disabled here. Building locally on my machine I don't get this dependency, but I'm unsure why.

Enable to compile HLSL to lib profiles

Is your feature request related to a problem? Please describe.
Instead of compiling the whole shader, compile fragments and link them later.

Describe the solution you'd like
Accept the input of HLSL fragments, compile to lib_6_0 profile. Then link them into one DXIL or SPIR-V before converting to other shading languages.

Describe alternatives you've considered
Concatenate HLSL strings to one, then compiling it.

Metal SL doesn't have GS, shouldn't support it

Describe the bug
Compile a geometry shader to MSL.

To Reproduce
ShaderConductorCmd -I Particle_GS.hlsl -S gs -T msl

Expected behavior
An error message about unknown execution model.

Additional context
A msl is generated, with "unknown" entry type inside.

ShaderConductorTest abort DxilModuleReflection::_GetConstantBufferByName

Describe the bug
run ShaderConductorTest ,and crash on LinkTest

To Reproduce

  1. vs2019 on Win10 Debug and x64 build
  2. commit id: 30a77c7 branch: maset
  3. start up with ShaderConductorTest
  4. console log :
    [----------] 1 test from LinkTest
    [ RUN ] LinkTest.LinkDxil --- after here, program crash

program is runing TEST(LinkTest, LinkDxil) at ShaderConductorTest.cpp

  1. crash info
    DxilContainerReflection.cpp
    DxilModuleReflection::_GetConstantBufferByName(LPCSTR Name)

crash on "if (it != m_StructuredBufferCBsByName.end()) " map/set iterators in range are from different containers

Name is "cbPS" which i found that in "ShaderConductor/Source/Tests/Data/Input/CalcLight.hlsl"

m_StructuredBufferCBsByName size() is 0
but
m_CBsByName size() is 1 and m_CBsByName contains "cbPS"

Remove STL containers from interface

STL containers, such as std::string, std::vector, are causing problems when calling from a program without _ITERATOR_DEBUG_LEVEL=0, or overrided memory allocation . Need to passing parameters with pure C types.

Static Library support

Is your feature request related to a problem? Please describe.
If I want to create a portable tool that has shader conversion support, or if I want to add the ability to generate shader code to a library, I need to worry about the DLLs.

Describe the solution you'd like
Use of BUILD_SHARED_LIBS in cmake or something. Whatever is easiest to implment.

Describe alternatives you've considered
N/A

Additional context
N/A

Build by Ninja

Adopt Ninja to speed up building. Currently AppVeyor CI fails because of the building time is over 60mins. Try to use Ninja to workaround this problem.

Enable to include an empty file

Describe the bug
Include an empty file in a HLSL, compiler returns "file not found".

Expected behavior
Include the empty file as no op.

Smoother ARM and ARM64 support

Currently you need to manually build x86 or x64 clang-tblgen and llvm-tblgen binaries, assign their path in CMake configurations, and generate arm and arm64 project files. We need to make it automatic.

HLSL compiled to DXIL won't load in DirectX11

Describe the bug
A HLSL vertex shader I am compiling to DXIL refuses to be created via ID3D11Device::CreateVertexShader due to the following error code:

D3D11 ERROR: ID3D11Device::CreateVertexShader: Vertex Shader is corrupt or in an unrecognized format. [ STATE_CREATION ERROR #166: CREATEVERTEXSHADER_INVALIDSHADERBYTECODE]
D3D11 ERROR: ID3D11Device::CreateVertexShader: Shader is corrupt or in an unrecognized format. [ STATE_CREATION ERROR #166: CREATEVERTEXSHADER_INVALIDSHADERBYTECODE]

To Reproduce
Shaders.zip
Steps to reproduce the behavior:

  1. Input shader attached [Mesh.vs], I have also attached the compiled vso [Compiled/Mesh.vso]
  2. Full parameters:

`Compiler::ResultDesc CompileShader(const std::string& data, ShaderStage stage)
{
Compiler::SourceDesc sourceDesc{};
Compiler::TargetDesc targetDesc{};

sourceDesc.source = data.c_str();
sourceDesc.stage = stage;
sourceDesc.entryPoint = "ShaderMain";
sourceDesc.numDefines = 0;
sourceDesc.defines = nullptr;
sourceDesc.loadIncludeCallback = LoadInclude;

Compiler::Options opts;
opts.optimizationLevel = 0;

targetDesc.asModule = false;
targetDesc.language = ShadingLanguage::Dxil;
targetDesc.version = nullptr;

auto result = Compiler::Compile(sourceDesc, {}, targetDesc);

return result;

}`
You may note that the constant buffer is commented out, this is due to another bug I have reported with the Dx shader compiler here: microsoft/DirectXShaderCompiler#3511

Expected behavior
I should be able to compile to DXIL, and load with Dx11.

Expose access to reflection data

Is your feature request related to a problem? Please describe.
With the current API it is not possible to get reflection data from built shaders.

Describe the solution you'd like
A way of accessing the SPIRV-Cross reflection data (if SPIRV-Cross is used) and DXC reflection data.

Additional context
There is an example of how to get DXC reflection data in their tests here, but looks like it's Windows only. SPIRV-Cross has a wiki page for their reflection API.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.