Hi, great work on this!

I have the following code

const camera = new THREE.PerspectiveCamera(40, window.innerWidth / window.innerHeight, 0.01, 250)
const velocityDepthNormalPass = new PostProcessing.VelocityDepthNormalEffect(this.getThreeScene(), camera);

I am currently getting the following error

I have no doubt that it is completely my problem....however, I am stumped at where to start to troubleshoot it so any suggestions would be very much appreciated. Im using Threejs r155

I get a couple of errors when i'm calling SSGI in the latest version of the repo. I built this myself since npm was a little behind. I also tested this on the older npm version and i get the same errors. I'm using a couple gltf's and a boxGeometry. I thought this had to do with uv's but it seems boxGeometry has accurate uv's so that can't be the issue. And my GLTF's rendered fine in the previous version but now they don't render at all with the updated build.

THREE.WebGLProgram: Shader Error 0 - VALIDATE_STATUS false

Material Name: undefined
Material Type: MeshPhysicalMaterial

Program Info Log: Vertex shader is not compiled.

VERTEX

ERROR: 0:400: 'uvundefined' : undeclared identifier
ERROR: 0:400: 'constructor' : not enough data provided for construction


  395: void main() {
  396: #if defined( USE_UV ) || defined( USE_ANISOTROPY )
  397: 	vUv = vec3( uv, 1 ).xy;
  398: #endif
  399: #ifdef USE_MAP
> 400: 	vMapUv = ( mapTransform * vec3( MAP_UV, 1 ) ).xy;
  401: #endif
  402: #ifdef USE_ALPHAMAP
  403: 	vAlphaMapUv = ( alphaMapTransform * vec3( ALPHAMAP_UV, 1 ) ).xy;
  404: #endif
  405: #ifdef USE_LIGHTMAP
  406: 	vLightMapUv = ( lightMapTransform * vec3( LIGHTMAP_UV, 1 ) ).xy;

I tried these configurations

Latest Github Repo build:

<postEffectsComposer ref={composerRef} args={[gl]}>
        <postRenderPass args={[scene, camera]} attach={(parent, self) => parent.addPass(self)} />
        <velocityDepthNormalPass
          ref={velocityDepthNormalPassRef}
          args={[scene, camera]}
          attach={(parent, self) => parent.addPass(self)}
        />
        {velocityDepthNormalPassRef.current && composerRef.current && (
          <postEffectPass
            args={[camera, new SSGIEffect(composerRef.current, scene, camera, velocityDepthNormalPassRef.current)]}
            attach={(parent, self) => parent.addPass(self)}
          />
        )}

Previous NPM install 1.1.2:

<postEffectsComposer ref={composerRef} args={[gl]}>
      <postRenderPass args={[scene, camera]} attach={(parent, self) => parent.addPass(self)} />
      <velocityDepthNormalPass
        ref={velocityDepthNormalPassRef}
        args={[scene, camera]}
        attach={(parent, self) => parent.addPass(self)}
      />
      {velocityDepthNormalPassRef.current && (
        <postEffectPass
          args={[camera, new SSGIEffect(scene, camera, velocityDepthNormalPassRef.current)]}
          attach={(parent, self) => parent.addPass(self)}
        />
      )}
  )

One thing i want to note is that i'm able to successfully utilize the following:

{velocityDepthNormalPassRef.current && composerRef.current && (
          <postEffectPass
            args={[
              camera,
              // new HBAOEffect(composerRef.current, camera, scene),
              new MotionBlurEffect(velocityDepthNormalPassRef.current),
              new SharpnessEffect(0.5),
              // new TRAAEffect(scene, camera, velocityDepthNormalPassRef.current),
              (() => {
                const effect = new SMAAEffect({
                  preset: SMAAPreset.MEDIUM,
                  edgeDetectionMode: EdgeDetectionMode.COLOR,
                  predicationMode: PredicationMode.DEPTH,
                });
                if (effect.edgeDetectionMaterial) {
                  const edgeDetectionMaterial = effect.edgeDetectionMaterial;
                  edgeDetectionMaterial.edgeDetectionThreshold = 0.02;
                  edgeDetectionMaterial.predicationThreshold = 0.002;
                  edgeDetectionMaterial.predicationScale = 1;
                }
                return effect;
              })(),
            ]}
            attach={(parent, self) => {
              parent.addPass(self);
              return () => {
              };
            }}
          />
        )}

specifically MotionBlur and SharpnessEffect

When clicking through the sample models on https://realism-effects.vercel.app/, I noted the Swordsman looks wrong around the cape / cutout parts:

I think either transparency or cutout may not be working correctly with SSGI on.

EDIT: even more apparent on "cyber samurai" and can also be seen on "amg" on the headlights, on "motorbike" on the glass parts, on "cyberpunk_bike" on the floor.

I understand there are limitations with depth-based effects and transparency - would be great if those objects would render kind of correct though.

Not exactly sure how to title this issue correctly;
while testing a few more "clean" models to learn more about how realism-effects works I ran into a strange phenomenon:

I encountered that a couple of times after refreshs as well, but I don't have clear reproduction steps unfortunately.
The model was this one:
AmbientOcclusionTest.glb.zip

Since recently, three version was restricted a lot more, is that absolutely required?

I ask this because currently @pmndrs/postprocessing for example only works with r148, but make other libs will benefit from a more loose peerDep?

I tried to test this pr(pmndrs/react-postprocessing#211) for test in react-postprocessing

I found when SSGIEffect instance created, Javascript VM instance has added so I think it is kind of workers but I can't find where the worker created in chrome Mac OS

I have 2 WebGLRenderer(Canvas) in my web.
However, The second one didn't make use of realism effects.
But the env map goes black when realism effects were used in the first canvas.

I've dug a bit and found the patches of ShaderChunk.envmap_physical_pars_fragment
(https://github.com/0beqz/realism-effects/blob/main/src/ssgi/utils/Utils.js)

I also found the SSGIEffect is trying to disable the patch at the end of each frame.
(https://github.com/0beqz/realism-effects/blob/main/src/ssgi/SSGIEffect.js)
But that's not working when another renderer tries to use envmap_physical_pars_fragment along the way.

globalIblRadianceDisabledUniform.value = true

this.rAF = requestAnimationFrame(() => {
	this.rAF2 = requestAnimationFrame(() => {
		globalIblRadianceDisabledUniform.value = false
	})
})

First of all, amazing what you have been doing here! Thank you for all your work here.

I was wondering if there will also be support for this on mobile. I tested with the AO effects and there seem to be issues on my Android device. Do you know a workaround for this?

Hello! I have a problem on my ipad pro with m1 chip when i bring the camera close to an object

Video

Just testing a bit on adjusting texture and material with realism effects SSGI passes
I notice the texture repeat, center, rotation and offset is not working, also in material normal strength, env map internsity also not working

Is there a recommended way to integrate the SSGI with @react-three/fiber? I tried adapting your code like this, but it did not work:

const SSGI = () => {
    const { camera, composer, scene } = useContext( EffectComposerContext )
    const velocityDepthNormalPass = new VelocityDepthNormalPass( scene, camera )
    composer.addPass(velocityDepthNormalPass)
    const ssgiEffect = new SSGIEffect( scene, camera, velocityDepthNormalPass, options )
    const traaEffect = new TRAAEffect( scene, camera, velocityDepthNormalPass )
    const motionBlurEffect = new MotionBlurEffect( velocityDepthNormalPass )
    return <EffectComposer>
        <primitive object={motionBlurEffect}/>
        <primitive object={traaEffect}/>
        <primitive object={ssgiEffect}/>
    </EffectComposer>
}

So helpers like TransformControls crash the render loop (same issue as the old SSR )

and THREE.Lines create black overlay/glitch problems when SSGI is turned on

So if we can use the selection class from postprocessing that would be nice to have eliminate these problems

I tested both locally and online, with your example code, as well as my own working code, and experienced a memory leak: the same model, loaded multiple times, and each load cleared the memory of the previous model, but the memory continued to rise. When I use normal rendering, this problem does not occur. The memory comparison diagram is as follows: The leak points to Realistic-Effects, and I think it may be that the Realistic-Effects rendered real particle geometry memory is not fully cleared

Hey, congrats on this release! I'm excited to give it a quick try at least, but it seems the VelocityDepthNormalPass is currently not being bundled/exported?

Thanks for the great project. I've just started using it and spent a lot of time trying to integrate it with react-three-fiber (Success!). Now, I'm just having fun.

TL;DR

I have found the issue that covers this one while I was writing it. :/ Feel free to close. I still think it's good for future reference if somebody has the same issue.

The related issue is https://github.com/0beqz/realism-effects/blob/eb03a9e4a50bd7f0242bafb77d8e9b0c4c73543b/src/ssgi/utils/Utils.js#L238C24-L238C24

Wasted effort

First of all, I'm using realism-effects with react-three-fiber and in general it works just fine. I discovered an issue though, that is architectural.

VelocityDepthNormalPass does deep replacement of materials in the whole scene. That's quite an intrusive operation as it's impossible to tell if some user of replaced material relies on custom property that is went missing.

In my case it's "color" on one of the handlers of TransformControls. The code tries to clone a color without doing undefined checks.

I've temporary fixed it by adding "color" to

const materialProps = ["vertexTangent", "vertexColors", "vertexAlphas", "vertexUvs", "uvsVertexOnly", "supportsVertexTextures", "instancing", "instancingColor", "side", "flatShading", "skinning", "doubleSided", "flipSided"];

Proxy can be used to fix at least part of property access errors but it is still going to be brittle. In my opinion it's good to implement both approches.

Partly related #14 as it will allow to exclude some of the offending objects in scene graph.

Relevant code

TransformControls:
https://github.com/pmndrs/three-stdlib/blob/4c04593ee49bb0b022025718844f3ce2b21f67bf/src/controls/TransformControls.ts#L1301C40-L1301C40

ssgi/.../Util:
https://github.com/0beqz/realism-effects/blob/eb03a9e4a50bd7f0242bafb77d8e9b0c4c73543b/src/ssgi/utils/Utils.js#L238C24-L238C24

Difficult to setup and integrate into an existing three.js project. Setting up the Offscreen Canvas to use elsewhere is not explained. Also doesn't seem to work at all on mobile because of this?

While the GLTFLoader always guaranteed only single material per mesh.

Multiple materials per mesh are natively supported by three.js.

It'd be nice if we cloud add this support. :)

Changing "color" in the material on runtime won't work.

It seems like propsPrimitive didn't have color property inherited from Material class.
(https://github.com/0beqz/realism-effects/blob/main/src/gbuffer/material/GBufferMaterial.js)

let props = Object.keys(gBufferMaterial)

props = props.filter(
	key => !key.startsWith("_") && !key.startsWith("is") && key !== "uuid" && key !== "type" && key !== "transparent"
)

const propsPrimitive = props.filter(
	key => typeof gBufferMaterial[key] === "string" || typeof gBufferMaterial[key] === "number"
)

There is quite some discrepancy between what's outlined in Usage and what's done in the example page.

Is there a more basic example somewhere that just shows minimal setup, all options at defaults, using a basic EXR skybox and one model?

If not, it would be great to have that – I'm not able to get something fully working from the Usage example code (e.g. no IBL, motion blur / taa don't actually work, ...).

If you run the demo and change the model to cyber_samurai, the whole model turns black within a few seconds.

I am running this on Mac OS 13.5.1 with M2 chip and google chrome 116.0.5845.140. Here is the

In the example, when changing the camera from perspective to orthographic, tracing does not seem to work, and because of this, the light creates a completely different picture, slightly overexposed and without such shadows as with a perspective camera.

// const camera = new THREE.PerspectiveCamera(40, window.innerWidth / window.innerHeight, 0.01, 250)
const frustumSize = 10
const aspect = window.innerWidth / window.innerHeight
const camera = new THREE.OrthographicCamera(
	-frustumSize * aspect,
	frustumSize * aspect,
	frustumSize,
	-frustumSize,
	1,
	1000,
)
scene.add(camera)

Perspective (default example)

Orthographic

Is it possible to achieve a similar result with an orthographic camera as a perspective one? Or is it not supported yet?

Hi @0beqz ,
Thanks for making the realism-effects. SSGI works great with HDRI images.

Does it support three js Lights? I tested it with three js Sunlamp, SSGI does not seems to work.

There seem to be some weird layers on the model? Interestingly, the problem gets resolved only when SMAA is applied.
I'm curious if you're aware of the underlying cause for this occurrence.
Your insights would be greatly appreciated :)

Hey there! Thanks for releasing such an amazing project!!

Are emissive textures supposed to work? They seem to have no effect, even when creating some random messages to test it out.

I'm trying to get a video texture to reflect on the floor like in one of your old examples but nothing seems to pick up light from emissive textures.

I just want to make sure I'm not missing something!

First of all: great work! Thanks.

Perhaps I'm missing something, but would it be possible to make it adhere to renderer.setClearColor() or canvas.style.background?

Seems that without the dome mesh the background is always black:

Setting scene.background to white yields artefacts in the corners:

Hello, my statement is more a question than an issue but i didn't know how to reach you so please, forgive me if you find this entry too much intrusive.

Basically, i can't understand the current status of your SSR effect. I mean, i've seen the other repository which at the moment seems to be deprecated due to an incompatibility with the last version of three.js.

At the moment, is the SSR effect implemented in you SSGI code? I've seen the SSR quote in the realism-effects description page but there is no description and so for me is a little bit misleading...

Anyway, thanks for your work, i really appreciate it!

Here is a minimal example.
https://codepen.io/benzsuankularb/pen/WNLzMyg

Basically, I've setup 2 canvas
One with Realism-Effect and one with normal MeshStandardMaterial.
If both canvases are enabled then the env map in MeshStandardMaterial won't work.
Both canvases didn't share any resources on the application layer.

I'm not sure if this is some issue on the Three / WebGL layer.
Or there are some modifications on the Realism-Effect applied to the MeshStandardMaterial shader.

continuing this thread from twitter https://twitter.com/vis_prime/status/1632602124269023233?s=20

I'm noticing trails being formed which does not happen on the demo page

Hello, I'm having an issue setting up the AO (both HBAO and SSAO) with varying scene sizes.

In this first example, I have a pretty standard setup with the HBAO effect. Everything works as expected.

When I now introduce a scalingFactor that scales all elements in the scene as in this second example, the results of the AO are not consistent. I tried to adjust the distance, thickness and radius by the scalingFactor with no success.

Is there a way to achieve consistent AO results depending on the scene scale?

If "NORMAL_IN_RGB" is not defined, the following line in "poissionDenoise.frag" causes a compilation error:

float center = texel.rgb;

For a solution see #21

See: https://twitter.com/edankwan/status/1659655493302796288?t=pWtM2sTx9CpX_W_mYfmvQg&s=19

Although, am not using any environment map for the scene, I quite love the way SSGI lifts up the scene. Is there a way to avoid the trailing effect at longer distance?

Am using this settings as option

options = {
  "distance": 1.5,
  "thickness": 20,
  "autoThickness": false,
  "maxRoughness": 1,
  "blend": 0.95,
  "denoiseIterations": 1,
  "denoiseKernel": 3,
  "denoiseDiffuse": 20,
  "denoiseSpecular": 0,
  "depthPhi": 14.022,
  "normalPhi": 0,
  "roughnessPhi": 0,
  "envBlur": 0.55,
  "importanceSampling": true,
  "directLightMultiplier": 1,
  "maxEnvLuminance": 50,
  "steps": 8,
  "refineSteps": 1,
  "spp": 3,
  "resolutionScale": 2,
  "missedRays": false
};

I used GPUComputationRenderer to compute the position and velocity for particles and save them as texture, and pass them to instancedMesh for rendering.

<instancedMesh>
       <boxGeometry args={[0.2, 0.2, 0.6]}>
               <instancedBufferAttribute attach="attributes-uvs" args={[uvs, 3]} />
        </boxGeometry>
        <ThreeCustomShaderMaterial/>
</instancedMesh>

In the vertex shader of the material for instancedMesh, I did something like this:

 void main() {
         vec3 pos = texture2D(positionTex, uvs.xy).xyz;
         csm_PositionRaw =  projectionMatrix * modelViewMatrix * vec4( position + pos, 1.0 );
},

When I apply the SSGI, all particles become bunch of boxes in the origin, and the positions I modified with the positionTex are not correctly reflected.
Is it some kind of the limit of SSGI?

Using OffscreenCanvas seems to result in issues when using Safari, so it should maybe be removed from the demo.

Hi,
First-of-all, thanks for this great library of effects! I'd like to try-out some recent realism-effects which are not yet available in the module version published to npm 1.1.2. I see you have deployments set-up for production/preview - which might just be hooks to vercel, and publish up-to-date demo pages after any commit.

Could you also push the updated realism-effects build to npm within the same vercel deployment task? Or set-up a Github Action to do so, eventually on versioned? Would be great to test new featues without building the lib ourselves.

I've noticed high heat spot artifacts when perfect reflection reflects each other.