Enva-XR - Environment Aware Augmented Reality

Web-based framework for environment-aware rendering and interaction in augmented reality based on WebXR using three.js
The framework handles geometry occlusions, matches the lighting of the environment, casts shadows, and provides physics interaction with real-world objects.
Material from three.js can be reused the shader code required for AR occlusion is injected into the existing shaders using the onBeforeCompile callback.
Capable of obtaining over 20 frames per second even on middle-range devices.
WebXR AR features supported by the framework

Getting Started

Download the repository from git gh repo clone tentone/enva-xr
Install Node and NPM.
Ensure that your browser is compatible with WebXR and check features support (e.g. depth-estimation, hit-test, lighting).
Install dependencies from NPM by running npm install and start the code running npm run start
To allow easier debugging in mobile devices eruda can be used as a development tools alternative.

Usage Example

Bellow is a simple usage example of the library the ARRenderer is responsible for most of the work required to setup the AR scene.
The ARRenderer receives a configuration object that indicates wich WebXR features should be enabled.
To enable AR rendering on existing three.js materials the AugmentedMaterial.transform() method should be used to transform regular materials into AR materials.
The example bellow demonstrates how to create a simple AR scene with occlusion and lighting enabled.
LightProbe object replicates the envornment lighting and position main light source position and direction. Internaly contains a three.js LightProbe and DirectionalLight with shadow casting enabled by default.

const renderer = new ARRenderer({
  depthSensing: true,
  depthTexture: true,
  lightProbe: true
});

let material: any = new MeshPhysicalMaterial({color: (Math.random() * 0xFFFFFF)});
material = AugmentedMaterial.transform(material);

let box = new Mesh(new BoxGeometry(), material);
box.receiveShadow = true;
box.castShadow = true;
renderer.scene.add(box);

const probe = new LightProbe();
renderer.scene.add(probe);

const floor = new FloorPlane();
renderer.scene.add(floor);

renderer.onFrame = function(time: number, renderer: ARRenderer) {
  box.rotation.x += 0.01;
};

renderer.start();

Rendering

Depth data provided by WebXR can be used for occlusion in the 3D scene.
Occlusion is calculated in the shader code injected using the AugmentedMaterial.transform() method.
To enable realistic rendering of the scene the MeshPhysicalMaterial material should be used alonside PBR assets.

Physics

cannon.js can be used for physics interaction between objects.
The VoxelEnvironment provides a probabilistic voxel based model that maps the environment from depth data that is updated every frame.
Alternativelly physics can rely on plane detection using the FloorPlane or Planes objects.
Currently performance is limited might be improved using WebXR Real World Geometry API

License

The code from the project is MIT licensed. The license is available on the project repository,

tentone / enva-xr Goto Github PK

enva-xr's Introduction

Enva-XR - Environment Aware Augmented Reality

Getting Started

Usage Example

Rendering

Physics

License

enva-xr's People

Contributors

Stargazers

Watchers

Forkers

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent