Giter Club home page Giter Club logo

codelab-webxr's Introduction

Building an augmented reality application with the WebXR Device API

This code has the resources you need for the codelab Building an augmented reality application with the WebXR Device API.

This is a work in progress. If you find a mistake or have a suggestion, please file an issue. Thanks!

What you'll learn

  • How to use the WebXR Device API
  • How to find a surface using augmented reality hit tests
  • How to load and render a 3D model synchronized with the real world camera feed

What you'll need

codelab-webxr's People

Contributors

devbridie avatar jsantell avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

codelab-webxr's Issues

Doesn't work !!! Chrome Canary/Dev v72 with Android 9

Android device: Pixel 2 XL with Android 9

ARCore 1.9.190422066

I have installed many version of Chrome Canary/Dev v70-73, and opened the webxr flags, but none work.

It show "unsupported browsers". I found that "xr" attribute doesn't exist on navigator object.

If anyone have a resolution or know the Chrome version which could run the samples, please tell me! Thank you!

Code not matching tutorial

Hello, I've done extensive testing and found the following: on step 4, part 4 of Add a reticle the following code is on the tutorial:

async onSessionStarted() {
  // ...

  // Setup an XRReferenceSpace using the "local" coordinate system.
  this.localReferenceSpace = await **_session_**.requestReferenceSpace("local");

  // Add these lines:
  // Create another XRReferenceSpace that has the viewer as the origin.
  this.viewerSpace = await this.**_session_**.requestReferenceSpace("viewer");
  // Perform hit testing using the viewer as origin.
  this.hitTestSource = await this.**_session_**.requestHitTestSource({ space: this.viewerSpace });

  // ...
}

and it doesn't work, it throws the onNoXRDevice exception.

The functional code, found in step-04 folder is the following:

// Setup an XRReferenceSpace using the "local" coordinate system.
this.localReferenceSpace = await this.**_xrSession_**.requestReferenceSpace('local');

// Create another XRReferenceSpace that has the viewer as the origin.
this.viewerSpace = await this.**_xrSession_**.requestReferenceSpace('viewer');
// Perform hit testing using the viewer as origin.
this.hitTestSource = await this.**_xrSession_**.requestHitTestSource({ space: this.viewerSpace });

How to implement animated 3D objects? (fbx or gtfl)

Maybe here is a better place for my problem:
(immersive-web/webxr#457)

I need your help. I have done the webxr tutorial with enthusiasm. (https://codelabs.developers.google.com/codelabs/ar-with-webxr/#0)

Now I would like to display animated objects. With blender I animated the object in fbx format and loaded it into an html with the FBXLoader of the three.js library. That works!
Unfortunately I did not succeed to use the fbx object with the same FBXLoader in the AR example of the tutorial.

Is there an example for this? Is gtlf object better usable than fbx objects? What needs to be changed in the code so I can use the loader?

Thanks a lot!

three.js FBX Loader:


var loader = new THREE.FBXLoader();
loader.load( 'models/fbx/AR_Tutorial.fbx', function ( object ) {

	mixer = new THREE.AnimationMixer( object );
        var action = mixer.clipAction( object.animations[ 0 ] );
	action.play();

	object.traverse( function ( child ) {

		if ( child.isMesh ) {

		       child.castShadow = true;
		       child.receiveShadow = true;

		 }

		} );

		scene.add( object );

	  } );

New

if let hiResCaptureVideoFormat = ARWorldTrackingConfiguration.recommendedVideoFormatForHighResolutionFrameCapturing {
// Assign the video format that supports hi-res capturing.
config.videoFormat = hiResCaptureVideoFormat
}
// Run the session.
session.run(config)

Only head-model hit testing is supported

Hey,
I'm trying to implement hit-testing like it's showing here. In the onSessionStarted function I set the frame of reference, but I'm forced to do this in 'head-model'. When I try to set it to 'eye-level' I get this error: Unhandled Promise Rejection: Only head-model hit testing is supported

This is my onSessionStarted function:

onSessionStarted = async () => {
        this.setState({ isARSessionStarted: true, arMessage: 'session started' });

    this.renderer = new THREE.WebGLRenderer({
        alpha: true,
        preserveDrawingBuffer: true,
    });
    this.scene = new THREE.Scene();

    this.renderer.autoClear = false;
    this.renderer.shadowMap.enabled = true;
    this.renderer.shadowMap.type = THREE.PCFSoftShadowMap;

    this.gl = this.renderer.getContext();

    await this.gl.setCompatibleXRDevice(this.session.device);

   
   
    this.session.baseLayer = new window.XRWebGLLayer(this.session, this.gl);

    const framebuffer = this.session.baseLayer.framebuffer;
    this.renderer.setFramebuffer(framebuffer);       
    
     // this.camera.matrixAutoUpdate = false;
    //  this.camera.position.z = 100;
    this.camera = new THREE.PerspectiveCamera();
    

    this.frameOfRef = await this.session.requestFrameOfReference('eye-level');
    this.session.requestAnimationFrame(this.onXRFrame);

   

};`

So the behavior of my app is strange. When I want to add an object on the surface, it instead "sticks on the display" and moves with the camera. Is this caused by 'head-model'? Is there a way to fix this?
I am on iOS in the XRViewer Browser.
Thanks!

Animation Questions

Hi there! I've been trying my best to integrate the animation code into my project. But it still does not work.

I added this.animate(); to the top of my code - under this.init - TBH I dont know if the 'this.' should be there? But I'm guessing it should... I read the article provided on async and promises, but I still don't understand how exactly it relates to integrating animation in my project.

Then under this.scene.add(this.model); I added -
async animate (); {
requestAnimationFrame( animate );
this.model.rotation.y+=1;
this.renderer.render(this.scene, thiscamera ); //But this already happens in 'if (pose)' and
//if I delete it from there, the app stops working.
}

Last question : I know that Aframe has a slack channel that specifically addresses these types of issues for noobs like myself. and also allows the community of outsiders to maybe work together.. Do you have one as well? Or - will you be implementing in the future?

Thanks so much!

Chrome unexpectedly exits after click "START AUGUMENTED REALITY"

When starting any of the demos, Chrome Canary unexpectedly exits after click the button "START AUGUMENTED REALITY".
image
Device: Samsung S7 (SM-G9308)
Android version 8.0.0
Chrome Canary version 73.0.3636.2。
ARCore version 1.5.180910133
I have turned on the #webxr and #webxr-hit-test flags and nothing else.

If I removed the option environmentIntegration: true for requestSession(), Chrome will render the page without camera stream.

Any idea what causing the problem?

How to remove the background in AR mode?

While projecting my phone screen on my glasses, I dont want to see the camera streaming on top off the real world, so
is it possible to remove the camera streaming on the screen while AR mode is still working?

Camera is not working properly in Canary 72

Camera is not working properly after updating Canary to version 72.0.3589.0 from 70 in Pixel2 Android 9.

Portrait Mode: Screen blinks different colors after clicking the button Start Augmented Reality.

Landscape Mode: Camera detects the surrounding but displays at different angle.

Everything was working fine in Canary 70.0.3538.2 (https://youtu.be/dXODcxYQMtM).

Doesn't Work !!! chrome(canary/dev) v72 with Oreo

I have two Android device (one is the latest phone(Samsung), another one is Nexus5x with Oreo)

I have to install older version of chrome because of this example limitation.

Finally, Nothing is possible to work....

I lost too much valuable time.

Changing the sunflower to another gltf

I had a gltf, and I changed the link here:

window.gltfLoader.load("https://immersive-web.github.io/webxr-samples/media/gltf/sunflower/sunflower.gltf", function(gltf) {
  const flower = gltf.scene.children.find(c => c.name === 'sunflower')
  flower.castShadow = true;
  window.sunflower = gltf.scene;
});

to something like http://mysite.com/mymodel.gltf and it is not working. I have no idea how I can change the sunflower properly. Now, I can successfully instantiate nothing on the screen 😂

Needed, Chrome Build

@jsantell,

Before the API is available on Canary, where can I get the build that was available for the IO attendees?
Thanks.

Trying to rotate obj

Hi! I am pretty new to three.js and google ar and I have been trying to find a way to animate the obj's that I import into the final example. I have mainly been looking at three.js tutorials on how to do this, however none of the tutorials I view have quite the same syntax as was used in the tutorial making it a little hard for me to translate the code to fit my needs. Are there any other resources to learn more about how rotate and translate the objs on the reticle? I know that I cannot export the animation already on the obj, as obj's do not support that. I am trying to use the code to animate.

Thanks so much!

"Unsupported Browser" message on Canary 72, Pixel XL, Android 9

Hi,
Although the phone I am using fullfills all requirements, I get the "unsupported Browser" screen when opening the test website. I have experimented with AR apps before (eg. three.ar.js via the older ARCore version), so it should in principle work.

  • Chrome Canary 72.0.3604.0
  • Android 9, Pixel XL, Build/PPR2.181005.003

Get "Installing AR module" toast and then nothing

Using Chrome Canary 72.0.3626.0, Android 9, Pixel 2. ARCore is installed and works with native apps. Canary has camera permission (verified with a getUserMedia demo). webxr and webxr-hit-test flags are enabled.

Visiting the code lab demo link: https://googlecodelabs.github.io/ar-with-webxr/final

On pressing the "Start Augmented Reality" button, I see a toast saying "Installing AR Module..." and then nothing else. Tried re-installing, re-tapping the link, force closing, reinstalling ARCore. No dice.

Is it just me?

Examples not working on Google Chrome 81 and Android 10

Hi , I'm getting the browser not unsupported message when trying the examples.Tried on chrome version 81.0.4044.138. android 10.

chrome://version

Google Chrome 81.0.4044.138 (Official Build) (32-bit)
Revision 8c6c7ba89cc9453625af54f11fd83179e23450fa-refs/branch-heads/4044@{#999}
OS Android 10; Pixel 2 XL Build/QQ2A.200405.005

WebXR in current canary

It's a question, sorry. Don't know where is the right place for that. WebXR looks very potential but I can not test it with any official available version of Canary. Is there a (serious) place where I find a working (72) version? The more relevant question is, when will current versions of Canary works for available WebXR Demos (https://web-education-ar-demo.appspot.com/). Maybe that's a stupid question, but is there any information online about time schedules for that?

Thanks, Matthias

404 page

I'm getting an Error 404 page when trying to access the codelabs website.

Not working on Samsung S7 on local server and total lagging on published demo

Hello,
After I saw the demo at https://web-education-ar-demo.appspot.com/ . Very cool and smooth.

I started digging around the browser-AR tech and found this repository and looks like a good starting point for me.

I followed the tutorial at https://codelabs.developers.google.com/codelabs/ar-with-webxr/#0 and I successfully served your app via the Web Server Chrome extension but if I visit the page using my smartphone it says that the browser is not supported.

If I visit your demo at https://googlecodelabs.github.io/ar-with-webxr/final/ it works. It lag a lot but at least it works.

From Canary I get these info:
App version: 69.0.3469.0
OS version: Android 8.0.0, SM-G930F Build/R16NW

Working demo screenshot: https://photos.app.goo.gl/PUnAkHRipBr9jUfb7

Not Working demo screenshot: https://photos.app.goo.gl/RNe9hbXZPzGEnnBE7

Can't make example work with Web Server for Chrome

Hi,
Thanks for the great tutorial.
I have setup Canary the way you describe in the tutorial. I then tried to host the website using Web Server for Chrome, went into work, but the webpage doesn't seem to load properly and it says "Unsupported Browser".
But apparently if I visit https://googlecodelabs.github.io/ar-with-webxr/final/ from the same device and same browser, I can access the AR mode.
I have briefly compared the app.js in the two folders but I cannot seem to find any visible difference. I have also tried to debug the webpage on Android using Chrome Dev tools and found out that at line 40 in app.js the navigator.xr is not defined (just on the website I serve with Web Server for Chrome, your final link works fine).

I am not sure what that means.

Am I missing something obvious or there is something odd about it?
Thanks in advance

Chrome stops feeding camera stream and shows gray background

Here is a video showing a problem I have with these demos:
https://youtu.be/htXAzbYrAkU

When starting any of the demos the background of the webpage turns gray and camera live feed stops. The tracking seems to still work when moving the phone.
screenshot_20180725-150143_chrome canary phone

Device: Samsung S8 (SM-G950F) and Samsung S7 Edge
Both devices have:

  • Android version 8.0.0
  • Chrome Canary version 70.0.3501.0
  • ARCore version 1.3.180604066
  • I have turned on the #webxr and #webxr-hit-test flags and nothing else.

The "phone-ar-hit-test.html" and "phone-ar.html" demos work nicely on the immersive-web github page:
https://github.com/immersive-web/webxr-samples/tree/master/proposals
Also the Chacmool demo works at: https://web-education-ar-demo.appspot.com/

Any idea what is causing the problem?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.