Giter Club home page Giter Club logo

threear's Introduction

THREE AR

Introduction

THREE AR aims to provide a simple to use Augmented Reality API for the web, leveraging three.js's 3D capabilities. THREE AR is written in TypeScript. You can see a description of the aims of THREE AR here.

Usage

THREE AR works with npm and also as a CDN. For npm you can install in the following manner:

npm install threear

using a CDN you can include a script tag from unpkg like so:

<script src="https://unpkg.com/threear"></script>

Demo

The examples folder features a basic demo, you can open it on a mobile device and point your camera at the following marker:

Hiro Marker

Documentation and Examples

Development and Contribution

Please see the development guide. If you are interested in contributing, it may be a good starting point to see the list of open issues on our GitHub issues page. Please take a moment to read the code of conduct.

Acknowledgements

  • ar.js on which this library is based
  • jsartoolkit the Emscripten library from the C code for artoolkit, on which this library is underpinned

License

All files within the src/artoolkit folder are licensed LGPLv3

All other files are MIT Licensed and are adapted from the work of Jerome Etienne

threear's People

Contributors

danilopeixoto avatar dependabot[bot] avatar dmbfm avatar gins3000 avatar jameslmilner avatar martinloretzzz avatar zahratee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

threear's Issues

Run ARToolkit in a Worker?

Assuming the data transfer times are low and that ARToolkit.detectMarker doesn't touch the DOM or related APIs, there's no reason we could put it into a worker.

Determine any unused code

It is very plausible some of the code in these files will be entirely unused and can probably be deleted. The code was ported from ar.js and it was hard to determine what was relevant and what was not.

Issue with the examples.

I use my android cellphone to test the examples. Only the basic.html works, for the others I got 'Uncaught ExistStatus' Exceptions.

Z-fighting bug

Same bug as in Ar.js, solved it by myself see:
jeromeetienne/AR.js#146 (comment)

THREEAR.initialize({ source: source }).then(controller => {
    // ...
    let m = camera.projectionMatrix;
    let far = 1000;
    let near = 0.1;

    m.elements[10] = -(far + near) / (far - near);
    m.elements[14] = -(2 * far * near) / (far - near);
    // ...
});

Handle flickering

It would be nice to prevent markers from flickering on and off when they are detected/not detected in short intervals (say, sub second).

Location based AR support

Location based tracking with gps position and device orientation

Likely it will only be usefull if the target position is further away than 25-50m, but it opens a lot of new possibilitys like some navigation or geosearch apps

There is an experiment in ar.js:
https://github.com/ARjs-org/location-based-ar

I will make some experiments next week

Setup CI

At the moment we don't have any tests, but we should at least determine the build works and the linter passes.

Basic usage guide

Alongside TypeDoc documentation and examples we should have some short guide that explains how to do basic operations like setup a marker.

Low quality rendering

Hi, nice adaptation from AR.js! Great improvement to be able to run in a div, turn on/off, etc. More modular.

An issue I had with AR.js and still with this library, is that, using this library, the rendering of my objects becomes really low quality. I haven't been able to figure out how to tweak it.

This is the object in plain THREE.js:
Screenshot from 2019-05-01 20-49-48

and this is the same object in the scene/rendering setup from ThreeAR:
Screenshot from 2019-05-01 20-45-20

I use the setup code from the Basic Example

I also noticed that, if I render using my own PerspectiveCamera with size of the container, the low quality rendering is resolved. But then, of course, the scaling, aspect ratio and positioning is off.

I can imagine it has something to do with rendering size, but I haven't figured it out yet. Any help or fix would be much appreciated!

Is it possible to use perspective camera?

currenty the library dosen't work with different size video source. Given a portait video source(height>width), the 3D object gets stretched. When I switch to perspective camera, the stretching issue gone, but the 3D object dosen't perfectly attatch to the marker. Is it possible to use perspective camera in the first place and aslo attatch to the marker correctly ?

Ability to change facingMode, Source.ts

Hello, thanks for this plugins !!

Could be great to have to ability to change the facingMode with a new parameter in Source.ts

Selfie is popular :)

Thanks !

Build static version of jsARtoolkit

Im trying to make THREEAR work in a webapp enviroment which disables eval() and new Function() syntax.
Is it possible to build a static version of jsARToolkit with NO_DYNAMIC_EXECUTION=1 flag?
I successfully compiled the jsARToolkit , but could not make it integrate with THREEAR.

Exclude three.js from libary

threearThreejs

Would be nice if we exclude three.js from threear, so both libarys could be imported independently. [What I need for an other project with a big webpack config]
Even the examples include both libs seperately

Remove Marker

Hello,
There a way to remove a marker ?

Once added, maker is always tracked. Even if i "dispose" the controller.
I need to track different marker, but one at the time.

For the moment i have hack your code with a "canBeSearch" flag, but maybe there are a better way ?

Thanks !

Bundle in camera_para?

At the moment you have to pass in an explicit URL. I think it would make sense to make it bundle in by default and then you can explicitly override it with a URL?

Should the API always prefix with AR?

At the moment everything is prefixed with AR, i.e. ARSource but when we use the library it is leverage like THREEAR.ARSource so perhaps the AR prefix is unnecessary?

Aframe?

A lot of the popularity of AR.js comes from being compatible with Aframe, which is even simpler than using three.js. Are you planning / thinking about supporting this as well?

NFT support

Hi, I am the lead maintainer of AR.js, I saw your project time ago, great great job!
I have a question: if you have time to give a look to this issue (jeromeetienne/AR.js#544): it is an attempt to bring NFT feature on AR.js (but it can also, obviously, be brought to THIS library of yours) using the NFT branch of jsartoolkit. Such branch is outdated, but it seems to work some time ago.

What do you think about that?
Did you have already done some experiments using other jsartoolkit version, like this one for example?

Thanks

Out of Memory FirefoxMobile

I get an out of memory exeption in FF Mobile on a Xiomi Mi8[MIUI10.2, Android] on all online examples
But Chrome Mobile works

Err

Error when using with React

Hey
So i am trying to make this work with my React and react-three-fiber project
i followed the repo you mentioned on #76 as here

I get the following console logs

Allocated videoFrameSize 1228800
Pattern detection mode set to 1.
Pattern ratio size set to 0.500000.

And then I am getting the following error
Uncaught TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLImageElement or SVGImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)'

I'm not sure where im wrong

More then one Marker/ Multi-Marker?

  • Is there an option to use more then one marker in this package?
    As example for the case: If marker one is detected, funtion one would be activated. If marker two is detected, function two would be activated. If both markers are detected, both option would be activated.
  • Is there an option to use a multi-marker (several markers acting as one)?

Basic browser testing

It would be good to test the library in an actual browser. I'm not sure how realistic this is in as we'd probably need to mock getUserMedia and some other things, but it could be quite powerful in detecting that the code loads without errors as a minimum and potentially regressions we got it working.

Lost markers are not always detected

Firstly, thank you for making this library. It is much easier to use and read, especially for an Angular project like what I'm working on.

We are having an issue with THREEAR where our barcode markers being reported as found long after they have been removed from the image. This happens when a single marker is being found and then the camera is moved off of it. The lostTimeout does not seem to change this behavior.

I believe I have found the reason for this. In ARToolKitController.ts when a frame is processed, if ARToolKit detects no markers the for loop is never gone through so the getMarker event is not emitted and thus the part of handleMarkerDetection() in Controller.ts that checks for lost markers isn't executed. This means that a marker will continue to be found until another marker or something that ARToolKit thinks is a marker at any level of certainty enters the frame.

The lost marker detection should probably be run on its own event that occurs once at the end of the process() function each frame.

Build our own version of ARToolkit

It might be possible to get a more optimal build that exports to an ES6 Module if we build our own verison of ARToolkit, potentially stripping out all the Node specific code too.

Create / Dispose doesn't work second time

Hello,

I try to dispose and recreate but it doesn't work.
I try with your example. But it's the same.

Wen we create, the Hiro marker is detected, i dispose it, recreate. And the hiro is no more detected.

Thanks !

marketLost event

Hello , thanks for closing all my issues/demand you rock !!

One more for you :)

We have markerFound event, but i don't see any markerLost ?
Something like that exist ?

Thanks !

Keep a changelog

Keeping a changelog will allow us to keep track of whats happened between releases. I am considering using https://www.npmjs.com/package/auto-changelog. Ideally I would like to use semantic versioning going forward. At the moment I am aiming to solidify the API so that we don't have to break major bump all the time in the future.

Disable aspect ratio

Hello.

We need to put video into the canvas to take a screenshot via canvas.toDataUrl('png')

For anyone we simply scene.background = new THREE.VideoTexture(source.domElement)

But when we shoot the canvas, the width and height is is the same than what we see.
You use a bigger canvas than the screen, and re-center it with margin-left or margin-top.

Can you add a param to disable the keep aspect ratio ?

Thanks again for this great work !

Destroy options.

I use it with in a SPA.

Could be great to add the possibility to destroy the sources and controller .

Thanks !

Allow camera source to work at different element sizes

At the moment the video source works completely on the basis that the element is full screen. This is a pretty large assumption and it would be nice to be able to use a different sized container (i.e. half screen for example)

running three AR using parcel

Hi, I'm attempting to run the basic example using parcel... this is my javascript file

`import * as THREE from "three";
//import fragment from "./shader/fragment.glsl";
//import vertex from "./shader/vertex.glsl";
let THREEAR = require("threear");
let OrbitControls = require("three-orbit-controls")(THREE);

export default class Sketch {
constructor(options) {
var renderer = new THREE.WebGLRenderer({
// antialias : true,
alpha: true
});
renderer.setClearColor(new THREE.Color('lightgrey'), 0)
// renderer.setPixelRatio( 2 );
renderer.setSize( window.innerWidth, window.innerHeight );
renderer.domElement.style.position = 'absolute'
renderer.domElement.style.top = '0px'
renderer.domElement.style.left = '0px'
document.body.appendChild( renderer.domElement );

// init scene and camera
var scene = new THREE.Scene();
var camera = new THREE.Camera();
scene.add(camera);

var markerGroup = new THREE.Group();
scene.add(markerGroup);

var source = new THREEAR.Source({ renderer, camera });

THREEAR.initialize({ source: source, lostTimeout: 500 }).then((controller) => {

    // add a torus knot		
    var geometry = new THREE.TorusKnotGeometry(0.3,0.1,64,16);
    var material = new THREE.MeshNormalMaterial(); 
    var torus = new THREE.Mesh( geometry, material );
    torus.position.y = 0.5
    markerGroup.add(torus);

    var geometry = new THREE.CubeGeometry(1,1,1);
    var material = new THREE.MeshNormalMaterial({
        transparent : true,
        opacity: 0.5,
        side: THREE.DoubleSide
    }); 
    var cube = new THREE.Mesh( geometry, material );
    cube.position.y	= geometry.parameters.height / 2;
    markerGroup.add(cube)

    var patternMarker = new THREEAR.PatternMarker({
        patternUrl: 'patt.hiro',
        markerObject: markerGroup,
        minConfidence: 0.3
    });

    controller.trackMarker(patternMarker);

    controller.addEventListener('markerFound', function(event) {
        console.log('markerFound', event);
    });

    controller.addEventListener('markerLost', function(event) {
        console.log('markerLost', event);
    });

    // run the rendering loop
    var lastTimeMsec = 0;
    requestAnimationFrame(function animate(nowMsec){
        // keep looping
        requestAnimationFrame( animate );
        // measure time
        lastTimeMsec = lastTimeMsec || nowMsec-1000/60;
        var deltaMsec = Math.min(200, nowMsec - lastTimeMsec);
        lastTimeMsec = nowMsec;
        // call each update function
        controller.update( source.domElement );
        // cube.rotation.x += deltaMsec/10000 * Math.PI
        torus.rotation.y += deltaMsec/1000 * Math.PI
        torus.rotation.z += deltaMsec/1000 * Math.PI
        renderer.render( scene, camera );
    });

    

});

}
}
new Sketch({
dom: document.getElementById("container")
});
`
I'm getting this error...

Allocated videoFrameSize 1228800
arjs.artoolkit.min.js:29 Pattern detection mode set to 1.
arjs.artoolkit.min.js:29 Pattern ratio size set to 0.500000.
arjs.artoolkit.min.js:29 Pattern Data read error!!
arjs.artoolkit.min.js:29 loadMarker(): Error loading pattern file /marker_0.
arjs.artoolkit.min.js:29 ARToolKitJS(): Unable to set up AR marker.
arjs.artoolkit.min.js:29 Out of memory!!
arjs.artoolkit.min.js:74627 Uncaught
ExitStatus
message: "Program terminated with exit(1)"
name: "ExitStatus"
status: 1
proto: Error at Object. (http://localhost:60728/newapp.5ac9656d.js:70317:767184) at r (http://localhost:60728/newapp.5ac9656d.js:70317:406) at Object. (http://localhost:60728/newapp.5ac9656d.js:70317:1300) at r (http://localhost:60728/newapp.5ac9656d.js:70317:406) at Object. (http://localhost:60728/newapp.5ac9656d.js:70317:15261) at r (http://localhost:60728/newapp.5ac9656d.js:70317:406) at Object. (http://localhost:60728/newapp.5ac9656d.js:70317:8584) at r (http://localhost:60728/newapp.5ac9656d.js:70317:406) at http://localhost:60728/newapp.5ac9656d.js:70317:1205 at http://localhost:60728/newapp.5ac9656d.js:70317:1214
constructor: ƒ ExitStatus(e)
stack: "Error↵ at Object. (http://localhost:60728/newapp.5ac9656d.js:70317:767184)↵ at r (http://localhost:60728/newapp.5ac9656d.js:70317:406)↵ at Object. (http://localhost:60728/newapp.5ac9656d.js:70317:1300)↵ at r (http://localhost:60728/newapp.5ac9656d.js:70317:406)↵ at Object. (http://localhost:60728/newapp.5ac9656d.js:70317:15261)↵ at r (http://localhost:60728/newapp.5ac9656d.js:70317:406)↵ at Object. (http://localhost:60728/newapp.5ac9656d.js:70317:8584)↵ at r (http://localhost:60728/newapp.5ac9656d.js:70317:406)↵ at http://localhost:60728/newapp.5ac9656d.js:70317:1205↵ at http://localhost:60728/newapp.5ac9656d.js:70317:1214"
proto:
constructor: ƒ Error()
message: ""
name: "Error"
toString: ƒ toString()
proto: Object

any help or suggestion would be appreciated.
Thanks!

Marker Size parameters

Is there any way I can assign marker size. Currently, a cube of 1mm length is fitting to the marker, so the marker size is considered as 1 unit. Can I change it to 0.1 units? So that the entire scene will be relatively smaller w.r.t. the marker in the AR scene. There was a parameter named 'size' in ThreeARtoolkit but I could not figure that out in this library.

Also, how to transform the origin where the camera is pointing at... currently its the origin (0,0,0) which is placed on to the marker. I sthere any way I can make another (x,y,z) point on top of the marker? I am using 'cameraTransformMatrix' as changeMatrixMode.

Example

Can not import THREEAR as a module

Hi @JamesMilnerUK i m testing your lib with React, maybe not the simple approach but was wondering that maybe is easier that others. i have this setup:

  1. i created a simple app with create-react-app command
  2. i have installed the packages with npm install three threear
  3. inside App.js i have this code:
import React from 'react';
import './App.css';
const THREE = require('three');
const AR = require('threear');

class App extends React.Component {

  render(){
    var scene = new THREE.Scene();
    var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.1, 1000 );
    var geometry = new THREE.BoxGeometry( 1, 1, 1 );
    var material = new THREE.MeshBasicMaterial( { color: 0x00ff00 } );
    var cube = new THREE.Mesh( geometry, material );
    scene.add( cube );

    camera.position.z = 5;

    var renderer = new THREE.WebGLRenderer();
    renderer.setSize( window.innerWidth, window.innerHeight );
    document.body.appendChild( renderer.domElement );

    function animate() {
	     requestAnimationFrame( animate );
	      renderer.render( scene, camera );
        cube.rotation.x += 0.01;
        cube.rotation.y += 0.01;
      }
    animate();
  return(
  <div></div>
  )
  }
}

export default App;

the Three code works fine If i uncomment the AR require, but with this i get:

/node_modules/threear/dist/THREEAR.js
Module not found: Can't resolve 'THREE' in '/home/walter/kalwalt-github/react-ARjs-experiments/react-threear-example/node_modules/threear/dist'
console.<computed> @ index.js:1375
handleErrors @ webpackHotDevClient.js:151
push../node_modules/react-dev-utils/webpackHotDevClient.js.connection.onmessage @ webpackHotDevClient.js:194
push../node_modules/sockjs-client/lib/event/eventtarget.js.EventTarget.dispatchEvent @ eventtarget.js:56
(anonymous) @ main.js:282
push../node_modules/sockjs-client/lib/main.js.SockJS._transportMessage @ main.js:280
push../node_modules/sockjs-client/lib/event/emitter.js.EventEmitter.emit @ emitter.js:53
WebSocketTransport.ws.onmessage @ websocket.js:36

Have you a working example with Node at least?

Type all the things!

At the moment most of the source code is typed as any which is a start but not particularly useful. Some of the types might require some runtime inspection because they are coming from the vendorered artoolkit library.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.