niklever / canvasui Goto Github PK

View Code? Open in Web Editor NEW

127.0 127.0 22.0 1.44 MB

A Three.JS WebXR UI. Enabling easy UI creation for immersive-vr sessions.

Home Page: canvas-ui-eight.vercel.app

License: GNU General Public License v3.0

JavaScript 100.00%

canvasui's People

Contributors

Stargazers

Watchers

canvasui's Issues

hi~ i'm starting learning three.js and my English is poor, /(ㄒoㄒ)/~~ i hope you can understand my question.
your work is great,and i'm trying to use it, and i click your online version but it seemed don't work,
the result is that:

so i download your code but the result is the same,and it shows that

how to solve them~~~thank you so much

add build process

$ npm run build
> [email protected] build
> node node_modules/@eviltik/devsrv/bin/devsrv.js -b && node node_modules/rollup/dist/bin/rollup -c ./rollup.config.js

Sat, 05 Mar 2022 11:57:59 GMT devsrv - info: use config file ..GIT\CanvasUI\devsrv.config.js
Sat, 05 Mar 2022 11:57:59 GMT devsrv - info: build success

C:\Users\T\Desktop\DATA\DEV\GIT\CanvasUI\dist\examples\jsm_src\CanvasUI.js => dist/examples/js/CanvasUI.js...
created dist/examples/js/CanvasUI.js in 929ms

C:\Users\T\Desktop\DATA\DEV\GIT\CanvasUI\dist\examples\jsm_src\VRButton.js => dist/examples/js/VRButton.js...
created dist/examples/js/VRButton.js in 55ms

C:\Users\T\Desktop\DATA\DEV\GIT\CanvasUI\dist\examples\jsm_src\CanvasUI.js => dist/examples/jsm/CanvasUI.js...
created dist/examples/jsm/CanvasUI.js in 200ms

C:\Users\T\Desktop\DATA\DEV\GIT\CanvasUI\dist\examples\jsm_src\VRButton.js => dist/examples/jsm/VRButton.js...
created dist/examples/jsm/VRButton.js in 35ms

should generate dist/

including both concatened jsm and js files, like threejs.

Warning, threejs version will be replaced by the defaultValue from ./devsrv.config.js in buildOptions config.

cosmetic: make eslint happy in vscode

Don't forget to install eslint globaly

npm install eslint -g

add input-text placeholder

implement placeholder for input-text

const config = {
    ...
    message: {
        type: 'input-text',
        placeholder: 'Click or select me',
        ...
    },
    label: {
        type: 'text',
        ...
    }
};

const content = {
    message: '',
    label: 'Keyboard demo'
};

not working properly with OrthographicCamera?

I am trying to use a isometric camera with CanvasUI for the UI elements but i cannot set the ui closer to the camera.
Say for example we have this code:

    const d = 60;
    var width = window.innerWidth;
    var height = window.innerHeight;
    var aspect = width / height;
// add camera
    const camera = new THREE.OrthographicCamera(
      -d * aspect,
      d * aspect,
      d,
      -d,
      1,
      1000
    );
   
    camera.position.set(22, 22, 22);
    ui = new CanvasUI(
      {
        header: "WHY",
        main: "This is the main text",
        footer: "Footer",
      },
      {
        header: {
          type: "text",
          position: { top: 0 },
          paddingTop: 10,
          height: 70,
        },
        main: {
          type: "text",
          position: { top: 70 },
          height: 372, // default height is 512 so this is 512 - header height:70 - footer height:70
          backgroundColor: "#bbb",
          fontColor: "#000",
        },
        footer: {
          type: "text",
          position: { bottom: 0 },
          paddingTop: 30,
          height: 70,
        },
      }
    );

    camera.attach(ui.mesh);
    ui.mesh.position.set(0, 0, -99); // it does not matter how close or far i set the camera position, it won't change at all. (Z)
    ui.add(uiRef.current.mesh);

Pretty sure that i must be doing something wrong, but is what i see all the time:

Just a small rectangle.

Is there a way to use this framework with OrthographicCamera?

fix raycaster overlap (keyboard example)

When several meshes overlap (example: keyboard expanded in front of another panel), each mesh has its own event handler, and its own raycasters.

A click/hover/select on 2 superimposed buttons will cause 2 events instead of one (the first button hiding the second because of its offset on the Z axis).

To improve this behavior and take into account the Z order of multiple CanvasUI components, 2 ideas should be explored:

make an intersectList object common?
raycaster manager?

change: upgrade threejs version

We are using r119.

In oculus browser, we have a non blocking error

The WebGL context was not marked as XR compatible. Call MakeXRCompatible to mark it as such. This will result in a fatal error in the future. See https://immersive-web.github.io/webxr/#dom-webglrenderingcontextbase-makexrcompatible

We should upgrade at least r125.

I've tested with r135, everything is smooth.

@NikLever are you ok for r135 ?

fix window scrollbar issue

In all examples, we have visible scrollbar.

To fix that i purpose to add a common CSS file for all examples

<link rel="stylesheet" href="../../css/reset.css"></link>

having this

body{
    margin: 0;
}

canvas {
    display: block; /* fix necessary to remove space at bottom of canvas */
}

No more scrollbars

add onRaycasterEnter, onRaycasterLeave events

example:


functon onSelect() {
}

function onRaycasterEnter() {
    // play sound
}

function onRaycasterLeave() {
    // play sound
}

mybutton: {
    type: "button",
    position:{ top: 64, left: 0 },
    width: 64,
    fontColor: "#bb0",
    hover: "#ff0",
    onSelect,
    onRaycasterEnter,
    onRaycasterLeave
},

change: examples common code

Plenty of code in examples can be stored in one js class witch can be extended :

App:

constructor
initScene
setupXR
resize
render

Final code example:

import { Environnement } from '../Environnement.js';
import { CanvasUI } from '../../jsm/CanvasUI.js';

class App extends Environnement {
    constructor() {
        super();
    }

    createUI() {
        ...
    }

}

The Environnement class will have little methods like setupRenderer, setupCamera, so we can override methods in the main class easily.

@NikLever will you accept this change ?

change: element/config refactor

First, we currently set positions, dimensions, font, colors, for each elements using attributes, but mixed with methods in the same object. Same thing for the config object which is a mix of different properties with differents usage (i.e element definition vs renderer, or panelSize vs an element name). This currently involves some filters when looping over object attributes in CanvasUI.

Let's take the current buttons example.

const config = {
    panelSize: { width: 2, height: 0.5 },
    height: 128,
    info: { type: "text", position:{ left: 6, top: 6 }, width: 500, height: 58, backgroundColor: "#aaa", fontColor: "#000" },
    prev: { type: "button", position:{ top: 64, left: 0 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onPrev },
    stop: { type: "button", position:{ top: 64, left: 64 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onStop },
    next: { type: "button", position:{ top: 64, left: 128 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onNext },
    continue: { type: "button", position:{ top: 70, right: 10 }, width: 200, height: 52, fontColor: "#fff", backgroundColor: "#2659a4", hover: "#3df", onSelect: onContinue },
    renderer: this.renderer
}

First goal : elements objetct, only having elements declaration, rather than a mixed object.

const elements = {
    'info'': { type: 'text', value:'' },
    'prev': { type: 'button', value:'...', [ ... ] onSelect: onPrev },
    'stop': { type: 'button', value:'...', [ ... ] onSelect: onStop },
    'next': { type: 'button', value:'...', [ ... ] onSelect: onNext },
    'continue': { type: 'button', value:'Continue', [ ... ] onSelect: onContinue },
};

Everything which is not an element attribute should be moved to the config object,

const config = {

   /**        
   * CanvasUI start with a panel having a size of 1x1 meter. 
   * But you can define your own size.
   */
    panel: {

        size: {
            width: 2,
            height: 0.5
        }

    } ,

   /**        
   * CanvasUI use a Canvas HTML element to renderer widgets into a texture.
   */
    canvas: {
      /**    
       * size: canvas size (width, height) of the offscreencanvas, default 512 (pixels)
       */
        size: {
            height: 128
        }
    },

    threejs: {

        /**
        * threejs attribute value can be your threejs application instance, on which canvasui will look for a renderer,
        * camera and scene attributes.
        */

        /**        
         * renderer
         * 
         * NEEDED if you want to use WebXR controllers (renderer.xr, renderer.xr.isPresenting, getControllers ...)
         */
        renderer: this.renderer, 

        /**        
         * camera
         * 
         * NEEDED if you want to use the Mouse controller, because the raycaster is attached to the camera
         */
        camera: this.camera,

        /**        
         * camera
         * 
         * OPTIONAL if you want CanvasUI to be able to add some extra meshes (i.e raycaster intercept mesh, 
         * like in the panel scrolling example )
         */
        scene: this.scene 

    }

    controller: {

        drivers: {

           /**
           * Mouse driver
           * You can use mouse to interact with CanvasUI elements
           * this required a camera in threejs.camera config
           */
            mouse: {

               /**
               * enable: true | false, default true
               */
                enable: true,

               /**
               * handler: follow, default follow ('capture' mode not yet implemented)
               * 
               * follow mode just follow the mouse cursor.
               */
                handler:'follow'
            },

           /**
           * WebXR driver
           */

            webxr: {

                   /**
                   * enable: true | false, default true
                   */
                    enable: true, 

                   /**
                   * raycasterLinesLength: in meters, default 10
                   */
                    raycasterLinesLength:10
           },

           // @nik, do you see more drivers for later ? gamepad :) ?

        },*

        /**
         * Select behavior
         * 
         * Common settings for both WebXR driver and Mouse driver
         * For WebXR driver: selectstart, select, selectend controlller events
         * For Mouse driver: mouseup/mousedown/mousemove events
         */

        select: {

              /**
               * Raycaster(s) hover behavior while element is selected
               * 
               * hover : true | false, default false
               *
               * true:  trigger onSelect once (keyboard like behavior)
               * false: trigger onSelect while raycaster is moving (select+hover) on the element. Colorpicker or Slider like behavior.
               */
                hover: false, 
    
                /**
                 * Select repeater behavior 
                 * 
                 * Only when hover = false. 
                 * Usefull for repeating onSelect event while keeping a select WebXR button pressed or mouse down. 
                 * In keyboard example, it simulate multiple keydown event
                 */   
                repeater: {

                     * When pressing WebXR select button controller or clicking the mouse on a CanvasUI element,
                     * the onSelect method of the element is triggered a first time. And that's all when hover = true.
                     * 
                     * But if hover = false, after the first onSelect method has been triggered, it will optionally wait 
                     * for timeoutms milliseconds before triggering onSelect with a setInterval().
                     * 
                     * Then if specified, onSelect event will be triggered every repeatms millisecond until 
                     * the selectend or mouseup event is triggered
                     *
                     * enable or disable the repeter
                     * enable: true | false, default false
                     * 
                     * timeoutms
                     * 0 mean setTimeout 0 , i.e onSelect will be triggered in the next event loop, similare to setImmediate.
                     * null or undefined value will trigger onSelect on the same event loop without setTimeout().
                     * timeoutms:  default 100
                     * 
                     * repeatms
                     * 0 mean setInterval 0 , i.e onSelect will be triggered in the next event loop, similare to setImmediate. (use with caution)
                     * null or undefined value will disable the repeater.
                     * repeatms: default 20
                     */
                    timeoutms:100, 
                    repeatms:20
                }
          }
    }
}

Typical compact/minimal usage :

const config = {
    panel: { size: { width: 2, height: 0.5 } } ,
    canvas: { size : { height: 128 } },
    threejs: {
        renderer: this.renderer, 
        camera: this.camera
    }
}

Then, instantiate like this

new CanvasUI( elements , config );

Benefits:

separate UI properties from methods or threejs object
a step to be "a-frame same logic"

add dropdown menu

Fondation for a descent DatGUIXR ?

Hi @NikLever,

Congratulations ! this is very clean. I'm a fan 🥇

There is no serious working datui/datguivr on both browser and webxr context today. Something more customizable (styles). Something like UI widgets suite, to make menus, drop down, sliders, color pickers ...)

I plan to make an aframe component on top of canvasui. But not sure. I want to work on some 3D UI widgets. I started with an html2canvas wrapper (aframe-htmlembed-component) , giving some nice results (demo) which is working great with latest aframe version (1.3.0), so latest threejs too.

aframe-htmlembed-component does not support scrolling and i didn't try to use it through a web worker yet.
CanvasUI support scrolling area. Nice shot.

Perhaps we can have really nice (static) effect mixing SVG+CSS (i have to make some tests).

What do you think about adding some features/helpers into canvasUI ?
So we can seriously think about a working DatGUIXR. What is missing ?

#19
a color picker (what about an image ? is it possible to read the color of the texture below the raycaster target ?)
a slider (input range, a la datui, why not)
working with mouse cursor too (webxr controllers, chrome addon or not, is mandatory to see demos for now)
a drop down (with scroll, like an html select component, with onclick event on items)
sliding panels
add an API to have the same logic as our old datui syntax

Please add the licence you want, i'd like to fork the repo and start playing with this new toy !

improve build process

devrv build process should optionaly also remove some code, i.e console.log, console.debug.

source code => cleanup =>production code => minify

It should provide both js/jsm files :

CanvasUI.js                  - bundled production code
CanvasUI.min.js           - bundled minified production code
CanvasUI.debug.js       - bundled source code

add (draft): html panel element

Just take a look if a bridge with https://github.com/niklasvh/html2canvas is easy to do.

I already used it : it's not perfect, but it make the job.

add mouseController

Parameter:

Behaviors:
- follow: will follow the mouse without moving camera
Reactivity
- slow: for button/keyboard behavior
- fast: for colorpicker/slider behavior
- norepeat: for unique mouse click event

change: consider using ES10 sugar syntax

1. Support classes: private, static & public members

Current syntax, we can't really define private / public methods/getter/setter/variable : everything is public.

class foo {
    constructor() {
        this.bar = true;
    }
}

The refactor will consist to separate public and private things, i.e

class foo {
   
    // private variable
    #foo_private= "bar_public";

    // public variable
    foo_public = "foo_public";

    constructor() {
        // modify private variable
        this.#foo_private = "bar2_private";
    
        // modify public variable
        this.foo_public = "bar2_public";

        // run a private method
        this.#myPrivateMethod();

        // run a public method
        this.myPublicMethod();
    }

    // private method
    #myPrivateMethod() {
    }

    // public method
    myPublicMethod() {
    }
}

2. Support classes: extend

Rather than something which is not currently used in all branches (i.e Prototype add/replace methods/variables, we should use something like this :

class CanvasText extends CanvasWidget {
}

3. Support classes: picking object notation

Rather than

let property;
if (object) {
    property = (typeof object.attribute != undefined) ? object.attribute : defaultValue;
}

We can (note the question mark)

const property = (typeof object?.attribute !== undefined) ? object.attribute : defaultValue;

4. All "Panels" (i.e Widget) should be a class

Not only ColorPicker, Keyboard, Slider ...

We should have Text, Button, Image

change (draft): CSS logic refactor

( just an idea, a discussion, looking for feedbacks )

First, we currently set positions, dimensions, font, colors, for each elements using attributes, but mixed with methods in the same object.
Same thing for the config object which is a mix of different properties with differents usage (i.e element definition vs renderer, or panelSize vs an element name). This currently involves some filters when looping over object attributes in CanvasUI.

Let's take the current buttons example.

const config = {
    panelSize: { width: 2, height: 0.5 },
    height: 128,
    info: { type: "text", position:{ left: 6, top: 6 }, width: 500, height: 58, backgroundColor: "#aaa", fontColor: "#000" },
    prev: { type: "button", position:{ top: 64, left: 0 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onPrev },
    stop: { type: "button", position:{ top: 64, left: 64 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onStop },
    next: { type: "button", position:{ top: 64, left: 128 }, width: 64, fontColor: "#bb0", hover: "#ff0", onSelect: onNext },
    continue: { type: "button", position:{ top: 70, right: 10 }, width: 200, height: 52, fontColor: "#fff", backgroundColor: "#2659a4", hover: "#3df", onSelect: onContinue },
    renderer: this.renderer
}

Below an imaginary refactor, a CSS file and a different CanvasUI setup.

CSS entries. element name are IDs. But we can use classes too. Some littles changes i.e fontColor become color to match css rules.

.button {
    color:#bb0;
    width:64px;
    top:64px;
    font-familly:"...";
    background-color:#000099;
    transition: background-color 0.2s;
}

.button:hover {
    background-color:#ff0;
    transition: background-color 1s;
}

#info {
    left:10px,
    top:10px,
    width:492px,
    height:50px,
    background-color:#aaa;
    color:#000
}

#continue {
    top:70px;
    right:70px;
    width:200px;
    height:50px;
    color:#fff;
}

#continue:hover {
    color:#0000ff;
}

.red {
    background-color:red;
}

Only elements declaration, rather than a mixed object.
Note the "style" attribute to override CSS definitions.
Note the "class" attribute to link custom css properties.
Both "style" and "class" should be like HTML attributes, but parsed by CanvasUI, with properties propagation when object is created.

const elements = {
    info: { type: 'text', value:'' },
    prev: { type: 'button', value:'...', style:{ left: 0 }, onSelect: onPrev },
    stop: { type: 'button', value:'...', style:{ left: 64 }, class:'red', onSelect: onStop },
    next: { type: 'button', value:'...', style:{ left: 128 }, onSelect: onNext },
    continue: { type: 'button', value:'Continue', onSelect: onContinue },
};

Everything which is not an element attribute should be moved to the config object

const config = {
    panelSize: { width:2, height:0.5 } ,
    height: 128,
    threejs:{
        renderer: this.renderer, // mandatory for XR
        camera: this.camera, // mandatory when mouse controller used
        scene: this.scene // mandatory for scrolling example, needed to add raycaster pointer when select
    }
    // this is defaults
    controller:{
        devices:{
            mouse:{ enable:true, handler:'follow' },
            xr:{ enable:true, raycasterLinesLength:10 }
        },
        select:{
            hover: false, // select once (keyboard behavior), 
            // hover:true, // trigger select event while raycaster is moving (select+hover) on the element (colorpicker or slider behavior)
            repeater:{
                // when hover=false, we have a repeater option, delayed trigger select event + repeat. 
                // usefull to handle "delete" keyboard key, select will repeat until you selectend
                timeoutms:100, // 0 mean immediate, 
                repeatms:20
            }
        }
    },
    
}

new CanvasUI( elements , config );

Benefits:

separate UI properties from other settings
a simple css file can define the look of multiples elements
realtime editing via chrome console ?
a step to be "a-frame same logic"
explorations: custom css attribute ?
we can parse/implement every interesting css attribute we want (display:none, :hover, transition should be a killer feature

There are multiples ways to read CSS values. One ot them don't need to push something in the DOM Tree, but as some logical limites.
https://stackoverflow.com/questions/324486/how-do-you-read-css-rule-values-with-javascript

I successfully tested the one from Derek Ziemba

//Inside closure so that the inner functions don't need regeneration on every call.
const getCssClasses = (function () {
    function normalize(str) {
        if (!str)  return '';
        str = String(str).replace(/\s*([>~+])\s*/g, ' $1 ');  //Normalize symbol spacing.
        return str.replace(/(\s+)/g, ' ').trim();           //Normalize whitespace
    }
    function split(str, on) {               //Split, Trim, and remove empty elements
        return str.split(on).map(x => x.trim()).filter(x => x);
    }
    function containsAny(selText, ors) {
        return selText ? ors.some(x => selText.indexOf(x) >= 0) : false;
    }
    return function (selector) {
        const logicalORs = split(normalize(selector), ',');
        const sheets = Array.from(window.document.styleSheets);
        const ruleArrays = sheets.map((x) => Array.from(x.rules || x.cssRules || []));
        const allRules = ruleArrays.reduce((all, x) => all.concat(x), []);
        return allRules.filter((x) => containsAny(normalize(x.selectorText), logicalORs));
    };
})();

face camera

Can CanvasUI face the camera? Like sprites do?

canvasUI refactor

Currently, we are creating Canvas UI widgets by instantiating a Canvas class.

Just thinking to tend to something like datGUIXR.

// only one CanvasUI instance, so we can shared data between widgets (raycaster, xr controllers events ..)
const canvasUI = new CanvasUI();

// then we can add widgets like this
const panelLeft = canvasUI.add(contentPanelLeft, configPanelLeft);
const panelRight = canvasUI.add(contentPanelRight, configPanelRight);

// open the door to new usefull methods ?
panelLeft.hide(); // show() // destroy // move // rotate // setModal // moveForward // moveBackward ..

// globals methods ?
canvasUI.hideAll(); // showAll // destroyAll // enable/visible ..

add slider

add (draft): console element

CanvasUI Console: console.log wrapper with info/warn/error/debug. Of course limited to text.

overflow scroll
automatic scroll behavior always at the bottom except if autoscroll stopped because onselect
limited to N lines

fix custom font (custom shapes example) not working

(master & dev branch)

We can see an Arial font used rather than the targeted one (Gochi Hand)

https://niksgames.com/webxr/dev/CanvasUI/shaped/

add: left menu examples

Let create a dynamic left menu in js/css so we have a way to quickly switch between examples. Like threejs examples left menu without pictures (puppeteer is a way to automate screenshot and make thumbail, but it should be an improvement for later).

Examples will be defined in the js file, something like this ;

const exampleList = [
{
    title: 'Buttons - Panel',
    url:'../buttons/'
}