Giter Club home page Giter Club logo

webxr-input-profiles's Introduction

WebXR Input Profiles

Build Status

Repository

This repository contains information necessary for User Agents to have conformance in WebXR XRInputSource objects for all known hardware devices. It also contains assets and a helper library for developers to visualize motion controllers reported though XRInputSource objects. A preview page is also included to allow end-to-end validation of new hardware. The master branch of this preview page is hosted on Github here:

Profile Validator and Viewer

Packages

  • The registry package contains JSON files which define the intrinsic values for each type of XRInputSource hardware to ensure User Agent conformity.
  • The assets package contains 3D assets and JSON files to describe the relationship between those assets and the associated XRInputSource profiles defined in the registry. The build step of this package merges its content with the JSON files in the registry package.
  • The motion-controllers package contains a javascript library able to load the JSON descriptions published from the assets package and create component-style representations of the XRInputSource data. Specifically, it maps the the Gamepad data exposed by the XRInputSource. This library is 3D engine agnostic.
  • The viewer package contains a webpage that uses the motion-controllers library to load and view the profiles and assets from the assets package.

Versioning

Packages will be published to npm as changes occur, with version numbers formatted as <Major>.<Minor>.<Patch> and updated according to the following guildlines:

Major

  • Significant design changes

Minor

  • Additional features added
  • Small breaking changes to schema
  • Breaking changes in source code or test code

Patch

  • Adds new mapping and/or asset files
  • Fixes to existing mapping and/or asset files
  • Critical, non-breaking security fixes
  • Occasional non-breaking fixes to schema, source code, or test code

Packages from this repo may update their Minor and Patch versions at a different cadence. Changes to Major versions are expected to be large enough that all packages will update in tandem.

webxr-input-profiles's People

Contributors

artyom17 avatar atarng-magicleap avatar bialpio avatar cabanier avatar codyjasonbennett avatar daoshengmu avatar david-pico avatar de-panther avatar dependabot[bot] avatar felixtrz avatar fordacious avatar himorin avatar lachlanfordms avatar lawwong avatar lojjic avatar manishearth avatar marlon360 avatar mhenry07 avatar msub2 avatar nellwaliczek avatar propuke avatar raananw avatar riccardogiorato avatar song-fangzhen avatar thetuvix avatar toji avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

webxr-input-profiles's Issues

README.md - Document steps for filing a bug

The README.md needs a section added explaining how to file a bugs. This may involve creating issue templates. It may mean separate instructions for source bugs and profile bugs.

Decide on versioning semantics

In the context of a major/minor/patch version scheme, we need to decide which types of changes to this repo would trigger a version bump. Throwing down a first proposal, I suggest:

Patch

  • Adding new mapping and asset files
  • Fixes to existing mapping or asset files

Minor

  • Fixing bugs in source code
  • Fixing bugs in tests
  • Non-breaking schema changes (additive features)
  • Non-breaking source changes (additive features)

Major

  • Breaking schema changes
  • Breaking source changes

Additionally, I'm wondering what we want to do about the version property in the mapping files. It seems a bit much to rev the version # in a mapping file simply because a new mapping file was added. Perhaps we change the version property in the mapping.json files to just be major/minor?

Figure out plan for asset formats

Currently thinking of requiring a .gltf model for all folders and optionally allowing other asset formats. This might make it a bit challenging to add automated tests for any random asset type, so perhaps there's an approved "secondary asset types" list? If we go down this route, we'll need to change the schema portion defining assets to be more flexible.

Generic profile names don't specify grip button presence

In src/profiles, there are generic names thumbstick-controller, touchpad-controller, and touchpad-thumbstick-controller. However, with those names and the functionality that their profiles.json files define, it's still ambiguous whether or not the controllers have a grip button (or if it's just a placeholder button required by the xr-standard mapping).

For example, ideally there would be separate profiles for:
touchpad-controller (grip button slot is placeholder required by xr-standard mapping)
grip-touchpad-controller (grip button is actually exposed)

Add behavior for unknown gamepads

"unknown" gamepad ids may appear if the hardware is unidentifyable by the user agent or if the user agent has provided users a way to mask their gamepad id for privacy reasons. In this situation, a mapping and asset should be present to fall back to. This same fallback should be used if the gamepad id does not have a mapping in the repo.

Investigate how to optimize schema tests

As of PR #4, the schema tests are a lot of copy/paste. I'd like to figure out how to get jest to consume a manifest of some sort and automatically run the same test against the manifest's supplied input.

Figure out what to do about screen-based input sources

As authored, this library invokes an asychronous method to fetch and load a profile. When the input source is of type 'screen', this creates unnecessary overhead. This issue tracks thinking about how we might choose to modify the library to avoid this.

Alternatively we can do nothing and expect developers to check the type of the XRInputSource before invoking the Profiles.createMotionController() method.

Define a registry for defining how UAs expose input devices

While the WebXR spec (after immersive-web/webxr#553 is merged) provides some guidelines around exposing controllers as Gamepads, there is still room for interpretation that could lead to different behavior among implementations. In addition, future controller/input mechanisms (i.e., hands as exposed by a specific technology) may not map well to "xr-standard" or otherwise have obvious mappings.

To encourage consistency and interoperability, details about how specific controllers should be exposed by user agent implementations should be defined in a registry. The registry would be separate from the spec, allowing out-of-band updates (per an established process). It would both clearly define how implementations should expose specific controllers and thus how applications and libraries (e.g., https://github.com/immersive-web/xr-gamepad-mappings) can expect them to be exposed.

The registry would map some well understood definition of a physical controller to:

  • id string
  • GamepadMappingType
  • The physical button that represents the primary action
  • A clear definition of how the physical buttons are exposed via the Gamepad interface
    • This would include which buttons are not to be exposed, such as those reserved by the system and/or user agents.

Ideally, the entry would also include some concrete information, such as USB vendor and product IDs and/or identifier exposed by native runtime(s).

A single entry could cover multiple devices (i.e., revisions of a controller) as long as the physical appearance (due to the intent for applications to use the id to choose a controller model) and button mapping are the same. If there are multiple versions of hardware with the same product name, they may be identified by version, year, or similar.

Such a registry would also solve a couple open issues:

  • Format of Gamepad identifiers (#550): While the registry could have guidelines, there wouldn’t be a need to define a perfect rule covering all hardware because the registry would specify the identifier.
  • As such, identically-/similarly-named controllers (e.g., “Oculus Touch” per immersive-web/webxr#550 (comment)) can be clearly and easily differentiated.


One additional possibility is that [XR-specific] GamepadMappingTypes also be specified in a registry. These would be added much more conservatively, but it might be preferable to be able to define new ones without going through the full spec process. This would also allow slimming the core WebXR spec a bit.


For reference, there is an open issue for the W3C Process Document (w3c/w3process#168) to define a process for handling registries. This shouldn’t block us from proceeding, and other specs have successfully used registries in the past.

Ensure schema can be used for both WebVR and WebXR

It could be that the way to do this is to have two separate folder hierarchies? This would allow us to have the "Oculus Touch (left)" for WebVR and the correct "OculusTouch" for WebXR. Alternatively, we could put another .json file at the root of the repo containing a list of reported gamepad IDs and which folder contains the mapping/asset.

Also, we'll need to add a component node for the target ray origin for WebVR.

Anything else?

Allow node paths to be relative

Perhaps this is just for visualResponses, but it seems like it would be useful for all objects with properties describing nodes to indicate if the node paths are relative to the parent's object's root or if they are absolute to the asset file. This might significantly simplify the number of separate visual response nodes necessary.

Reevaluate naming of targetRayOrigin in schema

So this is feedback that's a couple of PRs too late, but it didn't really click with me till I saw it used in context. The use of "origin" here feels to me like it only implies a position offset, so it read kind of weird to me at first. Is there a better term that we could use?

Originally posted by @toji in #38

Additional OpenXR-to-WebXR mapping package

We can drive further cross-UA conformance for UAs built atop OpenXR. Another package of JSON mappings could map each WebXR input profile to its corresponding OpenXR interaction profile, defining which button/axis index maps to which OpenXR /user path. OpenXR-based UAs could then simply merge in the latest OpenXR mappings once per release to inherit conformant support for all the latest controllers.

Originally posted by @thetuvix in #78

Oculus Quest mapping

Just FYI: Oculus Quest controllers mapping is exactly the same as Oculus Touch one. More over, even the name of the controller is the same, including the gamepad id (at least, currently; this is done for compatibility reasons in WebVR with existing Oculus Touch-enabled experiences). The only difference is the 3D model of the controllers, but to determine this you'd need to check the name of the headset (at the moment).

Several default profiles are ambiguous about the presence of a button

While working on updating to the xr-gamepad-mappings library to match the latest spec text, I discovered a fairly annoying profile definition issue. I'm expecting to have the library updates posted shortly, but in the meantime I figured I'd get this issue filed for discussion.

The XRInputSource.getProfiles() call is designed such that developers can iterate over the list returned and select the best profile they know how to render and interpret. We've defined several default profiles to make this easier such as button-controller, thumbstick-controller, touchpad-controller, and touchpad-thumbstick-controller. The trouble is we defined the latter three to be ambiguous about whether or not a button is also present. In order to utilize this optional button or render its movement appropriately, developers must also check if the gamepad's mapping is xr-standard and potentially load/animate a different renderable model.

I know it's not ideal to have to define a bunch of additional default profile names (button-thumbstick-controller, button-touchpad-controller, button-touchpad-thumbstick-controller), but I'm not sure what other better approach there is. I'm super open to ideas though!

/cc @thetuvix

Fill in missing sections in the README.md

Sections that still need filling in:

Motivation

  • Explaining WebVR and WebXR
  • Defining motion controllers

Design goals

  • Remove "Developers should not need to add code checks for specific Gamepad.id as aberrant user agent behavior is also documentable in the schema as bugs are discovered." until we have a solution for this.

Adding New Hardware

  • Describe the steps for adding a new mapping folder and how to test the chane
  • Create Issue and PR templates

Filing a bug

  • Describe which sorts of bugs are appropriate to file on this repo and how to do it
  • Create an Issue template

Hardware

  • Update the supported and missing lists

Allow for asset types other than glTF/glB/fbx

After some back-and-forth discussion, it was decided that the the mapping files will require assets to at least be available as FBX or glTF/glB. We'll look into what an extension schema would look like to allow other well-known formats.

SteamVR Tracker support

(not sure if this is the right location for this ticket so please inform me if it needs to be moved)

I am encountering difficulty in implementing HTC Vive Trackers in the current WebVR 1.1 standard. I'm curious how this is proposed to be supported in WebXR.

Here are the issues:

  • WebVR 1.1 browsers appear to expose controllers (and trackers) as an array accessible through Navigator.getGamepads()
  • There is inconsistency with how WebVR 1.1 browsers expose the trackers:
    -- Firefox sometimes exposes them, sometimes they are assigned handedness, sometimes not, unclear pattern to this behavior
    -- Exokit only explicitly exposes vive wands, does not expose trackers (related ticket exokitxr/exokit#884)
    -- Supermedium has similar behavior to Firefox
  • Further complicating things, Valve changed the behavior around managing inputs approx Oct 3 2018 during this release (search for "tracker") https://steamcommunity.com/games/250820/announcements/detail/3213772926319522701
  • In Oct 10 2018 Valve introduced a built-in end-user-accessible SteamVR tool for Vive Tracker Mapping. This allows the end-user to map a "role" to a tracker such as "Left Foot" or "Right Hand". (this also created confusion which led to them to release a blog post about these changes)
  • WebVR 1.1 browsers do NOT appear to pass the controller "role" that is set by the end-user using the new SteamVR Input Mapping configuration, making it impossible for WebVR applications to use specific trackers with reliability aside from random and/or rewriting a temporary mapping tool within the web application.

The ask for WebXR spec is to expose "role" that controllers may have as determined by SteamVR that allow targeting specific trackers from a web application.

Host the test page

The manual test should be updated and hosted from this repo so that new hardware can be tested easily

Add missing profiles and assets

There are a number of profiles missing for known valid hardware. In addition, the profiles that exist are missing assets to go with them. This issue tracks getting assets added, node names updated in existing profiles, and missing profiles added as well.

Allow profile.json without assets to support AR

The schema and library were authored under the assumption that there must always be 3D assets accompanying a profile.json. However, there is still value in a profile.json file describing a motion controller for use with an AR device so that button, thumbsticks, and touchpads can be easily reasoned about. However, AR devices have no need to render the model.

Given this, would it be a good idea to remove the requirement for a 3D asset to be present?

If so the following schema changes would be required when a 3D asset isn't available:

  • Each "handedness" in the profile.json should have no "root" or "asset"
  • Each "component" in the profile.json should have no "root"
  • No "visualResponses" should be allowed in the profile.json
  • When no asset, MotionController.assetPath should be ""
  • When no asset, Component.rootNodeName should be ""
  • When no asset, Component.labelNodeName should be ""

Rename neutral hand to "none"

Per the change in WebXR, the "neutral" hand should be referred to as "none". The "hands" node probably should be renamed "handedness"

Should we use "samsung-odyssey" instead of the VID/PID?

@jacobcdewitt:
Why not use "samsung-odyssey" instead of the VID/PID? That's more human-readable and the convention in this repo seems to be that we should use readable names.

@thetuvix:
Good question!

This naming directly flows from the WinMR Gamepad.id values used in the WebVR era as keys for render model CDNs - for example: https://github.com/BabylonJS/Babylon.js/tree/master/assets/meshes/controllers/microsoft/045E-065D This was done because the WinMR native APIs exposes VID/PID for each controller, and so a UA then didn’t need to be updated when a new WinMR controller came out.

The key question now is whether this still makes sense given the broader profile system, where an app could make use of a more generic profile before UAs update.

Originally posted by @thetuvix in #78

Add automatic linting to testHelpers.js and rollup* files

Currently the rollup build only runs linting on the files passed as input and jest is only configured to run linting on *.test.js files. This means no linting is currently automatically run on the following files:

  • src/testHelpers/testHelpers.js
  • rollup.config.js
  • rollup-plugin-profiles.js

Linting can currently be performed manually by "npm run lintAll", but this is error prone.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.