needle-tools / ar-simulation Goto Github PK

What workflow / functionality is missing?
Installing AR Simulation in a project using AR Foundation 4 requires users to enable the "AR Simulation" loader in Project Settings / XR Plug-in Management first. If not unity will throw warnings that no Subsystem is enabled.
In previous versions of AR Foundation (< 4) XR Management did automatically enable Subsystems.

Describe the solution you'd like
AR Simulation should enable itself as a loader in XR Plug-in Management on installation.

• plane shows up
• UI changes to "place object"
• plane visually fades out (but is still there)
• can not click/tap to place? UI has raycast hit target on?

What workflow / functionality is missing?
When Device Simulator is used in portrait mode mouse events are only simulated as touch events when inside the device view (which kind of makes sense).
However, that leads to pretty slow movement (not much mouse space to move).
(probably also an issue that fast mouse movements currently counter-intuitively result in less camera motion)

Describe the solution you'd like
Either allow touch simulation everywhere on the window
Or add a global setting for "sensitivity"

Describe alternatives you've considered
Sensitivity can currently be adjusted on ARCamera/DesktopPoseProvider however I don't want to add that component to stay flexible.

Does not work atm because the used example component "PlaceMultipleObjectsOnPlane" only supports touch input.

Hi, I just downloaded and started evaluating your ar-sim tool. Seemed great out of the gate as everything was just working but I quickly found that input clicks just stop working where in the build they don't so it's not the code.

I am using Unity 2019.3.15f1, AR Foundation 3.1.3, and XR Interaction Toolkit 0.9.4 on MacOS 10.15.5 with AR-Sim 1.0.2.

I am placing several objects via my code and then manipulating them with XRIT. As I said, works in a build but in AR-Sim it works for a bit and then just.... stops. No more input is accepted for interacting with objects in AR via XRIT after a short time though the interface still works and I can still place them.

Thanks much...

Describe the bug

Switching from Game View to Device Simulator after pressing play breaks movement simulation, even after switching back to GameView. Left click continues to work but drag not (neither LMB nor RMB).

Assumption: somehow when switching windows a simulated touch gets into a wrong state.
Sometimes after randomly clicking around and dragging with left mouse, movement starts working again.

Steps to reproduce the behavior

1. Open scene 'XRInteractionToolkit/AR/ARInteraction'
2. Switch to GameView
3. Press play
4. Move around
5. Switch to DeviceSim
6. Try to move around
7. Switch back to GameView
8. Try to move around

Expected behavior

Movement works

Actual behavior

Movement stops working after initial switch and stays broken afterwards

Screenshots / GIFs

Versions (please complete the following information):

• Unity 2019.3.15f1
• ARFoundation 3.0.3
• OS: Windows
• ARSimulation 1.0.0-preview.4

Using ARS but now have a hybrid AR/Non-AR scene. Meaning I need to start and stop AR and go between a regular 3D scene and an AR scene. I’m having a hard time debugging this with ARS active. I have 2 cameras one for AR and one for regular. I deactivate all the AR nodes (session, session origin, etc) and AR camera and activate the regular camera. I also use XRGeneralSettings.Instance.Manager.StopSubsystems() to stop all AR subsystems when switching to regular mode. But yet, ARS still seems to be active and screwing with things. Also, it highjacks the regular 3D camera and keeps changing the Culling Mask on it so none of my objects (on different named layers) show up in the Game View.

So first, need ARS to stop grabbing my non AR camera and messing with it. And second, how can I turn ARS on and off so I can switch between these two modes of the app? It seems that ARS should respect me turning off XR subsystems right?

• Unity 2021.1.13f1
• ARFoundation 4.1.1
• OS: Mac
• ARSimulation 1.2.0

Thank you.

This plugin is cool.but I can't find ARmeshmanager example scene now.can anyone know where is it?

Hi~
This is happening in my laptop wifi environment. I don't know why, but ㅠㅠ
Maybe it is related to wifi. Or maybe my location is related to Korea.
I confirmed that it works in other desktop environments. I hope this helps.

Versions (please complete the following information):

• Unity 2019.4.9f1
• ARFoundation 4.1.1
• Windows 10 HP Notebook

Describe the bug
When using Device Simulator, some events seem to be sent double, resulting in incorrect behaviour for e.g. Toggles (are toggled on and immediately off again or vice versa).

To Reproduce
Steps to reproduce the behavior:

1. Clone https://github.com/needle-tools/arfoundation-samples
2. Open SceneLoader scene
3. Open Device Simulator window
4. Press Play
5. Interact with the "Menu" toggle

Expected behavior
Each click results in the toggle changing state

Actual behaviour
Sometimes, clicks result in toggle changing state twice in consecutive frames

Screenshots

Versions (please complete the following information):

• Unity 2019.3.12f1
• ARFoundation 4.0.0-preview.3
• Device Simulator 2.2.2
• OS: Windows
• ARSimulation 1.0.0-preview.3

Using ARSimulation 1.0.2

When you run any scene and just select objects from the Hierarchy it still places objects at the corner of the screen.
https://assetstore.unity.com/packages/tools/utilities/ar-simulation-173443#reviews

1. AR Foundation 4 Create an empty new scene
2. Click "Tools / AR Simulation / Setup"
3. Click Play -> crashes editor the moment we access position/rotation of main camera (called during Constructor OR Resume in Session Subsystem. Code is at "SceneSetup.EnforceCorrectCameraPositionAtStartup"

• Setup Linux environment
• Build native plugin on Linux

figure out if that is something we do wrong

in arfoundation-samples project "Anchor" sample

Could not install or assign any package with id com.neede.xr.arsimulation. Check if there are any other errors in the console and make sure they are corrected before trying again.
Failure reason: Error installing package com.neede.xr.arsimulation. Package installation timed out. Check Package Manager UI to see if the package is installed and/or retry your operation. Error message: Unable to add package [com.neede.xr.arsimulation]:
Package [com.neede.xr.arsimulation] cannot be found
UnityEditor.EditorApplication:Internal_CallUpdateFunctions()

Focus AR Camera to a point on screen just as in SceneView. Originally requested by @Scrantch

Prototype videos on twitter https://twitter.com/marcel_wiessler/status/1292097471880089602

Current features include:

• Customizeable Key
• Customizeable Focus Point (Screen Center or Mouse Position)
• Focus Distance

Describe the bug
When using the AR Point Cloud component, generated point cloud data is saved to the scene file and thus repeatedly shows up in our version control system (and has to be discarded). As it is no persistent information it should probably not be a SerializedField (or just not persist after play mode).

Steps to reproduce the behavior
Don't use Auto Scene Setup (but Auto Point Cloud is enabled),
but use manual setup via Tools -> AR Simulation -> Convert To Extended AR Scene

Versions (please complete the following information):

• Unity 2019.4.22f
• ARFoundation 4.1.5
• OS: macOS 11.2.3
• ARSimulation 1.2.0-preview.4

• 2019.3 Old Input System
• 2019.3 New Input System
• 2019.3 ARFoundation 3
• 2019.3 ARFoundation 4
• 2019.3 Universal Rendering Pipeline
• 2019.3 Built-in Render Pipeline
• 2019.3 XR Interaction Toolkit
• 2019.3 Device Simulator
• 2019.3 MARS
• 2019.3 ARRemote + ARSimulation
• 2020.1 New Input System
• 2020.1 ARFoundation 3
• 2020.1 ARFoundation 4
• 2020.1 Device Simulator
• 2020.1 MARS

Describe the bug
Before buying your asset I wanted to test it using the package here on Github (I really like it so far btw 👍).

However, I ran into a problem on my Linux system. Basically, all camera movement and interaction with AR objects is not reacting to input. UI elements and detection of point clouds/planes are working as expected.

I tried this with all three input systems (old, new and both).

Steps to reproduce the behavior

1. Open any sample scene from ar-simulation or the arfoundation-samples repository
2. Press play
3. Try to move by holding RMB + WASD keys
4. Try to click on a detected plane to spawn an object
5. Nothing happens

Expected behavior

• Being able to move around while holding RMB + WASD keys
• Being able to interact with AR objects, e.g. spawning objects on a detected plane and moving them around

Actual behavior

• Nothing happens, no error is logged

Probable Cause/Fix
What got it working for me was changing

#if !UNITY_EDITOR_OSX

to

#if !UNITY_EDITOR_OSX && !UNITY_EDITOR_LINUX

in Runtime/Compatibility/InputHelper.cs on line 257.
Not quite sure why, but Event.current.pointerType returns some integer such as 364674904 instead of the expected PointerType.Mouse enumeration.

Versions:

• Unity 2020.2.0f1
• ARFoundation 4.1.1
• OS: Linux (OpenSUSE Tumbleweed 20201221)
• ARSimulation 1.0.5
• no relevant packages

I also got a somewhat unrelated question.
Somewhere I read that meshing simulation is only supported for Windows and OSX (the former I can confirm). Is this true? Because on my system nothing happens when I run the provided Meshing Example sample. Just want to make sure that it actually is an issue before looking into it.

Describe the bug
When installing AR Simulation to a fresh project it should enable itself in XR Plug in Management automatically by default. This does not work though because XR plugin management only initialized when the settings window gets opened (creating necessary scriptable objects and hooking everything up for the first time).

Steps to reproduce the behavior

1. create a new project with e.g. 2019.3
2. drag drop the ar simulation installer package in project
3. wait for installation to finish
4. Note: no XR folder/ missing XR/XRGeneralSettings in Assets and no log message about "XR Loader has been enabled"
5. open a sample scene and enter play mode
6. Node: warnings in console about no XR Subsystem being available
7. leave play mode
8. open Project Settings and select "XR Plug in Management"
9. Note: XR folder with XR General Settings in Assets gets created

Versions (please complete the following information):

• Unity 2019.3.15
• AR Simulation Installer 1.0.0-preview.5

Version 1.0.2

Also in the AR Simulation Getting started pop up. There's an indicator icon next to Update Sample, but when I click it nothing happens. I'm not sure if I'm up to date or there's an issue with the code.
https://assetstore.unity.com/packages/tools/utilities/ar-simulation-173443#reviews

UnityEngine.XR.ARFoundation.ARInputManager:OnEnable() (at Library/PackageCache/[email protected]/Runtime/AR/ARInputManager.cs:23)"

The Readme mentions a SimulatedReality component which seems to allow a gradual plane detection from the simulated point cloud as it would appear on device. Does that feature not exist anymore?

Describe the bug

Toggling "Is Active" on Simulated AR Environment multiple times doesn't work.

Screenshots / GIFs

(difficult to see: in the end I'm trying to click on "Is Active")

Versions (please complete the following information):

• ARSimulation [1.0.0-preview.7]

What workflow / functionality is missing?
Multitouch simulation and/or touchscreen support in Editor. Currently only single-mouse-click-to-touch is supported.

Describe the solution you'd like

• a way to emulate/simulate multitouch (e.g. 2-finger-rotate) in Editor with a mouse as input
• proper support for Windows touchscreens in AR Simulation / Device Simulator

Currently blocked by Unity's touch injection code randomly breaking when more than one touch ID is used.

Currently only single touch is supported. We are exploring working around that, but hopefully at some point Unity DeviceSimulator will support this.

Workaround on our end will require simulating two touches with one mouse, e.g. similar to how LeanTouch works.

Describe the bug
Long clicking when placing an object seems to offset its position, it keeps moving while you hold the click down but keep the cursor still - see video

Steps to reproduce the behavior
1.Using arfoundation samples - open Scenes > SimpleAR > SimpleAR
2. Hit play mode and keep the left click button down when placing an object

Expected behavior

Object should be placed and stay still with the mouse if it isn't moving

Actual behavior

Object keeps moving in a forward-ish direction even though the mouse is kept still.

Screenshots / GIFs
video

Versions (please complete the following information):

• Unity [2020.2.0a13]
• ARFoundation [e.g. 4.0]
• OS: [Windows 10]
• ARSimulation [e.g. 1.0.0 - preview 4]

Describe the bug

The scene does not contain a Simulated AR Environment component for the simulated environment. This will result in the scene being included in a build.

Steps to reproduce the behavior

1. Open scene 'RaycastPlanes'
2. Build to device

Expected behavior

Scene not in build, it's just a simulated environment after all.

Actual behavior
Scene in build

Versions (please complete the following information):

• ARSimulation [e.g. 1.0.0-preview.7]

Describe the bug
Everything works except for navigation within the simulated AR scenes.
ie Right Mouse Button and WASDQE.

This is for both Walk and Fly modes. Also applies to Focus. No keypress results in focus either at mouse position or screen center.

If this can be resolved will definitely purchase.
Unity 2020.20b14 Windows.
AR Foundation and ARKit 4.1

Describe the bug
There was a confusing case where the AR camera appeared at a seemingly random place when activating AR (AR was not active from the start of the experience).
The underlying issue seems to have been that when entering some text in the app, the camera was moved by WASD key presses. This probably shouldn't happen when any input field has focus.

Expected behavior
Camera doesn't move when text is entered into an Input Field

Actual behavior
Camera moves around

• 2019.3 AR Foundation 3
• 20193. AR Foundation 4
• 2020.1 AR Foundation 3
• 2020.1 AR Foundation 4
• Build AR Foundation Example scene?!

• 2019.3 AR Foundation 3
• 20193. AR Foundation 4
• 2020.1 AR Foundation 3
• 2020.1 AR Foundation 4
• Build ARFoundation-Samples and test with all scenes

• Make sample scenes more appealing
• Remove New Input Example Scene(?!)

Describe the bug
Build failed (aborted) by an ARKit Preprocessor check.
As expected, an obvious fix for me was to just enter any non-zero width value in the ReferenceImageLibraryExample.asset.
Also, I am not using Image Tracking on this project. I could have also removed the Samples file from the project.

Steps to reproduce the behavior
I can't share my project, but if it helps, I have lost some reference for shaders in the ARSimulation samples materials.

Expected behavior
A success project build for iOS (I am building remotely on OSX)

Actual behavior
BuildFailedException

Screenshots / GIFs
I've pasted the 2 main errors

BuildFailedException: ARKit requires dimensions for all images. Reference image at index 0 named 'imageTrackingExample' in library 'Assets/Samples/AR Simulation/1.0.2/Getting Started/Assets/ReferenceImageLibraryExample.asset' requires a non-zero width.
UnityEditor.XR.ARKit.ARKitReferenceImageLibraryBuildProcessor+<ResourceGroups>d__0.MoveNext () (at Library/PackageCache/[email protected]/Editor/ARKitReferenceImageLibraryBuildProcessor.cs:36)
UnityEditor.XR.ARKit.ARKitReferenceImageLibraryBuildProcessor+Preprocessor.OnPreprocessBuild (UnityEditor.Build.Reporting.BuildReport report) (at Library/PackageCache/[email protected]/Editor/ARKitReferenceImageLibraryBuildProcessor.cs:86)
UnityEditor.Build.BuildPipelineInterfaces+<>c__DisplayClass15_0.<OnBuildPreProcess>b__1 (UnityEditor.Build.IPreprocessBuildWithReport bpp) (at <d7545a46516941d4b2f2dec578cd41ee>:0)
UnityEditor.Build.BuildPipelineInterfaces.InvokeCallbackInterfacesPair[T1,T2] (System.Collections.Generic.List`1[T] oneInterfaces, System.Action`1[T] invocationOne, System.Collections.Generic.List`1[T] twoInterfaces, System.Action`1[T] invocationTwo, System.Boolean exitOnFailure) (at <d7545a46516941d4b2f2dec578cd41ee>:0)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)

Error building Player: BuildFailedException: ARKit requires dimensions for all images. Reference image at index 0 named 'imageTrackingExample' in library 'Assets/Samples/AR Simulation/1.0.2/Getting Started/Assets/ReferenceImageLibraryExample.asset' requires a non-zero width.

Versions:

• Unity 2019.4.8f1]
• ARFoundation 3.1.3
• OS: Windows 10
• ARSimulation 1.0.2
• ARKit 3.1.3

Describe the bug

Reloading the scene in editor breaks links to the AR Session on buttons - not sure if this is a glitch from AR Foundation samples not being ready for editor simulation?

Steps to reproduce the behavior

1.Using arfoundation samples - open Scenes > SimpleAR> SimpleAR
2. Setup ARSimulation for the project and hit play
3. Use the 'reload' button in the scene, see all other button links break after that.

Expected behavior

Scene should work exactly the same when reloaded in editor play mode

Actual behavior

Button links to the AR Session component get lost somehow!

Screenshots / GIFs

Video

Versions (please complete the following information):

• Unity [2020.2.0a13]
• ARFoundation [e.g. 4.0]
• OS: [Windows 10]
• ARSimulation [e.g. 1.0.0 - preview 4]

1. open template scene
   1b) make sure ar simulation renderer feature is not already added to the forward renderer asset

2. add simulated ar environment component to example scene root gameobject (the content gameobject)

3. enable active toggle -> gameview turns blue, node: renderer feature gets added to forwardrenderer.

rendering works once you entered playmode, it only looks broken on the very first assignment.

Currently we dont have support for meshing using the ARMeshManager so we can't simulate Apples LiDAR scanning yet.

Technical Details

• Meshing is implemented via native plugins only
• Registering and discovering a native meshing plugin can be archieved via UnitySubsystemsManifest.json
• For editor subsystem discovery in a package Unity requires the keyword xreditorsubsystem to be present in the package.json:keywords array

Describe the bug
We enable AR by enabling the various AR Foundation components and disable it during runtime of our App by disabling them again. Doing the same with AR Simulation will only work the first time, after that e.g. no planes are detected.

Versions (please complete the following information):

• Unity 2019.4.21f
• ARFoundation 4.1.3
• OS: macOS 11.2.1
• ARSimulation 1.2.0-preview.2

Describe the bug

Holding right click, moving the cursor then clicking with left click produces an error 'transform.position assign attempt for 'AR Camera' is not valid'

Error: transform.position assign attempt for 'AR Camera' is not valid. Input position is { NaN, NaN, NaN }. UnityEngine.Transform:set_position (UnityEngine.Vector3) Needle.XR.ARSimulation.Simulation.DesktopPoseProvider:LateUpdate () (at Library/PackageCache/[email protected]/Runtime/InputDevices/DesktopPoseProvider.cs:128)

Steps to reproduce the behavior
1.Using arfoundation samples - open Scenes > SimpleAR > SimpleAR
2. hit play and hold down right click then move the cursor and left click
refer to video if unsure

Expected behavior

Should be no error message

Actual behavior

Error message, that never goes away in play mode!

Screenshots / GIFs

Video

Versions (please complete the following information):

• Unity [2020.2.0a13]
• ARFoundation [e.g. 4.0]
• OS: [Windows 10]
• ARSimulation [e.g. 1.0.0 - preview 4]

It didnt work in a built app on device to verify how it is supposed to work. Maybe the sample is broken?

Tested with AR Foundation 4

Reported by user via email:

Whenever I disable and later on enable the ARPlaneManager I get the following exception:

InvalidOperationException: Trackable C79FE40000000000-1B8DA44000000000 being updated this frame has at least one other action associated with it.
UnityEngine.XR.ARSubsystems.ValidationUtility`1[T].AddToSetAndThrowIfDuplicate (UnityEngine.XR.ARSubsystems.TrackableId trackableId, System.Boolean shouldBeInDictionary, System.String action) (at Library/PackageCache/com.unity.xr.arsubsystems@4.1.3/Runtime/TrackingSubsystem/ValidationUtility.cs:65)
UnityEngine.XR.ARSubsystems.ValidationUtility`1[T].ValidateAndThrow (UnityEngine.XR.ARSubsystems.TrackableChanges`1[T] changes) (at Library/PackageCache/com.unity.xr.arsubsystems@4.1.3/Runtime/TrackingSubsystem/ValidationUtility.cs:32)
UnityEngine.XR.ARSubsystems.ValidationUtility`1[T].ValidateAndDisposeIfThrown (UnityEngine.XR.ARSubsystems.TrackableChanges`1[T] changes) (at Library/PackageCache/com.unity.xr.arsubsystems@4.1.3/Runtime/TrackingSubsystem/ValidationUtility.cs:55)
UnityEngine.XR.ARSubsystems.XRPlaneSubsystem.GetChanges (Unity.Collections.Allocator allocator) (at Library/PackageCache/com.unity.xr.arsubsystems@4.1.3/Runtime/PlaneTracking/XRPlaneSubsystem.cs:81)
UnityEngine.XR.ARFoundation.ARTrackableManager`4[TSubsystem,TSubsystemDescriptor,TSessionRelativeData,TTrackable].Update () (at Library/PackageCache/com.unity.xr.arfoundation@4.1.3/Runtime/AR/ARTrackableManager.cs:187)

Unity version: 2019.4.19f1
ARFoundation version: 4.1.3
ARKit XR plugin: 4.1.3
Operating system: Windows 10

• Add support to SimulatedTrackedImage for rendering the image on a mesh.

• Should also work/support Environment/CameraBackground rendering and support urp and builtin pipeline out of the box.