microsoft / mixedreality-qrcode-sample Goto Github PK

View Code? Open in Web Editor NEW

95.0 9.0 26.0 30.03 MB

A single repository of Mixed Reality samples in Unity.

License: MIT License

mixedreality-qrcode-sample's Introduction

page_type

name

description

languages

products

sample

QR code tracking in Unity

This sample shows you how to use QR Codes in Unity projects using a HoloLens or Windows Mixed Reality immersive headset.

csharp

windows-mixed-reality

hololens

QR code tracking in Unity

Supported Unity versions	Built with XR configuration
Unity 2020 or higher	Windows XR Plugin

This sample shows you how to use QR Codes in Unity projects using a HoloLens or Windows Mixed Reality immersive headset. Covered features include displaying a holographic square over QR codes and associated data such as:

GUID
Physical size
Timestamp
Decoded data

File/folder	Description
`Sample/Assets`	Unity assets, scenes, prefabs, and scripts.
`Sample/Packages`	Project manifest and packages list.
`Sample/ProjectSettings`	Unity asset setting files.
`Sample/UserSettings`	Generated user settings from Unity.
`.gitignore`	Define what to ignore at commit time.
`README.md`	This README file.
`LICENSE`	The license for the sample.

Prerequisites

Install the latest Mixed Reality tools
Install the recommended Unity version
Download the QR code NuGet package
For Windows Mixed Reality headsets: QR code tracking on desktop PCs is only supported on Windows 10 Version 2004 and higher.

Setup

Clone or download this sample repository.
Open the Sample folder in Unity Hub and launch the project

Running the sample

Hit Play in the Unity editor
In the HoloLens connected to Unity or with Remoting, a QR pops up in the scene in front of the user
When the QR code is read, a rectangle object is placed at the QR code coordinates

Key concepts

You can find the full article on QR code tracking on Microsoft Docs. The following is a breakdown of each C# script in the sample app:

QRCode.cs - This script is attached to the QR code object and populates the text displayed in the scene with the QR code properties on Start.

QRCodesManager.cs - This is the main class that handles the QR SDK, including:

Loading the plugin
Checking if tracking is supported
Requesting access to the webcame
Adding event listeners for new, updated, and removed QR codes
Handling events through callback functions
Starting and stopping QR code tracking
Maintaining a local list of QR codes

QRCodesSetup.cs - Kicks off the QR code manager tracking functionality in QRCodesManager.

QRCodesVisualizer.cs - Handles all QR code visualizing in the scene and instantiates all QR codes in the local list kept in QRCodesManager.

SpatialGraphNodeTracker.cs - This script is attached to the QR code object and transforms real-world QR code coordinates into the Unity coordinate system. The script also places the virtual QR code in the scene at the same location as the real-world QR code.

SpatialGraphNode.cs - This script is abstracting the tracking of a spatial graph static node, which represents the tracking information of QR code in a GUID id.

OpenXR sample

Note that the "main" branch of this sample is working with Unity's "Windows XR Plugin" which works with the WinRT APIs in in Unity 2019 or 2020 LTS versions.

After upgrading to Unity 2020 or Unity 2021, You can also use "OpenXR plugin" for HoloLens 2 developement. With OpenXR plugin, the app can use the built-in support for SpatialGraphNode, and the QR code tracking will work mostly the same way as above.

To view the OpenXR version of the QRCode tracking on HoloLens 2, please checkout the "openxr" branch of this sample repro, https://github.com/microsoft/MixedReality-QRCode-Sample/tree/OpenXR.
You can also inspect how to make modifications to existing QRCode Unity project to support OpenXR from this pull request: #18

API Reference

You can find the complete API reference for Microsoft.MixedReality.QR NuGet package on Microsoft Docs.

Best practices

Quiet zones: To be read correctly, QR codes require a margin around all sides of the code. This margin must not contain any printed content and should be four modules (a single black square in the code) wide.
- The QR spec contains more information about quiet zones.
Lighting and backdrop: QR code detection quality is susceptible to varying illumination and backdrop. In a scene with bright lighting, print a code that is black on a gray background. Otherwise, print a black QR code on a white background.
- If the backdrop to the code is dark, try a black on gray code if your detection rate is low. If the backdrop is relatively light, a regular code should work fine.
Size: Windows Mixed Reality devices don't work with QR codes with sides smaller than 5 cm each. For QR codes between 5 cm and 10-cm length sides, you must be fairly close to detect the code. It will also take longer to detect codes at this size.
- The exact time to detect codes depends not only on the size of the QR codes, but how far you're away from the code. Moving closer to the code will help offset issues with size.
Distance and angular position: The tracking cameras can only detect a certain level of detail. For small codes - < 10 cm along the sides - you must be fairly close. For a version 1 QR code varying from 10 cm to 25 cm wide, the minimum detection distance ranges from 0.15 meters to 0.5 meters.
- The detection distance for size increases linearly, but also depends on QR version or module size. The higher the version, the smaller the modules, which can only be detected from a closer position. You can also try micro QR codes if you want the distance of detection to be longer. QR detection works with a range of angles += 45 deg to ensure we have proper resolution to detect the code.

Next steps

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

mixedreality-qrcode-sample's People

Contributors

Stargazers

Watchers

mixedreality-qrcode-sample's Issues

Hello, I've encountered an issue at runtime where the editor isn't reporting any errors, but the recognition process isn't successful. Upon checking for errors, I found that QRCodeManager.isSupported() is returning false. This issue persists whether it's the Windows XR branch or the OpenXR branch. Additionally, during testing, the QR code prefab wasn't successfully added, and the script on the QR code didn't run as expected. I've checked settings like the Webcam, but haven't found any issues. Has anyone faced a similar problem, and if so, how was it resolved?

I'm using Unity 2020.3.38f1c2, the closest available version to the official examples, and for MRTK, I'm using version 2.8.2.
I outputted 'issupported' and found it consistently false. In addition, I outputted 'accessStatus' and found it initially as 'DeniedBySystem', then changed to 'Allowed'. The specific runtime screenshots are as follows:
The code in the image below is within the QRCodeManager script

QR-Codes are not recognized on digital displays

I'm working on a demo for the HoloLens 2 that requires the physical input of generated links. To workaround the laborious process of typing hundrets of letters on the HoloLens, I'm using QR-Codes.

Everything works as intended, when I print the QR-Codes on paper. The goal is to scan QR-Codes from my phone or computer display. Now the HoloLens seems to have big problems to recognize QR codes on digital displays. On my PC monitor I need to stare at it for a long time, to get it to scan and on my phone it never works.

I suspect that the reflection of the infrared light interfering with camera image might be the problem here. Does anyone have an idea how to tackle this?

Using the latest OS:
22621.1252.arm64fre.ni_release_svc_sydney_rel_prod.231108-1545

PS: The same happens in the main menu of the HoloLens, not just my application

Get a "COMException" error in standalone build

Hello, I use Unity 2020.3.14f1 and remoting to Hololens 2 in editor, after fix some missing reference issue, everything is good, I can detect and track QR codes.
After I build a windows standalone, I get something like "... cannot find vcruntime140_app.dll ...", then I copy the dlls from Plugins folder to the the same folder of exe and a new error shows, a "COMExcpetion" error:

COMException
  at System.Runtime.InteropServices.Marshal.ThrowExceptionForHR (System.Int32 errorCode) [0x0000a] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at WinRT.Platform.RoGetActivationFactory (System.IntPtr runtimeClassId, System.Guid& iid) [0x0000f] in <8f8054538ba54a4781946220bf85f841>:0 
  at WinRT.WinrtModule.GetActivationFactory (WinRT.HString runtimeClassId) [0x00015] in <8f8054538ba54a4781946220bf85f841>:0 
  at WinRT.ActivationFactory`1[T]..ctor () [0x00098] in <8f8054538ba54a4781946220bf85f841>:0 
  at (wrapper managed-to-native) System.Reflection.MonoCMethod.InternalInvoke(System.Reflection.MonoCMethod,object,object[],System.Exception&)
  at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00002] in <39cd5065f72f426c85b196a374d4b1db>:0 
Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
  at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00014] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.RuntimeType.CreateInstanceMono (System.Boolean nonPublic) [0x000a8] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.RuntimeType.CreateInstanceSlow (System.Boolean publicOnly, System.Boolean skipCheckThis, System.Boolean fillCache, System.Threading.StackCrawlMark& stackMark) [0x00009] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.RuntimeType.CreateInstanceDefaultCtor (System.Boolean publicOnly, System.Boolean skipCheckThis, System.Boolean fillCache, System.Threading.StackCrawlMark& stackMark) [0x00027] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.Activator.CreateInstance[T] () [0x00021] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at WinRT.WeakLazy`1+<>c[T].<.ctor>b__2_0 () [0x00000] in <8f8054538ba54a4781946220bf85f841>:0 
  at WinRT.WeakLazy`1[T].get_Value () [0x00020] in <8f8054538ba54a4781946220bf85f841>:0 
  at WinRT.ActivationFactory`1[T].As[I] () [0x00000] in <8f8054538ba54a4781946220bf85f841>:0 
  at Microsoft.MixedReality.QR.QRCodeWatcher+Statics..ctor () [0x00000] in <8f8054538ba54a4781946220bf85f841>:0 
  at (wrapper managed-to-native) System.Reflection.MonoCMethod.InternalInvoke(System.Reflection.MonoCMethod,object,object[],System.Exception&)
  at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00002] in <39cd5065f72f426c85b196a374d4b1db>:0 
Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
  at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00014] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.RuntimeType.CreateInstanceMono (System.Boolean nonPublic) [0x000a8] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.RuntimeType.CreateInstanceSlow (System.Boolean publicOnly, System.Boolean skipCheckThis, System.Boolean fillCache, System.Threading.StackCrawlMark& stackMark) [0x00009] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.RuntimeType.CreateInstanceDefaultCtor (System.Boolean publicOnly, System.Boolean skipCheckThis, System.Boolean fillCache, System.Threading.StackCrawlMark& stackMark) [0x00027] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.Activator.CreateInstance[T] () [0x00021] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at WinRT.WeakLazy`1+<>c[T].<.ctor>b__2_0 () [0x00000] in <8f8054538ba54a4781946220bf85f841>:0 
  at WinRT.WeakLazy`1[T].get_Value () [0x00020] in <8f8054538ba54a4781946220bf85f841>:0 
  at Microsoft.MixedReality.QR.QRCodeWatcher.IsSupported () [0x00000] in <8f8054538ba54a4781946220bf85f841>:0 
  at QRTracking.QRCodesManager+<Start>d__29.MoveNext () [0x0000f] in E:\20210615 EPBS ID4 AR\Projects\Unity_ID4AR\Assets\RemoteQRTrack\Scripts\QRCodesManager.cs:82 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__6_0 (System.Object state) [0x00000] in <39cd5065f72f426c85b196a374d4b1db>:0 
  at UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () [0x00002] in <1c1473f1ae4d4d669c2e1ef28e35b413>:0 
UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
UnityEngine.DebugLogHandler:LogException(Exception, Object)
UnityEngine.Logger:LogException(Exception, Object)
UnityEngine.Debug:LogException(Exception)
UnityEngine.WorkRequest:Invoke()
UnityEngine.UnitySynchronizationContext:Exec()
UnityEngine.UnitySynchronizationContext:ExecuteTasks()

What can I do to solve it?

Update to MRTK3

This sample currently uses MRTK2. It would be great to update to MRTK3 (I'll take an initial look at this!).

https://github.com/microsoft/MixedRealityToolkit-Unity/tree/mrtk3

Not QR pops up

In the HoloLens connected to Unity by Remoting, there is not QR pops up in the scene in front of the user.

MRTK3 Sample

It's possible to update the sample to MRTK3?

The sample doesnt work with standalone remoting app with Hololens 2

Overall

It is impossible to use this sample project in Remoting standalone app mode build for Windows while remoting to Hololens 2. In Unity editor, everything works fine.

To Reproduce

It can be easily reproduced just by building the sample project targeting Windows standalone x64 and launching it. It will always give this error on start or anytime there is an attempt to initialize the QR code tracking

The error is the same also when remoting is correctly setup in the project and QR setup is only launched after the headset is succesfuly connected to the app on pc.

I have tried to temporarily include all necessary dlls in Windows environment path, which fixed the first error, but subsequently it gives the following error.

I have also tried to build the project with Il2CPP to find out if the issue is not only on Mono, but it is unbuildable on both pcs i tried it on. I should have all neccessary things for Visual Studio installed, i even added bunch of old optional stuff from Visual studio installer/Desktop development with C++ section that i figured could be linked to this problem, but no luck. I also build relatively often with Il2CPP and never had build issue like this.

I have Microsoft Visual C++ installed from 2005 all the way to 2022

I have tried every solution on both different computers, both give the same results.

Due to issue being with something with WinRT library, i have tried also OpenXR branch, which unfortunately gives the same errors. To me it appears there might be something wrong with how the QR tracking is linked to WinRT.

In Editor, there are these two special scripts to initialize important dlls on project start

Might this be a reason why everything works in Editor, but not in build?

Is there any known solution how to get QR code tracking working while Remoting to Hololens 2 from Standalone Windows app?

Possible to get pose of QR code with Holographic Remoting?

Hey everyone,

this is a question, not a bug report. I was wondering if it is possible to obtain the pose of the QR code with Holographic Remoting? Unfortunately, I don't have a HoloLens currently - otherwise I would try myself.

Thanks!

Prefab Missing Issue

Hello Sir,

please help me to provide a solution for bellow issue.

QRCodeSample scene gives an error when try to upgrade the project in 2020.3.13 unity version. i will also installed Mixed Reality OpenXR Plugin and MixedReality ToolKit using mixed reality feature toolkit and Microsoft.MixedReality.QR but still the prefab scripts are missing.

issue is:- Axis and QRCodeManage prefab is missing

Does not build to Windows IL2CPP

Which sample are you reporting a bug in?

Latest commit in OpenXR branch

Describe the bug
When attempting to build the project for a desktop target of IL2CPP, it fails to build with the following error:

   Creating library Library/Bee/artifacts/WinPlayerBuildProgram/u1oik/GameAssembly.dll.lib and object Library/Bee/artifacts/WinPlayerBuildProgram/u1oik/GameAssembly.dll.exp
j55xj1sds3ws.obj : error LNK2019: unresolved external symbol RoGetActivationFactory referenced in function IL2CPPLinkage_RoGetActivationFactory_m28A995C87663759610B3651CE3C0531D7D1E411A
j55xj1sds3ws.obj : error LNK2019: unresolved external symbol WindowsCreateString referenced in function IL2CPPLinkage_WindowsCreateString_m7A2ACFBAA6CF8ADB276044CD79B504D7FA70CBA4
j55xj1sds3ws.obj : error LNK2019: unresolved external symbol WindowsGetStringRawBuffer referenced in function IL2CPPLinkage_WindowsGetStringRawBuffer_m43E89486907592139AB0D52752B2D8D483E4FEE6
j55xj1sds3ws.obj : error LNK2019: unresolved external symbol WindowsDeleteString referenced in function IL2CPPLinkage_WindowsDeleteString_m8175860EEA9E79F2F530A1EC165CE65FDD475933
Library\Bee\artifacts\WinPlayerBuildProgram\u1oik\GameAssembly.dll : fatal error LNK1120: 4 unresolved externals

This appears to be caused by issues in one of the DLLs included with the sample.

To Reproduce
Steps to reproduce the behavior:

Install the sample
1a. (Optional) Make and test a Mono build to ensure that it is working correctly
Set Player Settings to the IL2CPP Scripting Backend
Build

Expected behavior
A working build is created

Actual behavior
Project fails to generate a working build

Screenshots

Additional context
Having the QR working with IL2CPP is critical for any project that has strict performance requirements. Having run into this issue in my main project I manage, I tested the samples to see if this IL2CPP build failure existed here as well. Sadly, it appears to. Because the internals of the Microsoft.MixedReality.QR are not public, this seems to be the best place to put this bug report. It would help the project I manage considerably if this build issue could be solved!

Unity .meta files missing in repo

The Unitys .meta files are missing for this repository, this invalidates the created prefabs as Unity loses track of the referenced scripts.
Unity .meta files should be stored in version control.

QRCodeManager spawn old QRCode

Which sample are you reporting a bug in?
MixedReality-QRCode-Sample

Describe the bug
When I start my app, the QRCodeManager spawn multiple QRCode in the scene that has been seen in a previous session

To Reproduce
Steps to reproduce the behavior:

Start the sample, look at 1 QRCode (id = 32 for example)
The QRCode is displayed
Quit the app
Restart the app
Don't look at any QRCode
There is a QRCode in the scene with the id=32

Expected behavior
I would expect the QRCode Maanger to spawn only the QRCode shown to the camera after the app is started

I have met some problems about this project

Packages folder doesnt exists!

Please add the Packages folders

Cannot Build or Run due to missing references

Which sample are you reporting a bug in?

MixedReality-QRCodeSample

Describe the bug
When trying to run/build the sample i get the following errors:

To Reproduce
Steps to reproduce the behavior:

Install Unity (Same Version as sample)
Open Sample downloaded of github
Switch Platform to Universal Windows Platform with settings for HoloLens devices
Fix missing nuget packages (QR Code package as described in readme) and install fundation and openXR with ms mixed reality feature tool

Expected behavior
After installing the fundation and qr code packages no errors should remain in the project

Configuration

Language/architecture/flavor: c#, unity
Windows version: 22000.795

Crashes On Restarting The Tracking

if I stop and start the Qr Code tracking the hololens starts to render the space partially and eventually crash without errors

microsoft / mixedreality-qrcode-sample Goto Github PK

mixedreality-qrcode-sample's Introduction

QR code tracking in Unity

Contents

Prerequisites

Setup

Running the sample

Key concepts

OpenXR sample

API Reference

Best practices

Next steps

Contributing

mixedreality-qrcode-sample's People

Contributors

Stargazers

Watchers

Forkers

mixedreality-qrcode-sample's Issues

Overall

To Reproduce

Recommend Projects

Recommend Topics

Recommend Org