Giter Club home page Giter Club logo

microsoft / mixedrealitytoolkit Goto Github PK

View Code? Open in Web Editor NEW
856.0 157.0 276.0 298.74 MB

The MixedRealityToolkit is a collection of scripts and components intended to accelerate the development of mixed reality applications targeting Windows Mixed Reality.

License: MIT License

C++ 92.48% C 1.88% C# 3.89% Batchfile 0.34% Shell 0.01% Makefile 0.02% CSS 0.18% Java 0.30% HLSL 0.42% SWIG 0.49%
holotoolkit mixed-reality hololens mixedrealitytoolkit

mixedrealitytoolkit's Introduction

MixedRealityToolkit (MRTK)

The mixed reality toolkit is a collection of scripts and components intended to accelerate the development of applications targeting Windows Mixed Reality.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

MixedRealityToolkit for Unity

If you are developing your app using Unity - please visit MixedRealityToolkit-Unity.

MixedRealityToolkit for Unreal

If you are developing your app using Unreal - please visit MixedRealityToolkit-Unreal.


Spatial Mapping

Plane Finding

Standalone Visual Studio solution containing the source code for the PlaneFinding DLL. Building this solution produces two variants of the DLL (one for use in the Unity Editor, and another for use at runtime on a HoloLens), along with a few simple tests for sanity testing changes to the code.


Spatial Understanding

Standalone Visual Studio solution containing the source code for the SpatialUnderstanding DLL.

SpatialUnderstanding library encapsulates the world understanding technology by for Conker and Fragments. It allows you to quickly find empty spaces on the walls, place objects on the ceiling, identify placed for character to sit, and a myriad of other spatial understanding queries.

There are three primary interfaces exposed by the module: topology for simple surface and spatial queries, shape for object detection, and the object placement solver for constraint based placement of object sets. Each of these is described below.

In addition to the three primary module interfaces, a ray casting interface can be used to retrieve tagged surface types and a custom watertight playspace mesh can be copied out.


Sharing

The HoloToolkit.Sharing library allows applications to span multiple devices, and enables collaboration between users in the same room or working remotely. Features include:

  • Runs on any platform, and can work with any programming language
  • Lobby & Session system
  • Synchronization System
  • Visual Pairing
  • Anchor Sharing
  • Profiler

Learn More...


Microphone Stream Selector

The MicStreamSelector library allows applications to easily access the different Microphone Stream Categories of any windows 10 device. On HoloLens, those capture types can be optimized for either: high-quality speech capture, VOIP transmission, or general room captures. The library can record wav files of indeterminate length, and it also allows realtime access to the mic data. The microphone audio data can be polled as desired (which works well for game engines that are running on a framerate, like Unity), or can be provided via an embedded callback in your app to deliver realtime data as it is available (in a state driven app, like flat XAML).

The provided XAML demo app is a uses the embedded callback method. It shows how to use the raw data from the microphone and also how to record wav files. https://github.com/microsoft/MixedRealityToolkit-Unity/blob/htk_release/Assets/HoloToolkit-Examples/Input/Scripts/MicStreamDemo.cs has an alternate example showing how to use this library inside of Unity.

To use, you want to call the proper MicInitialize function for your app, whether you will poll the data yourself every frame, like in Unity, or whether you want the plugin to provide the data to the app whenever it is ready, like in the provided XAML example. After initialization, you can call any of the functions in any order. When you are closing your application, you want to call MicDestroy() to properly deconstruct. If you have called MicDestroy() already in your app, you need to call MicInitialize before using again.

This plugin assumes categories where { SPEECH=0, COMMUNICATIONS=1, MEDIA=2 }. On HoloLens: SPEECH is low-quality, beam-formed voice. COMMUNICATIONS is high-quality, beam-formed voice. MEDIA is a general room capture.

Initialize & poll audio for Game Engines (e.g. Unity)

Initialize with callback for passive, state-driven apps (e.g. XAML)

  • MicInitializeDefaultWithGraph(int category, AudioGraph appGraph); Same as default, but in this case you must declare the AudioGraph in the app before passing to the plugin.

  • MicInitializeCustomRateWithGraph(int category, int samplerate, AudioGraph appGraph); Same as above, but uses custom samplerate.

When using this method, you should almost definitely be providing a micsignal callback delegate function when calling MicStartStream, as defined below and is as shown in the provided example. This delegate function is called every time a frame of audio is ready from the device.

Universal APIs

  • MicStartStream(bool keepData, bool previewOnDevice, LiveMicCallback micsignal); Starts streaming raw audio data from microphone. keepData will never drop data while recording and could cause huge memory use if used incorrectly -- defaults to false. previewOnDevice, if true, will playback the audio in realtime for monitoring/listening -- doesn't affect the actual acquisition of data. micsignal is a delegate function called by the plugin whenever audio data is ready -- should generally be used by state-driven apps (XAML) and not game engines (Unity).

  • MicStopStream(); Stops the raw audio streaming. Won't stop a recording, if in progress.

  • MicStartRecording(string filename, bool previewOnDevice); filename expects extension, e.g. "MySound.wav". previewOnDevice will play the audio stream in speakers exactly as MicStartStream would -- call MicStopStream to mute after using.

  • MicStopRecording(StringBuilder sb); sb returns the full path on disk to the wav file you just recorded.

  • MicDestroy(); Releases everything -- call when closing application or not using Microphone for long period of time.

  • MicPause(); Literally pauses everything happening -- useful for minimizing or backgrounding.

  • MicResume(); Resumes from paused state.

  • MicSetGain(float gain); Can call anytime. Default is 1, 0 is muted, and can go to +inifinity although that would be painful.

  • MicGetDefaultBufferSize(); If you did a default setup, you might want to know how much data to expect from the audio device in one chunk.

  • MicGetDefaultNumChannels(); If you did a default setup, you might want to know how many channels are in your microphone stream.


Useful resources on Microsoft Windows Dev Center

Discover Discover Design Design Develop Develop Distribute) Distribute
Learn to build mixed reality experiences for HoloLens and immersive headsets (VR). Get design guides. Build user interface. Learn interactions and input. Get development guides. Learn the technology. Understand the science. Get your app ready for others and consider creating a 3D launcher.

mixedrealitytoolkit's People

Contributors

alexdrenea avatar amollis avatar andymumu avatar angelahillier avatar arthur-tomlin avatar azure-pipelines[bot] avatar botrif avatar brycehutchings avatar chrisfromwork avatar cre8ivepark avatar dependabot[bot] avatar fieldsjacksong avatar forresttrepte avatar jason-cooke avatar jevertt avatar jheuvel-ms avatar jwittner avatar keveleigh avatar maj-tom avatar matthejo avatar microsoft-github-policy-service[bot] avatar mortonfox avatar mrbobbybobberson avatar msftgits avatar neerajw avatar shaynie avatar srinjoym avatar trzy avatar wiwei avatar yl-msft avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mixedrealitytoolkit's Issues

Sharing Access violation in NetworkConnection::RemoveListener

When I attempt to remove a listener which doesn't exist, Unity crashes. Can we update the interfaces so that the RemoveListener would return a bool if it doesn't exist, and not throw a access violation?

Unity:

public enum RoomMessageID : byte
{
     Owner = MessageID.UserMessageIDStart,
     Min,
     HeadTransform,
     Max
}

_serverConnection.AddListener((byte)RoomMessageID.Owner, _connectionAdapter);
_serverConnection.AddListener((byte)RoomMessageID.HeadTransform, _connectionAdapter);
_serverConnection.AddListener((byte)RoomMessageID.Max, _connectionAdapter);
_serverConnection.RemoveListener((byte)RoomMessageID.Min, _connectionAdapter)

WinDbg:

(8768.8794): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
*** WARNING: Unable to verify checksum for D:\Sandbox\Demo\Assets\HoloToolkit\Sharing\Plugins\x64\SharingClient.dll
SharingClient!std::vector<XTools::ListenerList<XTools::NetworkConnectionListener>::ListenerEntry,std::allocator<XTools::ListenerList<XTools::NetworkConnectionListener>::ListenerEntry> >::size [inlined in SharingClient!XTools::NetworkConnectionImpl::OnDisconnected+0xe4]:
00007ffb`97d3fcb4 488b4518        mov     rax,qword ptr [rbp+18h] ss:00000000`00000018=????????????????
0:000> k
 # Child-SP          RetAddr           Call Site
00 (Inline Function) --------`-------- SharingClient!std::vector<XTools::ListenerList<XTools::NetworkConnectionListener>::ListenerEntry,std::allocator<XTools::ListenerList<XTools::NetworkConnectionListener>::ListenerEntry> >::size [c:\program files (x86)\microsoft visual studio 14.0\vc\include\vector @ 1180] 
01 (Inline Function) --------`-------- SharingClient!XTools::ListenerList<XTools::NetworkConnectionListener>::NotifyListeners [d:\github\holotoolkit\sharing\src\source\common\public\utils\listenerlist.h @ 72] 
02 00000000`005fd640 00007ffb`97d3f409 SharingClient!XTools::NetworkConnectionImpl::OnDisconnected+0xe4 [d:\github\holotoolkit\sharing\src\source\common\private\networkconnectionimpl.cpp @ 381] 

SessionManagerUniversal.UI fails to build cleanly for ARM

Looks like the CSharpWrapperAPI project has no ARM target SessionManagerUniversal.UI cannot find it's referenced source files and fails.

Should I add ARM platforms for the CSharpWrapperAPI project or can I just enable the Win32 build for ARM since its only job is to output code?

Sharing new sessions leaks ports on Windows

Hello,

I am creating a new session and the Sharing service (from how I read the source code) opens a new port for that session. When the user leaves that session, the session is destroyed and the port is reclaimed.

After the 15th session, everything goes in a bad state, I pasted the log below. Netstat still states the port has been claimed:

UDP    0.0.0.0:20600          *:*
UDP    0.0.0.0:20601          *:*
UDP    0.0.0.0:20602          *:*

Any help on resolving this would be appreciated. I am running Windows Service 2016 Nano Edition.

SharingService.log

SharingService:  ** Logging Session Began at 17:23:36, 4-11-2017
SharingService: ***********************************
SharingService: ****** Sharing Service OnStart ******
SharingService: ***********************************
SharingService: Server Info:
        Build Version: 1.1.0.0
        Schema Version: 17
SharingService: Listening for session list connections on port 20602 of all network devices of the local machine.
SharingService: Local IP addresses are:
SharingService:         10.1.0.4
SharingService: Created Session "Default" with ID 0 on port 20601
SharingService: Created Session "NewSession" with ID 1 on port 20600
SharingService: User User0 at address 10.1.0.5 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 2 on port 20600
SharingService: User User0 at address 10.1.0.5 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 3 on port 20600
SharingService: User User0 at address 10.1.0.5 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 4 on port 20600
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 5 on port 20600
SharingService: User User0 at address 10.1.0.5 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 6 on port 20600
SharingService: User User0 at address 10.1.0.6 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 7 on port 20600
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 8 on port 20600
SharingService: User User0 at address 10.1.0.6 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 9 on port 20600
SharingService: User User0 at address 10.1.0.6 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 10 on port 20600
SharingService: User User0 at address 10.1.0.5 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 11 on port 20600
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 12 on port 20600
SharingService: User User0 at address 10.1.0.5 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 13 on port 20600
SharingService: User User0 at address 10.1.0.5 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 14 on port 20600
SharingService: User User0 at address 10.1.0.5 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 15 on port 20600
SharingService: User User0 at address 10.1.0.6 joined session NewSession
SharingService: User User0 left session NewSession
SharingService: Created Session "NewSession" with ID 16 on port 20600
Error: SharingService [..\..\Source\Common\Private\XSocketManagerImpl.cpp (673)]: Failed to create incoming connection o
n port 20600.  Error code: 5
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 17 on port 20600
Error: SharingService [..\..\Source\Common\Private\XSocketManagerImpl.cpp (673)]: Failed to create incoming connection o
n port 20600.  Error code: 5
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 18 on port 20600
Error: SharingService [..\..\Source\Common\Private\XSocketManagerImpl.cpp (673)]: Failed to create incoming connection o
n port 20600.  Error code: 5
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 19 on port 20600
Error: SharingService [..\..\Source\Common\Private\XSocketManagerImpl.cpp (673)]: Failed to create incoming connection o
n port 20600.  Error code: 5
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 20 on port 20600
Error: SharingService [..\..\Source\Common\Private\XSocketManagerImpl.cpp (673)]: Failed to create incoming connection o
n port 20600.  Error code: 5
SharingService: Session NewSession closed
SharingService: Created Session "NewSession" with ID 21 on port 20600
Error: SharingService [..\..\Source\Common\Private\XSocketManagerImpl.cpp (673)]: Failed to create incoming connection o
n port 20600.  Error code: 5

HoloToolkit PlaneFinding doesn't build on latest Windows SDK Visual Studio Update 2

  1. Install Visual Studio Update 2 with its default Windows SDK version.
  2. Open PlaneFinding.sln.
  3. Build Solution

result: C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Platforms\Win32\PlatformToolsets\v140\Toolset.targets(34,5): error MSB8036: The Windows SDK version 10.0.10502.0 was not found. Install the required version of Windows SDK or change the SDK version in the project property pages or by right-clicking the solution and selecting "Retarget solution".
expected: Builds successfully.

Following the directions in the error to "Retarget solution" didn't fix the problem. I found a way to update the version by manually editing the project file. Stay tuned. I'll make a pull request with this fix and get feedback on whether we should roll forward to this version and whether my fix looks correct.

Cleanup copyright license headers for all files

Currently several files have inconsistent copyright headers. Replace with:

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.

Errors when running BuildAll.bat

Not sure if missing dependencies or if my environment is reconfigured.

  • Windows 10 v1607 OS Build 14393.351
  • Java SE Dev Kit 8 Update 112 (32 & 64-bit)
  • Microsoft Visual C++ 2015 14.0.24215 (32 & 64 bit)
  • Visual Studio 2015 update 3
  • Windows SDK 10.0.10586.212
  • Windows SDK 10.0.14393.33
  • Windows SDK 10.0.26624
  • Windows SDK AddOn
  • .NET Framework SDKs 4.5-4.5.2, 4.6 - 4.6.2 (plus targeting packs)

Seems to fail building:

  • ClientUniversalCSharp
  • ClientWindowsCSharp
  • ClientWindowsJava
  • SessionServer
  • CommonDesktop.Tests
  • SideCar.Tests
"D:\External\HoloToolkit-master\Sharing\Src\Solutions\VisualStudio\HoloToolkit.Sharing.sln" (default target) (1)->
"D:\External\HoloToolkit-master\Sharing\Src\Solutions\VisualStudio\..\..\Projects\ClientUniversalCSharp\ClientUniversalCSharp.vcxproj.metaproj" (default target) (6) ->
"D:\External\HoloToolkit-master\Sharing\Src\Projects\ClientUniversalCSharp\ClientUniversalCSharp.vcxproj" (default target) (8) ->
(Link target) ->
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(638,5): error MSB6006: "link.exe" exited with code -1073741819. [D:\External\HoloToolkit-master\Sharing\Src\Projects\ClientUniversalCSharp\ClientUniversalCSharp.vcxproj]

Spatial Understanding fails to build due to errors in post build event

1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: The command "copy /Y "C:\Users\jowitt\Source\HoloToolkit\SpatialUnderstanding\Src\bin\Debug\SpatialUnderstanding\SpatialUnderstanding.dll" "C:\Users\jowitt\Source\HoloToolkit\SpatialUnderstanding\Src..\Unity-Sample\Assets\HoloToolkit\SpatialUnderstanding\Plugins\WSA\x86\SpatialUnderstanding.dll"
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: :VCEnd" exited with code 1.

Can I just delete this event?

Access Denied to SpatialAnchorTransferManager, because of Remoting?

I am trying to share anchors while remoting, and SpatialAnchorTransferManager::RequestAccessAsync() gives me SpatialAnchorTransferManager::DeniedBySystem although I think I have set up the spatialPerception capability successfully.
Another strange thing is that SpatialSurfaceObserver::RequestAccessAsync() results in SpatialAnchorTransferManager::Allowed. How can this happen, as the access requesting methods depend on the same capability (spatialPerception)?

I just hope that it is not generally unsupported to share anchors when remoting. In this already closed thread about the Unity HoloToolkit microsoft/MixedRealityToolkit-Unity#274, NeerajW stated that in Unity, there is no way to use remoting and sharing at the same time...

I would be so grateful for anyone pointing me into the right direction!

built/test files show up as adds, cluttering changes list, should be ignored

Repro:

  1. Checkout HoloToolkit
  2. Open PlaneFinding.sln
  3. Build > Build Solution. Tests > Run > All Tests. Do this in Debug and Release configurations for AnyCPU and x86.

result: 81 changes shown on your system, adding files generated by the build.
expected: Built files are ignored in your git changes list.

I'm new to Git, but I'm planning to investigate use of a gitignore file to filter these out.

SpatialUnderstanding TestApp crashes in UniversalDebug configuration

Repro:

  1. Get HoloToolkit version 6e72ca9.
  2. Open SpatialUnderstanding\SpatialUnderstanding.sln, set TestApp as your startup project.
  3. Build and deploy the UniversalDebug x86 config on your local machine (or HoloLens device).
  4. Copy dynMeshTest_0.out test file to LocalState, as described in "Note: To run these tests" comment in MainPage.xaml.cpp.
  5. Set a breakpoint on line 207 of MainPage.xaml.cpp. (The call to DebugData_LoadAndSet_DynamicScan_InitScan.)
  6. Run the app. Click the "Click Me To Test" button.
    [Your breakpoint is hit.]
  7. Debug, Step Into. (Steps into the DebugData_LoadAndSet_DynamicScan_InitScan function.)
    RESULT: Unhandled exception at 0x75693992 (ucrtbase.dll) in TestApp.exe: An invalid parameter was passed to a function that considers invalid parameters fatal. [The call stack is long, so I'll paste it into a separate comment.]
    EXPECTED: No crash.

Note that this crash doesn't occur in the UniversalRelease configuration.

Sharing: ImageTag C#-wrappers use functions which are not present in PINVOKE

When importing the latest sharing SDK into Unity i get compilation errors on all the .cs files related to ImageTag.

All errors are similar to this:
"Assets/HoloToolkit/Sharing/Scripts/SDK/ImageTagManager.cs(33,32): error CS0117: HoloToolkit.Sharing.SharingClientPINVOKE' does not contain a definition for delete_ImageTagManager"

Perhaps it is just missing SWIG definitions?

A simple workaround is to delete the ImageTag*.cs files from the Unity project.

Sharing Service on Raspberry Pi using Windows 10 IoT Core

Hi, is it possible to deploy sharing server on Windows 10 IoT core? I tried to compile the HoloToolKit package and deployed in Raspberry Pi (running Windows 10 IoT core) but it only deployed SessionManagerUniversal.UI project. How to deploy SharingServer as well?

Remove open-source code that's not needed for HoloToolkit from externally-included projects

External projects that are included in HoloToolkit have source code that is not needed for building HoloToolkit. In addition, some functionality included in HoloToolkit is covered by other projects and is not needed in HoloToolkit.

  • libcurl can be removed. Projects can use their own copy of libcurl or a number of other download management libraries.
  • swig sources can be removed in favor of binaries.
  • RakNet has dependent extensions that are not used in HoloToolkit and can be removed.
  • AprilTags support can be removed. For replacement functionality, the HoloLensCompanionKit poster calibration sample can be used.

As part of these changes, HoloToolkit's use of unsafe code can be removed, making projects that use HoloToolkit WACK compliant.

Readme update suggestions

The readme file should include the prerequisite SDKs and other setup steps as well as the proper build order for each solution.

SpatialUnderstanding fails to build for ARM

I get these build errors:

HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(15): error C4235: nonstandard extension used: '__asm' keyword not supported on this architecture
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(16): error C2065: 'fld': undeclared identifier
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(16): error C2146: syntax error: missing ';' before identifier 'n'
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(16): error C2143: syntax error: missing ';' before '__asm'
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(16): error C4235: nonstandard extension used: '__asm' keyword not supported on this architecture
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(17): error C2065: 'fsqrt': undeclared identifier
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(17): error C2143: syntax error: missing ';' before '__asm'
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(17): error C4235: nonstandard extension used: '__asm' keyword not supported on this architecture
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(18): error C2065: 'fstp': undeclared identifier
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(18): error C2146: syntax error: missing ';' before identifier 'result'
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(18): error C2143: syntax error: missing ';' before 'return'
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(84): error C4235: nonstandard extension used: '__asm' keyword not supported on this architecture
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(84): error C2065: 'fld': undeclared identifier
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(84): error C2146: syntax error: missing ';' before identifier 'a'
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(85): error C4235: nonstandard extension used: '__asm' keyword not supported on this architecture
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(85): error C2065: 'fistp': undeclared identifier
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlinePc_Z.h(85): error C2146: syntax error: missing ';' before identifier 'b'
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlineEngine_Z.h(20): error C4235: nonstandard extension used: '__asm' keyword not supported on this architecture
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlineEngine_Z.h(20): error C2065: 'fld': undeclared identifier
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlineEngine_Z.h(20): error C2146: syntax error: missing ';' before identifier 'a'
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlineEngine_Z.h(21): error C4235: nonstandard extension used: '__asm' keyword not supported on this architecture
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlineEngine_Z.h(21): error C2065: 'fistp': undeclared identifier
HoloToolkit\SpatialUnderstanding\Src\Engine\MathInlineEngine_Z.h(21): error C2146: syntax error: missing ';' before identifier 'b'

Build scripts always do a full rebuild

This can be really time consuming if iterating on functionality through projects that leverage the HoloToolkit, e.g. HoloToolkit-Unity.

Thinking of updating current Build* scripts to pass %* forward to MSBuild and adding Rebuild* scripts that call into Build* with /t:rebuild specified.

Not sure how likely it is that the Build* scripts rebuild functionality by default is relied upon however.

HoloToolkit - How can I create PERSISTENT session from console/web based client (and close from another independent client)?

Hi,
I'm using HoloToolkit library in C# console (and web) applications, only to create a session (to which other HoloLens based clients will connect later). But it is only creating ADHOC session despite passing SessionType as PERSISTENT.

Following is the Main() function code:

XToolsApp xToolsApp = new XToolsApp();
Timer timer = new Timer(60);
timer.Elapsed += (source, e) => { xToolsApp.Update(); };
timer.Start();
NetworkConnectionData networkConnectionData = new NetworkConnectionData(xToolsApp)
{
	UserName = "TestUser",
	ServerAddress = "localhost:20602",
	ViewerAddress = "localhost:20603"
};
xToolsApp.SessionManagerListener.SessionAdded += (session) =>
{
	if (session.GetName().GetString().Equals("Default", StringComparison.OrdinalIgnoreCase))
	{
		xToolsApp.CreateSession("TestSession", SessionType.PERSISTENT);
	}
};
Console.ReadLine();

In above Main() function code, the statement xToolsApp.CreateSession("TestSession", SessionType.PERSISTENT) calls the following function:

public bool CreateSession(string sessionName, SessionType sessionType)
{
	XString nameXString = new XString(sessionName);

	LogWriteLine("Creating Session " + sessionName);
	if (!this.SessionManager.CreateSession(nameXString, sessionType))
	{

		LogWriteLine("Failed to request a new session!");

		return false;

	}

	return true;

}

Then, finally, in above function, the statement this.SessionManager.CreateSession(nameXString, sessionType) calls following method which was auto generated by HoloToolkit.Sharing library and I think this code calls native code (inside SharingClient.Dll file which is placed in bin folder):

public virtual bool CreateSession(XString sessionName, SessionType type)
{
	bool ret = SharingClientPINVOKE.SessionManager_CreateSession__SWIG_1(swigCPtr, XString.getCPtr(sessionName), (int)type);
	return ret;
}

So, you see that I'm passing Session type as PERSISTENT but I still always get only ADHOC session created, which closes itself as soon as my client leaves that session (as no other user is present in that session). What should I do to get PERSISTENT session created from my client code?

Finally, how can I close it whenever required from (any) client, by getting reference to it?

Enable Network Discovery in SharingClient

I've noticed that Profiler Tool has a very nice feature to auto discover the server (and other clients) on the current network.
I think that this would be a great feature to have on the SharingClient side so it can be integrated in your HoloLens client app.
I've already made the updates and would like to see what you guys think - I'll submit the PR soon.

The idea is, that if you guys approve, we can update the HoloToolkit-Unity SharingStage class with an AutoDiscover flag that can be used instead of IP. I already have a working version of that too, but I will hold off that PR until you guys confirm that it's a good idea.

build script for mac fails

the script BuildAllMac.sh fails with the following errors:

xcodebuild: error: The project named "zint" does not contain a scheme named "zint". The "-list" option can be used to find the names of the schemes in the project.

xcodebuild: error: The project named "zxing" does not contain a scheme named "zxing". The "-list" option can be used to find the names of the schemes in the project.

./BuildAllMac.sh: line 7: cd: Src/External/apriltag-2015-03-18/: No such file or directory

./BuildAllMac.sh: line 8: ./BuildLib.sh: No such file or directory

chmod: Src/External/swigwin-3.0.2/swig: No such file or directory

xcodebuild: error: 'Src/Solutions/Xcode/XToolsClient/XToolsClient.xcodeproj' does not exist.

System.EntryPointNotFoundException-->'CSharp_SharingManager_Create'

Hi,
I am trying to run SharingService in a 4.5 Framework application. I follow the startup with:
HoloToolkit.Sharing.ClientConfig config = new HoloToolkit.Sharing.ClientConfig(HoloToolkit.Sharing.ClientRole.Primary);
HoloToolkit.Sharing.SharingManager sharingMgr = HoloToolkit.Sharing.SharingManager.Create(config);
I create the config file but as soon as I execute second line. I receive "System.EntryPointNotFoundException". 'CSharp_SharingManager_Create' cannot be found in 'SharingClient'. I have added the library generated in Src\bin\Debug\Win32\ClientWindowsCSharp

can't capture photos at 1280x720 resolution after using SpatialUnderstanding

This is a duplicate of issue 520 that I have entered in the HoloToolkit-Unity project and was advised to enter here as well.

REPRO CASE:

  1. Open HoloToolkit-Unity in Unity 5.5.1f1.
  2. Create new scene.
  3. Create a game object with the atached PhotoCaptureExample.cs script. (This script is based on https://docs.unity3d.com/ScriptReference/VR.WSA.WebCam.PhotoCapture.html, but the resolution is changed to 1280x720.)
  4. Add the SpatialMapping prefab to your scene.
  5. Add the SpatialUnderstanding prefab to your scene. In the Spatial Understanding component, uncheck the Auto Begin Scanning property.
  6. In player settings, publishing, check the WebCam checkbox.
  7. Build and run the project on HoloLens.

result: The following error occurs and then the application unexpectedly exists:
Failed to set capture resolution (hr = 0xC00D36B4)
(Filename: C:/buildslave/unity/build/Runtime/VR/HoloLens/WebCam/PhotoCapture.cpp Line: 1180)

expected: PhotoCaptureExample script succeeds and prints "OnCapturedPhotoToMemory succeeded". Note that you can currently get the expected result either by changing the resolution to 2048x1152 or by skipping over the call to SpatialUnderstanding_Init.

You can access a repro case from my fork at https://github.com/ForrestTrepte/HoloToolkit-Unity/tree/issues/520_SpUndPhCap_FailedMinRepro. Use the scene Assets/Issue520.unity.

PhotoCaptureExample.txt

Strings are truncated if they contain '\0' character

std::string is technically not 0-terminated, and so you can construct valid std::strings with '\0' characters in them. This is especially useful because you can wrap raw binary data into an std::string and still cart it around properly.

The issue is that at the network level, the Sharing library uses .c_str() to put the std::string into a RakString before sending. RakString is a 0-terminated string, so it will truncate the std::string to the 0 terminator.

The fix is to not use RakString. There are plenty of solutions, but what I chose was to send a uint32 size, and then WriteAlignedBytes for the string data. Similarly, on the read side, instead of reading a RakString, I read the uint32 size and ReadAlignedBytes into an std::string.

I have written this and it works. I can go through the hassle of branching and doing a pull request, but this is simple enough that it seems it could be fixed on your end using your own style/preferences.

NOTE: The way I did this, it caused old servers to crash, so the server has to be replaced before using.

SpatialUnderstanding truncates pointers in x64

I get these warnings which scare me when building for x64:

1>PlaySpace\PlaySpaceInfos_W.cpp(3682): warning C4311: 'type cast': pointer truncation from 'U8 *' to 'U32'
1>PlaySpace\PlaySpaceInfos_W.cpp(3682): warning C4302: 'type cast': truncation from 'U8 *' to 'U32'

It isn't obvious if SpatialUnderstanding TestApp succeeded or failed

When you run TestApp from SpatialUnderstanding.sln and press the Click Me To Test button there is no feedback about whether the test succeeded or failed. I suppose that you would know if it crashed, but there is no verification of correct results and at some places if things aren't working the test silently returns. In order to verify the test, you need to step through the code in the debugger.

SpatialUnderstanding TestApp silently skips running the test if test content is not installed

Repro:

  1. Get HoloToolkit version 6e72ca9 .
  2. Open SpatialUnderstanding\SpatialUnderstanding.sln, set TestApp as your startup project.
  3. Build and deploy the UniversalDebug x86 config on your local machine (or HoloLens device).
  4. Set a breakpoint on line 213 of MainPage.xaml.cpp. "if (f == NULL)"
  5. Run the app. Click the "Click Me To Test" button.
    [Your breakpoint is hit.]
  6. Debug, Step Over.
    RESULT: Steps to line 215, the return statement. f is null, causing the function to return early. A naïve user might think that the test function executed successfully when, in fact, no test was run.
    EXPECTED: For someone new to the project, it isn't obvious that the test requires you to copy test content to the application as described in "Note: To run these tests". It would be nice for the test to debug break or print out an error telling the user what needs to be done in order to run a valid test. Or, better yet, would be to include the dynMeshTest_0.out file in the TestApp project in such a way that it is automatically deployed with the app so the user doesn't need to copy the file manually.

No way to represent internal session state without joining session

I have a use case where I want to display some modifiable state of the session on a list of the available sessions. The state can't be contained in the name of the session because session names cannot be changed after creation.

I'm thinking of adding an API to request sync data via URI like paths relative to the Root sync element - e.g. "my/special/data". The result can provide a const Sync::Element interface for each requested path. One could provide a list of these sync paths in the ListSessionsRequest.

Thoughts? I'll probably start heading in this direction soon.

Sharing relies heavily on outdated Windows SDK version 10.0.10240.0

From a clean install of Visual Studio Update 3 with the newest Windows SDK (version 10.10.10586.0) there are failures to build (HoloToolkit/Sharing/BuildAll.bat) in numerous dependencies.

Was thinking we'd just remap every project, leaving the minimum at the current where possible.

PlaneFinding.Editor build x64

Hello,

The HoloToolkit.Unity repro has an PlaneFinding.dll compiled as x64.
But the PlaneFinding.sln does not contain a build configuration for x64.
Is this one missing? What do I have to change to enable a build in x64 to use in the Unity Editor?

Thanks in advance.

SharingServer Library

Is there any reason why SharingServer class cannot be included in the sharing library that the client devices use? Also, did you guys think about a P2P version of the SharingServer (and maybe other classes like the lobby/etc) that would take advantage of the ReplicaManager2 and FullyConnectedMesh2 classes available in Raknet?

I am thinking of using the sharing library for a project that combines Holo and PC/tablet but I'd like to move away from a dedicated server model. How far off am I?

Sharing: DiscoveryClient does not find running clients/servers

Hello,
We are using the sharing bit from this repo for Hololens with Unity, but use our own Unity-wiring for the sharing, not the Holotoolkit-Unity scripts. We have used the auto-discovery for months now without problems. Yesterday I updated the binaries to the latest version (recompile this from master and export), and client discovery no longer works. I went back to before pull request #69, built and recompiled and discovery works again.

I tried looking at some relevant cpp files, but none of the ones I find relevant had any significant changes.
I am way out of my depth with C++ and the libraries used here to be able to investigate further.

tldr: Something in the last pull request broke the DiscoveryClient.

Relevant Holotoolkit-Unity issue

SpatialUnderstanding TestApp Click Me To Test button cannot be seen on HoloLens

Repro:

  1. Get HoloToolkit version 6e72ca9.
  2. Open SpatialUnderstanding\SpatialUnderstanding.sln, set TestApp as your startup project.
  3. Build and run the UniversalDebug x86 config on your HoloLens device.
    RESULT: Blank white form. The Click Me To test button from MainPage.xaml is not visible. (Is it positioned offscreen?)
    EXPECTED: Click Me To test button is visible and can be clicked to run the test.

SpatialUnderstanding TestApp silently skips running the test if test content is not installed

Repro:

  1. Get HoloToolkit version 6e72ca9 .
  2. Open SpatialUnderstanding\SpatialUnderstanding.sln, set TestApp as your startup project.
  3. Build and deploy the UniversalDebug x86 config on your local machine (or HoloLens device).
  4. Set a breakpoint on line 213 of MainPage.xaml.cpp. "if (f == NULL)"
  5. Run the app. Click the "Click Me To Test" button.
    [Your breakpoint is hit.]
  6. Debug, Step Over.
    RESULT: Steps to line 215, the return statement. f is null, causing the function to return early. A naïve user might think that the test function executed successfully when, in fact, no test was run.
    EXPECTED: For someone new to the project, it isn't obvious that the test requires you to copy test content to the application as described in "Note: To run these tests". It would be nice for the test to debug break or print out an error telling the user what needs to be done in order to run a valid test. Or, better yet, would be to include the dynMeshTest_0.out file in the TestApp project in such a way that it is automatically deployed with the app so the user doesn't need to copy the file manually.

TapToPlace Small Issue

Probably just a HoloToolkit-unity issue.

Have a feeling that we should check if the anchorManager.AnchorStore exists and are ready before Attach or Remove Anchors in the OnInputClicked handler.

Something like:
if (anchorManager.AnchorStore != null)
{
// Add world anchor when object placement is done.
anchorManager.AttachAnchor(gameObject, SavedAnchorFriendlyName);
}

Visual Pairing Question

On the Sharing page here: https://github.com/Microsoft/HoloToolkit/tree/master/Sharing there's a quick description of Visual Pairing which I've copied below

Visual Pairing

Connect devices just by looking at them

One device displays a QR code with connection info and security code
Other device sees QR code, connects, and validates with security code
Can also detect location in 3D space using built-in fiducial marker support

My question relates to how visual pairing would be enabled. It sounds as if this is for peripheral devices that have screens? Can you confirm this?

This repo is very fascinating and the documentation is quite good, thanks!

SessionManager UI Universal

Hi guys,

As I was building my HoloLens shared experience, I quickly found a need to have an app that can monitor the session. I then discovered the SessionManager.UI sample which did exactly that, but as a classic windows app and not UWP.
So I started building the UWP version and have it at a point where it can join sessions and send some commands to the shared HoloLens expertience.

If there's interest, I can update the UI to match the sample you have and have a pull request for you guys.

Later on, not sure if it's even possible but, would be interesting to try and get it working on ARM as well so you can monitor the session from your phone.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.