• Downloaded sample file, and unzipped.
• Opened /dotnet/WPF/ScreenCapture with latest Visual 2019

Build fails.

Severity Code Description Project File Line Suppression State
Error CS0006 Metadata file 'C:\Users***\Downloads\Windows.UI.Composition-Win32-Samples-master\Windows.UI.Composition-Win32-Samples-master\dotnet\WPF\ScreenCapture\Composition.WindowsRuntimeHelpers\bin\Debug\netstandard2.0\Composition.WindowsRuntimeHelpers.dll' could not be found CaptureSampleCore C:\Users\jken\Downloads\Windows.UI.Composition-Win32-Samples-master\Windows.UI.Composition-Win32-Samples-master\dotnet\WPF\ScreenCapture\CaptureSampleCore\CSC 1 Active

Can we use this API in UWP app without calling GraphicsCapturePicker? What if I want create desktop streaming application, so that the client could connect to host and begin streaming whole screen without the need of calling GraphicsCapturePicker on the host? I see there is CreateForMonitor function in Win32 version of screen capture.

I have a question on how to build a GraphicsCaptureItem from current window's HWND in Win10 1809. We want to use Windows.Graphics.Capture APIs to capture our own win32 C++ application to a video. The problem is that we're all on Win10 1809, and CreateForWindow is added in Win10 1903. So it's a problem for us to get the GraphicsCaptureItem item for our window the capture. Any idea on how I can do this? Appreciated for any help or comment!

Im using 'winrt::Windows::UI::Composition::Desktop::DesktopWindowTarget'.
But, Probably could get same consequence with direct composition, Im not sure tho.

when left window below to right window.

when right window below to left window. ( i reversed z-order with 'SetWindowPos')

Hello,

Is it possible to get a reference to the Texture from the Framepool outside of the frame arrived function?

I have implemented OpenGL interop with my capture. At the moment I've made the back buffer a shared resource with OpenGL, when I receive a new frame, I copy to the back buffer, and then use it in OpenGL. However, I would like not to have to do the copy to the back buffer because that kind of defeat the optimization that the glInterop is supposed to provide to my app.

So I would like to set up the interop to share the resource with the texture in the Framepool.

I would like to achieve the following in my init code:

Psuedo example (Not real code)

m_framePool  = Direct3D11CaptureFramePool::Create(...) 

//Get the texture from a frame object.
framePoolTexure =GetDXGIInterfaceFromObject<ID3D11Texture2D> (m_framePool.GetFrameObject(index).Surface().get())

//Share with openGL
openGLInteropSharedTexHandle = CreateOpenGLSharedResource(openGLTexture,framePoolTexure )

//Create session
  m_session = m_framePool.CreateCaptureSession(m_item);
 m_framePool.FrameArrived(winrt::auto_revoke,{ this, &myClass::OnFrameArrived });

On top of this, there is one more thing that's needed for this to work, and that is that the parameters of the frame texture are compatible with the OpenGL interop. If I create my own texture I have control over these things. But I have no knowledge of what parameters the frame pool textures are created with. Is that something you have documented somewhere which you could share with me? I am aware that I might be chasing something that's not even possible for this reason.

Thanks for all the great work you do and thanks for the help!
M

I am developing a WPF app by using Windows.Graphics.Capture, like ScreenCapture in this repository.
I was succeed to exclude the specific window in my app by using SetWindowDisplayAffinity and WDA_MONITOR but it returns black area in my video. I heard that WDA_EXCLUDEFROMCAPTURE can make the area transparent in C++, how to define or use WDA_EXCLUDEFROMCAPTURE in C#?

Is it possible to change color, thickness or hide the yellow border?
Or some workaround to do this.

I was trying to place one button over the acrylic control but the button appears to be under the control like this:

I have tried changing _compositorDesktopInterop.CreateDesktopWindowTarget(HwndHost, true, out _compositionTarget);

second argument to false (topmost), but it doesn't work. is this because Direct composition can draw only on top of all other controls.

Hello,

I have noticed that CreateForWindow crashes in case the HWND provided does not have a border. Is this normal ?

Best Regards,
Nick

I've played around with the ScreenCaptureForHWND C++ sample project, but I need it to be faster from capture to display. I have tried different framerates (30, 60 & 120 Hz) and there is almost always a two frame delay from the captured window to the app window. Does anyone know where this delay coming from and how to lower it?

Despite the comment saying a DispatcherQueue is required, I don't instantiate one in my application, and window capture seems to work fine. What are the consequences of not doing this?

Steps

In WinMain function, add the following codes:

1. Set hwndTarget window that is the window that we want to create replica for (capture it
   Example:

HWND hwndTarget = (HWND)0x00000000000309EC;

2. Register UWP API

	// Init COM
	init_apartment(apartment_type::single_threaded);

	// I don't know why it is but I need this section to make it work
	Windows::System::DispatcherQueueController controller{ nullptr };

	DispatcherQueueOptions options
	{
		sizeof(DispatcherQueueOptions),
		DQTYPE_THREAD_CURRENT,
		DQTAT_COM_STA
	};

3. Register the window class

WNDCLASSEX wcex = {};
	wcex.cbSize = sizeof(WNDCLASSEX);
	wcex.style = CS_HREDRAW | CS_VREDRAW;
	wcex.lpfnWndProc = WndProc;
	wcex.cbClsExtra = 0;
	wcex.cbWndExtra = 0;
	wcex.hInstance = instance;
	wcex.hIcon = LoadIcon(instance, MAKEINTRESOURCE(IDI_APPLICATION));
	wcex.hCursor = LoadCursor(NULL, IDC_ARROW);
	wcex.hbrBackground = (HBRUSH)(COLOR_WINDOW + 1);
	wcex.lpszMenuName = NULL;
	wcex.lpszClassName = L"ScreenCaptureforHWND";
	wcex.hIconSm = LoadIcon(wcex.hInstance, MAKEINTRESOURCE(IDI_APPLICATION));
	WINRT_VERIFY(RegisterClassEx(&wcex));

4. Create container window will draw the replica (the captured pixels of the hwndTarget)

RECT rectLayer;
	DwmGetWindowAttribute(hwndTarget, DWMWA_EXTENDED_FRAME_BOUNDS, &rectLayer, sizeof(RECT));

	HWND hwnd = CreateWindowEx(WS_EX_NOACTIVATE | 0x080000 | WS_EX_TRANSPARENT | WS_EX_LAYERED | WS_EX_TOPMOST, wcex.lpszClassName, NULL, 0x80000000,
		rectLayer.left, rectLayer.top, rectLayer.right - rectLayer.left, rectLayer.bottom - rectLayer.top, hwndTarget, NULL, instance, NULL);



	ShowWindow(hwnd, SW_SHOW);




	res = CreateDispatcherQueueController(options, reinterpret_cast<ABI::Windows::System::IDispatcherQueueController * *>(put_abi(controller)));
	if (res != S_OK)
		return -1;


	auto compositor = Compositor();

	auto interop = compositor.as<ABI::Windows::UI::Composition::Desktop::ICompositorDesktopInterop>();
	DesktopWindowTarget desktopWindowTarget{ nullptr };
	res = interop->CreateDesktopWindowTarget(hwnd, true, reinterpret_cast<ABI::Windows::UI::Composition::Desktop::IDesktopWindowTarget * *>(put_abi(desktopWindowTarget)));
	if (res != S_OK)
		return -1;


	auto containerRoot = compositor.CreateContainerVisual();
	containerRoot.RelativeSizeAdjustment({ 1.0f, 1.0f });

	auto m_root = compositor.CreateSpriteVisual();
	m_root.RelativeSizeAdjustment({ 1, 1 });
	containerRoot.Children().InsertAtTop(m_root);

	//root.Size({ 300,300 });
	desktopWindowTarget.Root(containerRoot);


	winrt::Windows::UI::Composition::SpriteVisual m_renderElement = compositor.CreateSpriteVisual();

	containerRoot.RelativeSizeAdjustment({ 1, 1 });

	m_renderElement.AnchorPoint({ 0.5f, 0.5f });
	m_renderElement.RelativeOffsetAdjustment({ 0.5f, 0.5f, 0 });
	m_renderElement.RelativeSizeAdjustment({ 1, 1 });


	// Create brush for the render element
	auto m_brush = compositor.CreateSurfaceBrush();
	m_renderElement.Brush(m_brush);
	m_brush.HorizontalAlignmentRatio(0.5f);
	m_brush.VerticalAlignmentRatio(0.5f);
	m_brush.Stretch(CompositionStretch::Uniform);

	// Add the render element to the container root
	m_root.Children().InsertAtTop(m_renderElement);

5. Start capture

	auto d3dDevice = CreateD3DDevice();
	auto dxgiDevice = d3dDevice.as<IDXGIDevice>();
	winrt::Windows::Graphics::DirectX::Direct3D11::IDirect3DDevice m_device = CreateDirect3DDevice(dxgiDevice.get());


	auto item = CreateCaptureItemForWindow(hwndTarget);

	std::unique_ptr<SimpleCapture> m_capture{ nullptr };
	m_capture = std::make_unique<SimpleCapture>(m_device, item);

	auto surface = m_capture->CreateSurface(compositor);
	m_brush.Surface(surface);

	m_capture->StartCapture();

	//m_capture->TestArea(hwnd2);

	// Message pump
	MSG msg;
	while (GetMessage(&msg, NULL, 0, 0))
	{
		TranslateMessage(&msg);
		DispatchMessage(&msg);
	}


	return 0;

The result is:

This code reproduced a case that I created a replica window with the exact same size of the hwndTarget and it mirroring the target. It seems that for some reason that if the replica is the exact same size and position, it will capture hwndTarget with some small "window" as I showed in the image in red color..

I attached the full source code that produce the problem.

ScreenCaptureforHWND.zip

Please fix it.

Thanks.

Hello, I tried to run the example
ScreenCaptureforHWND

Using Visual Studio 2019 on Windows 10 64bit with SDK 10.0.18362.0

The example app was successfully built and working until I select some window to capture.

When I select a window, it always throws an exception

Unhandled exception at 0x00007FFD3B8B9129 (KernelBase.dll) in ScreenCaptureforHWND.exe: 0xC000041D: An unhandled exception was encountered during a user callback.

It is no matter what window I select. It always fails in this like.

Please fix the example.

Thank you.

The WPF sample apps came up in discussion for a C#/WinRT issue. We thought it would be good to get these samples ported to net5, and @robmikh said he would be glad to port them when he gets time.

The interop documentation at C#/WinRT may be helpful, for information on how certain things have changed (like QI becoming an As<> cast).

Is it possible to create a CompositionPath from WPF? It seems I need to pass a Windows.Graphics.IGeometrySource2D to the contructor of CompositionPath but I do not know how to get a Windows.Graphics.IGeometrySource2D implementation from WPF. Preferably something like CanvasPathBuilder.

I'm not sensitive to this myself, but I'm hearing reports that capturing any window causes the Windows mouse cursor to lag in general. Some think hardware cursor is being disabled, but I don't know how to verify that. Should this be happening?

We plan to work around this by disabling WGC cursor capture, and using D3D11/GDI interop to DrawIconEx into our target texture, but I'm not sure if that will restore mouse performance.

When instantiating Windows.UI.Composition.Compositor via:

compositor = new Compositor();

I am getting this:

  HResult=0x80070005
  Message=Access is denied.

Access is denied.

  Source=Windows.UI
  StackTrace:
   at Windows.UI.Composition.Compositor..ctor()
   at CaptureMonkey.MainWindow.MainWindow_Loaded(Object sender, RoutedEventArgs e) in C:\Users\max\source\repos\CaptureMonkey\CaptureMonkey\MainWindow.xaml.cs:line 79
   at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
   at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
   at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
   at System.Windows.UIElement.RaiseEvent(RoutedEventArgs e)
   at System.Windows.BroadcastEventHelper.BroadcastEvent(DependencyObject root, RoutedEvent routedEvent)
   at System.Windows.BroadcastEventHelper.BroadcastLoadedEvent(Object root)
   at MS.Internal.LoadedOrUnloadedOperation.DoWork()
   at System.Windows.Media.MediaContext.FireLoadedPendingCallbacks()
   at System.Windows.Media.MediaContext.FireInvokeOnRenderCallbacks()
   at System.Windows.Media.MediaContext.RenderMessageHandlerCore(Object resizedCompositionTarget)
   at System.Windows.Media.MediaContext.RenderMessageHandler(Object resizedCompositionTarget)
   at System.Windows.Media.MediaContext.Resize(ICompositionTarget resizedCompositionTarget)
   at System.Windows.Interop.HwndTarget.OnResize()
   at System.Windows.Interop.HwndTarget.HandleMessage(WindowMessage msg, IntPtr wparam, IntPtr lparam)
   at System.Windows.Interop.HwndSource.HwndTargetFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
   at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(DispatcherPriority priority, TimeSpan timeout, Delegate method, Object args, Int32 numArgs)
   at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam)

What could be the cause of this?

After few attempts to call this code:

auto activation_factory = winrt::get_activation_factory<winrt::Windows::Graphics::Capture::GraphicsCaptureItem>();
auto interop_factory = activation_factory.as<IGraphicsCaptureItemInterop>();
winrt::Windows::Graphics::Capture::GraphicsCaptureItem item = { nullptr };
interop_factory->CreateForWindow(hwnd, winrt::guid_of<ABI::Windows::Graphics::Capture::IGraphicsCaptureItem>(), reinterpret_cast<void**>(winrt::put_abi(item)));
return item;

I get exception:

And my code does not catch it.

It is very random. most of the time it happen after a lot of attempts to create the capture item.

Questions:

1. What can cause such exception and how to prevent it? Is there a common user mistake that cause this error?
2. How to handle the error? How to catch the error in case it happned? My code does not catch it. I don't want that the program will crash at the worst case scenario

Thanks.

Hi. T

Trying to use the new Graphics Capture API to capture a WPF window. Can anyone give me some pointers as to how I would use CreateFromVisual using a WPF window rather than UWP?

Joe

I use this to control frame rate,but occasionally throw exceptions in here.

Is there a way to stop capturing the mouse cursor in the WPF screen capture sample?

I found the following blog post that mentions a new IsCursorEnabled api:
https://blogs.windows.com/windowsdeveloper/2019/09/16/new-ways-to-do-screen-capture/#ptMq3QzycLCpvCuT.97

However I cannot find this anywhere. Is it simply not exposed in the C# side of things or something?

Good day!

Are there any plans to add a way to capture a user-defined rectangle to the API? It would be a neat functionality to have. I imagine something like this:

At the moment I am working on implementing that myself with a few approaches in mind.

1. Capture "All monitors" and use a custom RECT to "CopySubresourceRegion".
   This has two implications, firstly is the big overhead of capturing all the monitors just to output a (potentially) small rectangle. Second is that the virtual screen coordinate space that the dimensions is most likely based on does not correlate to that of the texture:
   I am not sure about the following, correct me if I am wrong, but it seems to me that the output texture of "capture all" is the same as the virtual screen, but with different coordinates?
   

This means, that if you have a region in monitor 2, it will have values that are < 0 if you got it from the virtual screen space. Which will cause a crash in the cropping stage.

2. Call "MonitorFromRect()" with the coordinates you want to capture, capture the Monitor returned. Convert the coordinates from the virtual screen space to the individual monitor.
   This comes with the issue that the region can't overlap between monitors.

3. Create a transparent, style, and borderless window, start a capture with that HWND.

• I can see quite a bit of headache with this, but would probably be the nicest as it would provide the yellow border, thus staying consistent with the other things that are captured. My limited knowledge of custom windows makes me think that it's not possible to have a completely transparent window (without title bar, borders, border shadows, etc) and still be able to move and change its size.

At the moment I have a window to set the desktop area to capture. I use the windows client coordinates to get a RECT. But that is in virtual screen coordinates.

Regarding the picker, I think it's a fantastic and nice-looking UI to use for the capture features. Is there a way to add custom UI drawing to it with WM_PAINT in a winproc callback, or does this window belong to the DWM fully?

I apologize if this question is a bit irrelevant to the usage of the API. I'm keen to hear if you (the devs) have had any ideas of this kind of feature and if there is something along the likes planned for a future version.

Have a great day!
Martin

Update:
I went with option 2 which works well. DPI scaling was a bit of an issue but setting the process to per-monitor DPI-aware fixed that.
I also realized that in the image above, the origin for DX textures are in the top left corner (not the bottom), in case anyone gets confused.

My program does not print a.

Hello, after I remix the example cpp/ScreenCaptureforHWND to this code:

int CALLBACK WinMain(HINSTANCE instance, HINSTANCE previousInstance, LPSTR cmdLine, int cmdShow)
{

	HWND hwndTarget = (HWND)0x00000000008106D2; // HWND of the window to capture. Edit this as needed
	RECT rectTarget;

	HWND hwndDisplay;

	HRESULT res;


	// Init COM
	//init_apartment(apartment_type::single_threaded);

	// Create the window
	WNDCLASSEX wcex = {};
	wcex.cbSize = sizeof(WNDCLASSEX);
	wcex.style = CS_HREDRAW | CS_VREDRAW;
	wcex.lpfnWndProc = WndProc;
	wcex.cbClsExtra = 0;
	wcex.cbWndExtra = 0;
	wcex.hInstance = instance;
	wcex.hIcon = LoadIcon(instance, MAKEINTRESOURCE(IDI_APPLICATION));
	wcex.hCursor = LoadCursor(NULL, IDC_ARROW);
	wcex.hbrBackground = (HBRUSH)(COLOR_WINDOW + 1);
	wcex.lpszMenuName = NULL;
	wcex.lpszClassName = L"ScreenCaptureforHWND";
	wcex.hIconSm = LoadIcon(wcex.hInstance, MAKEINTRESOURCE(IDI_APPLICATION));
	WINRT_VERIFY(RegisterClassEx(&wcex));

	HWND hwnd = CreateWindow(
		L"ScreenCaptureforHWND",
		L"ScreenCaptureforHWND",
		WS_OVERLAPPEDWINDOW,
		CW_USEDEFAULT,
		CW_USEDEFAULT,
		800,
		600,
		NULL,
		NULL,
		instance,
		NULL);
	WINRT_VERIFY(hwnd);


	ShowWindow(hwnd, SW_SHOW);




	// I don't know why it is but I need this section to make it work
	Windows::System::DispatcherQueueController controller{ nullptr };

	DispatcherQueueOptions options
	{
		sizeof(DispatcherQueueOptions),
		DQTYPE_THREAD_CURRENT,
		DQTAT_COM_STA
	};

	res = CreateDispatcherQueueController(options, reinterpret_cast<ABI::Windows::System::IDispatcherQueueController * *>(put_abi(controller)));
	if (res != S_OK)
		return -1;




	//using namespace Windows::Foundation;
	//using namespace Windows::UI;
	//using namespace Windows::UI::Composition;
	//using namespace Windows::Graphics::Capture;

	// Initialize Composition
	auto compositor = Compositor();

	auto interop = compositor.as<ABI::Windows::UI::Composition::Desktop::ICompositorDesktopInterop>();
	DesktopWindowTarget desktopWindowTarget{ nullptr };
	res = interop->CreateDesktopWindowTarget(hwnd, true, reinterpret_cast<ABI::Windows::UI::Composition::Desktop::IDesktopWindowTarget * *>(put_abi(desktopWindowTarget)));
	if (res != S_OK)
		return -1;


	auto containerRoot = compositor.CreateContainerVisual();
	containerRoot.RelativeSizeAdjustment({ 1.0f, 1.0f });
	//root.Size({ 300,300 });
	desktopWindowTarget.Root(containerRoot);


	winrt::Windows::UI::Composition::SpriteVisual m_renderElement = compositor.CreateSpriteVisual();

	containerRoot.RelativeSizeAdjustment({ 1, 1 });

	m_renderElement.AnchorPoint({ 0.5f, 0.5f });
	m_renderElement.RelativeOffsetAdjustment({ 0.5f, 0.5f, 0 });
	m_renderElement.RelativeSizeAdjustment({ 1, 1 });


	// Create brush for the render element
	auto m_brush = compositor.CreateSurfaceBrush();
	m_renderElement.Brush(m_brush);
	m_brush.HorizontalAlignmentRatio(0.5f);
	m_brush.VerticalAlignmentRatio(0.5f);
	m_brush.Stretch(CompositionStretch::Uniform);

	// Add the render element to the container root
	containerRoot.Children().InsertAtTop(m_renderElement);



	auto d3dDevice = CreateD3DDevice();
	auto dxgiDevice = d3dDevice.as<IDXGIDevice>();
	winrt::Windows::Graphics::DirectX::Direct3D11::IDirect3DDevice m_device = CreateDirect3DDevice(dxgiDevice.get());


	auto item = CreateCaptureItemForWindow(hwndTarget);

	std::unique_ptr<SimpleCapture> m_capture{ nullptr };
	m_capture = std::make_unique<SimpleCapture>(m_device, item);

	auto surface = m_capture->CreateSurface(compositor);
	m_brush.Surface(surface);

	m_capture->StartCapture();


	// Message pump
	MSG msg;
	while (GetMessage(&msg, NULL, 0, 0))
	{
		TranslateMessage(&msg);
		DispatchMessage(&msg);
	}

	return (int)msg.wParam;
}

I found something very strange.

If you change the code to this:

class classTest
{
public:

	int Test(HINSTANCE instance)
	{
		HWND hwndTarget = (HWND)0x00000000008106D2;
		RECT rectTarget;

		HWND hwndDisplay;

		HRESULT res;


		// Init COM
		//init_apartment(apartment_type::single_threaded);

		// Create the window
		WNDCLASSEX wcex = {};
		wcex.cbSize = sizeof(WNDCLASSEX);
		wcex.style = CS_HREDRAW | CS_VREDRAW;
		wcex.lpfnWndProc = WndProc;
		wcex.cbClsExtra = 0;
		wcex.cbWndExtra = 0;
		wcex.hInstance = instance;
		wcex.hIcon = LoadIcon(instance, MAKEINTRESOURCE(IDI_APPLICATION));
		wcex.hCursor = LoadCursor(NULL, IDC_ARROW);
		wcex.hbrBackground = (HBRUSH)(COLOR_WINDOW + 1);
		wcex.lpszMenuName = NULL;
		wcex.lpszClassName = L"ScreenCaptureforHWND";
		wcex.hIconSm = LoadIcon(wcex.hInstance, MAKEINTRESOURCE(IDI_APPLICATION));
		WINRT_VERIFY(RegisterClassEx(&wcex));

		HWND hwnd = CreateWindow(
			L"ScreenCaptureforHWND",
			L"ScreenCaptureforHWND",
			WS_OVERLAPPEDWINDOW,
			CW_USEDEFAULT,
			CW_USEDEFAULT,
			800,
			600,
			NULL,
			NULL,
			instance,
			NULL);
		WINRT_VERIFY(hwnd);


		ShowWindow(hwnd, SW_SHOW);




		// I don't know why it is but I need this section to make it work
		Windows::System::DispatcherQueueController controller{ nullptr };

		DispatcherQueueOptions options
		{
			sizeof(DispatcherQueueOptions),
			DQTYPE_THREAD_CURRENT,
			DQTAT_COM_STA
		};

		res = CreateDispatcherQueueController(options, reinterpret_cast<ABI::Windows::System::IDispatcherQueueController * *>(put_abi(controller)));
		if (res != S_OK)
			return -1;




		//using namespace Windows::Foundation;
		//using namespace Windows::UI;
		//using namespace Windows::UI::Composition;
		//using namespace Windows::Graphics::Capture;

		// Initialize Composition
		auto compositor = Compositor();

		auto interop = compositor.as<ABI::Windows::UI::Composition::Desktop::ICompositorDesktopInterop>();
		DesktopWindowTarget desktopWindowTarget{ nullptr };
		res = interop->CreateDesktopWindowTarget(hwnd, true, reinterpret_cast<ABI::Windows::UI::Composition::Desktop::IDesktopWindowTarget * *>(put_abi(desktopWindowTarget)));
		if (res != S_OK)
			return -1;


		auto containerRoot = compositor.CreateContainerVisual();
		containerRoot.RelativeSizeAdjustment({ 1.0f, 1.0f });
		//root.Size({ 300,300 });
		desktopWindowTarget.Root(containerRoot);


		winrt::Windows::UI::Composition::SpriteVisual m_renderElement = compositor.CreateSpriteVisual();

		containerRoot.RelativeSizeAdjustment({ 1, 1 });

		m_renderElement.AnchorPoint({ 0.5f, 0.5f });
		m_renderElement.RelativeOffsetAdjustment({ 0.5f, 0.5f, 0 });
		m_renderElement.RelativeSizeAdjustment({ 1, 1 });


		// Create brush for the render element
		auto m_brush = compositor.CreateSurfaceBrush();
		m_renderElement.Brush(m_brush);
		m_brush.HorizontalAlignmentRatio(0.5f);
		m_brush.VerticalAlignmentRatio(0.5f);
		m_brush.Stretch(CompositionStretch::Uniform);

		// Add the render element to the container root
		containerRoot.Children().InsertAtTop(m_renderElement);



		auto d3dDevice = CreateD3DDevice();
		auto dxgiDevice = d3dDevice.as<IDXGIDevice>();
		winrt::Windows::Graphics::DirectX::Direct3D11::IDirect3DDevice m_device = CreateDirect3DDevice(dxgiDevice.get());


		auto item = CreateCaptureItemForWindow(hwndTarget);

		std::unique_ptr<SimpleCapture> m_capture{ nullptr };
		m_capture = std::make_unique<SimpleCapture>(m_device, item);

		auto surface = m_capture->CreateSurface(compositor);
		m_brush.Surface(surface);

		m_capture->StartCapture();

	}


};

int CALLBACK WinMain(HINSTANCE instance, HINSTANCE previousInstance, LPSTR cmdLine, int cmdShow)
{

	classTest test;

	test.Test(instance);


	// Message pump
	MSG msg;
	while (GetMessage(&msg, NULL, 0, 0))
	{
		TranslateMessage(&msg);
		DispatchMessage(&msg);
	}

	return (int)msg.wParam;
}

Then the window will not draw the screenshot of the target window to capture.
The only difference here is that the while loop with the GetMessage is out of the Test function inside the class.

Now, If I move the while loop to the end of the Test function, it will work.
I have no idea why it is that..

It is blocking me from using the API.

Please help me to solve this problem.
I want that the message loop will be outside of the class while it still draw the window screenshot.

Thanks

I have recreated the example Acrylic Effect in WPF and made it to fit the entire window. But when i resize the window it flickers a lot and also a kind on lagging in resizing.

i have added a preview of my issue here

Click here to see the preview

Just notice the starting of the gif.

is there any way to fix it.i have noticed similar behavior while creating ACRYLIC_BLUR_BEHIND using setWindowCompositionAttribute() @microsoftopensource

Note: The Window that i use Currently use without the Acrylic blur does not flicker

Hi,

I use the CPP Hello Composition sample. Just change AddElement method in CompositionHost.cpp by this:

void CompositionHost::AddElement(float size, float x, float y)
{
	if (m_target.Root())
	{
		auto visuals = m_target.Root().as<ContainerVisual>().Children();

		// create a layer
		auto layer = m_compositor.CreateLayerVisual();
		layer.Size({ size, size });
		layer.Offset({ x, y, 0.0f, });

		// add a shadow
		auto shadow = m_compositor.CreateDropShadow();
		shadow.BlurRadius(10);
		shadow.Color({ 255, 100, 100, 100});
		shadow.SourcePolicy(CompositionDropShadowSourcePolicy::InheritFromVisualContent);
		layer.Shadow(shadow);

		// add a child element
		auto element = m_compositor.CreateSpriteVisual();
		uint8_t r = rand() % 255;
		uint8_t g = rand() % 255;
		uint8_t b = rand() % 255;
		element.Brush(m_compositor.CreateColorBrush({ 255, r, g, b }));
		element.Size({ size, size });

		// insert layer
		layer.Children().InsertAtTop(element);
		visuals.InsertAtTop(layer);
	}
}

When you run the sample and add elements, it doesn't look good. What's worse is if you open another window (for example you open Notepad) and move this new window around and over the sample window with elements, it creates very strange artifacts on drawn elements that look like a bug in the composition engine.

If I remove the shadow.SourcePolicy line or remove the shadow completely, it seems to work as expected.
Is there anything that can be done to fix this? any workaround?

I've not found a way to capture the screen because the system updates the surface properly as soon as we change focus, or capture the screen (print screen, etc.).

I'm using Windows 10 20H2 19042.789 (latest non insider) and compiled using Visual Studio 2019, x64.

Hello.
I noticed this issue when using the Win32CaptureSample project. When picking a window to share, there is a scenario where some windows capture item reports a size of 0 x 0, even if the window is restored and visible at the front. This will cause the function CreateSwapChainForComposition to throw a hresult_error exception. I can't say for sure that it is the size that causes the crash, but it is the difference between the windows that work and the ones that don't.

The window I've encountered this on 100% of the time is of the app "Inno Setup Compiler".

I tried this with the Win32CaptureSample running on SDK/Win10 version 10.0.19041.0.

I've tried to create the swap chain with a fixed size (e.g 1024x1024) in these scenarios, which does not produce a crash, but there is no output.

The thing that confuses me is that the picker is able to capture the window in the miniature view,. Correct me if I am wrong, but I was thinking that the view in the picker might be using the same capture item that is returned upon selection? (it's a wild guess).

Does anyone have any suggestions on how to work around this? Can you exclude windows from the picker, (which I would have to do in my own app if it fails, to let the user know that a window simply can't be captured)?

Additionally, trying to create a CaptureItem with the HWND of the app window in question creates a debug abort() error in "CreateForWindow", probably due to the same reason (although the HWND reports having the actual size of the window.)

As a slightly related question, is there a way to get the HWND from the Graphics Capture Item that the picker returns? The only way I can see is to use the item.DisplayName and do FindWindowA. I'm wondering if that is thread-safe and if the DisplayName will always be the same as the WindowName needed for FindWindowA, so that it can be a reliable solution?

Thanks in advance!
M

Hello, while debugging the reason why the codes

framePool = Direct3D11CaptureFramePool::Create(
	graphicDevice->device,
	DirectXPixelFormat::B8G8R8A8UIntNormalized,
	2,
	size);

captureSession = framePool.CreateCaptureSession(captureItem);


captureLastSize = size;
frameArrived = framePool.FrameArrived(auto_revoke, { this, &CaptureLayer_Win10API::Callback_OnFrameArrived });


.... 

captureSession.StartCapture();

void Callback_OnFrameArrived(
	winrt::Windows::Graphics::Capture::Direct3D11CaptureFramePool const& sender,
	winrt::Windows::Foundation::IInspectable const& args)
{

	if (isClosed) return;


	using namespace winrt;
	using namespace Windows::Graphics::DirectX;
	using namespace Windows::Graphics::DirectX::Direct3D11;

	auto frame = sender.TryGetNextFrame();

	return;
}

works slower than your original example on 4K (noticeable by eye on 4K only)

I found that the root cause is the 3d device I use.
I choose intentionally to create ID3D11Device on the dedicated GPU that is NVIDIA GTX 1050TI in my case.

I found that if the ID3D11Device is based on the integrated video that is INTEL GPU card, there is noticeable performance improvement in the capture rate.

Is there any reason why it works faster with a GPU that less powerful?

After startup NullReferenceException occured @ BarGraph.UpdateSize

My fix was in MainWindow.xaml.cs in constructor to initialize barGraphHost.DataContext using the first item.
Thank you!

Hello,

how is it possible to capture only user content of selected window in ScreenCapture project?

I want to achieve to capture window without Window's control bar on the top and capture just the user content area.
Similar approach is presented here using GetDC() instead of GetWindowDC() from User32

Thank you for your assistance!

Hi,
I am getting this runtime error when I try to run the dotnet WPF AcrylicEffect solution. I've selected Debug x64 build and I haven't altered any code.

I am using Visual Studio 2019 v 16.5.5
Windows 10 LTSC Version 1809 Build 17763

Hello,
I am using nodejs load dll A and then dll A load dll B which internal calls IGraphicsCaptureItemInterop.CreateForWindow but I encounter winrt::hresult_access_denied
when I using MFC just to load dll B internal calls IGraphicsCaptureItemInterop.CreateForWindow work fine.
is it a program permission problem?

In my WPF application, I want use window capture by hwnd, and also my application is using CEFSharp.Wpf, if I first invoke Cef.Initialize(), then call IGraphicsCaptureItemInterop.CreateForWindow() to get a GraphicsCaptureItem, it will throw a exception. If I don't invoke Cef.Initialize(), it work well.
By the way, use GraphicsCapturePicker.PickSingleItemAsync() to get a GraphicsCaptureItem will work well whatever use cef or not.
Is there any conflict between IGraphicsCaptureItemInterop.CreateForWindow and CEF?
I know IGraphicsCaptureItemInterop is a COM interface, but I can't find what it is actually.
My application is base on .net framework 4.7.2, and CEFSharp version is 79.1.360.
Hope someone give me some advice, thanks.

Second Question: How to make OnFrameArrived callback running in a non-UI thread?

Sample Code will not run:

   public Form1() : base()
    {
        InitializeComponent();

        // Get graphics device.
        **canvasDevice = CanvasDevice.GetSharedDevice();** => This line causes the exception

        // Create effect graph.
        acrylicEffect = CreateAcrylicEffectGraph();
    }

Exception:

System.IO.FileNotFoundException
HResult=0x8007007E
Message=(Module not found)Das angegebene Modul wurde nicht gefunden. (Ausnahme von HRESULT: 0x8007007E)
Source=mscorlib
StackTrace:
at System.StubHelpers.StubHelpers.GetWinRTFactoryObject(IntPtr pCPCMD)
at Microsoft.Graphics.Canvas.CanvasDevice.GetSharedDevice()
at AcrylicEffect.Form1..ctor() in G:\Data\Projects\git\c#\SampleCode\Windows.UI.Composition-Win32-Samples\dotnet\WinForms\AcrylicEffect\AcrylicEffect\Form1.cs:line 32
at AcrylicEffect.Program.Main() in G:\Data\Projects\git\c#\SampleCode\Windows.UI.Composition-Win32-Samples\dotnet\WinForms\AcrylicEffect\AcrylicEffect\Program.cs:line 19

The project compiles without problem on
VS 2019. I copied also Microsoft.Graphics.Canvas.dll to the debug folder
I installed all current updates.

I wonder what file or module can't be found ?

What were the changes to cursor capture in Windows 11?
I need to take a capture and show the image on the same computer. I used to hide the system cursor with ShowSystemCursor(FALSE), but the cursor was still captured and shown in the final image. Now on Windows 11 (22000.100), if I hide the cursor, it is also not captured, if I don’t hide it, I see 2 cursors.

Hello,
Is it possible to downscale the resolution of the capture frame from the API used in the example?

I want to reduce the resolution for example from 4K to 1080P without damaging performances too much.

I will appreciate any example or best instructions on how to do it efficiently.

I need to resize the actual pixels and not the way I display it on the screen.
I want at the end to get fewer pixels - instead of 4k, get 1920x1080 pixels

Thanks!

The following code does not compile anymore:

Windows::Foundation::Collections::IVectorView<Windows::AI::MachineLearning::ILearningModelFeatureDescriptor> inputFeatures; // OK, this does not compile, it's just for the example.
auto d = inputFeatures.GetAt(0); // But this compiled before and does not anymore.

With the following error log:

1>------ Build started: Project: DeepRaw2RGB, Configuration: Debug x64 ------
1>DeepRaw2RGBNetwork.cpp
1>D:\ggourdin\framework\src\Correction\src\LibDeepRaw2RGB\WinML\DeepRaw2RGBNetwork.cpp(232,27): error C3779: 'winrt::impl::consume_Windows_Foundation_Collections_IVectorView<winrt::Windows::Foundation::Collections::IVectorView<winrt::Windows::AI::MachineLearning::ILearningModelFeatureDescriptor>,T>::GetAt': a function that returns 'auto' cannot be used before it is defined
1>        with
1>        [
1>            T=winrt::Windows::AI::MachineLearning::ILearningModelFeatureDescriptor
1>        ]
1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\cppwinrt\winrt/impl/Windows.Foundation.Collections.0.h(575): message : see declaration of 'winrt::impl::consume_Windows_Foundation_Collections_IVectorView<winrt::Windows::Foundation::Collections::IVectorView<winrt::Windows::AI::MachineLearning::ILearningModelFeatureDescriptor>,T>::GetAt'
1>        with
1>        [
1>            T=winrt::Windows::AI::MachineLearning::ILearningModelFeatureDescriptor
1>        ]
1>D:\ggourdin\framework\src\Correction\src\LibDeepRaw2RGB\WinML\DeepRaw2RGBNetwork.cpp(228): message : while compiling class template member function 'bool `anonymous-namespace'::WinMLCode<`anonymous-namespace'::WinMLImplementation::kNative>::Pimpl::HasHalfFloatLayers(void) const'
1>D:\ggourdin\framework\src\Correction\src\LibDeepRaw2RGB\WinML\DeepRaw2RGBNetwork.cpp(170): message : see reference to function template instantiation 'bool `anonymous-namespace'::WinMLCode<`anonymous-namespace'::WinMLImplementation::kNative>::Pimpl::HasHalfFloatLayers(void) const' being compiled
1>D:\Softwares\VS2019\VC\Tools\MSVC\14.27.29110\include\type_traits(819): message : see reference to class template instantiation '`anonymous-namespace'::WinMLCode<`anonymous-namespace'::WinMLImplementation::kNative>::Pimpl' being compiled
1>D:\Softwares\VS2019\VC\Tools\MSVC\14.27.29110\include\variant(956): message : see reference to class template instantiation 'std::is_trivially_destructible<DxO::DeepRaw2RGB::DeepRaw2RGBNetwork::Pimpl::WindowsImpl>' being compiled
1>D:\Softwares\VS2019\VC\Tools\MSVC\14.27.29110\include\variant(956): message : see reference to variable template 'const bool conjunction_v<std::is_trivially_destructible<`anonymous namespace'::WinMLCode<0>::Pimpl>,std::is_trivially_destructible<`anonymous namespace'::WinMLCode<1>::Pimpl> >' being compiled
1>D:\Softwares\VS2019\VC\Tools\MSVC\14.27.29110\include\variant(1011): message : see reference to alias template instantiation 'std::_Variant_destroy_layer<DxO::DeepRaw2RGB::DeepRaw2RGBNetwork::Pimpl::WindowsImpl,DxO::DeepRaw2RGB::DeepRaw2RGBNetwork::Pimpl::MicrosoftImpl>' being compiled
1>D:\ggourdin\framework\src\Correction\src\LibDeepRaw2RGB\WinML\DeepRaw2RGBNetwork.cpp(295): message : see reference to class template instantiation 'std::variant<DxO::DeepRaw2RGB::DeepRaw2RGBNetwork::Pimpl::WindowsImpl,DxO::DeepRaw2RGB::DeepRaw2RGBNetwork::Pimpl::MicrosoftImpl>' being compiled

Any clues?

Thank you.

Guillaume.

Hello, thank you very much for the new API to capture windows (cpp\ScreenCaptureforHWND) .

I will be very grateful to you if you add option to disable the yellow border (in addition to the option I will make to disable mouse capture). Or at least change the color of the border to different color

I am talking about:

Thanks.

Hi, i was trying to implement the acylic effect in c++ winrt but when i call the function createHostBackdropBrush() , it only creates a black visual it is not showing the content behind the window. I don't know why ? tried multiple times , seems not working did anyone have any examples which implements this in c++. Or is it not supported ?

I get the following errors

C2144 syntax error: 'IDXGIAdapter4' should be preceded by ';' AdvancedColorImages C:\Program Files (x86)\Windows Kits\10\Include\10.0.20348.0\shared\dxgi1_6.h 52

C4430 missing type specifier - int assumed. Note: C++ does not support default-int AdvancedColorImages C:\Program Files (x86)\Windows Kits\10\Include\10.0.20348.0\shared\dxgi1_6.h 52

Also, Intellisense is showing an error at line 144 of AdvancedColorImages.cpp

m_winComp = new WinComp(); // object of abstract type WinComp is not allowed

The dotnet WPF HelloComposition example is great starting point for an application I need to develop, but I would like to know how to display an image behind the rectangles. For example, I have an image of people and I want to draw a box around each persons face. The image will be updated at 60 fps coming from a camera. The image comes in as a IntPtr to a byte array, of which i know the image width, height, and stride so it can be converted to a Bitmap. I have successfully taking in the images and displayed them on a WindowsFormHost control with a picturebox. But that is not ideal and I don't know how to draw boxes and such on that in realtime. The visual layer seems like it would have the performance I desire since it uses the GPU. I would like to display the images, and draw sprite boxes at the location of each persons face. I have an algorithm to calculate the x,y coordinates and sizes of the boxes. If the IntPtr thing is too complicated, for now i would be happy with how to load a .bmp file from disk, at runtime, and display it on the CompositionHostElement, and then add random sprite boxes on top as the demo currently does.

Any help with this would be greatly appreciated.

Hi,

I would like to try to send the captured Texture2D over the network.
For this, I need to send a byte array;

Could you provide an example of how could I get the Bitmap from :

using (var backBuffer = swapChain.GetBackBuffer<SharpDX.Direct3D11.Texture2D>(0))
using (var bitmap = Direct3D11Helper.CreateSharpDXTexture2D(frame.Surface))
{
    d3dDevice.ImmediateContext.CopyResource(bitmap, backBuffer);
    //How to convert the backBuffer to Bitmap or byte[] ?
}

Occasionally, calling SimpleCapture::Close() will crash. It seems to happen only when I have too many window captures running at the same time. I have multiple SimpleCapture instance, each capturing one different window.

This might be relevant to the same issue.
And here is my .dump file

Let me know if you need any more information.

I've just cloned the repo, opened the HelloVectors solution file and upgraded to SDK 10.0.18362.0 and toolset v142, and attempted to build. I get the following errors:

1>------ Build started: Project: HelloVectors_win32_cpp, Configuration: Debug x64 ------ 1>pch.cpp 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(980,26): error C2039: 'wait_for': is not a member of 'winrt::impl' 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(100): message : see declaration of 'winrt::impl' 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(982): message : see reference to class template instantiation 'winrt::impl::consume_Windows_Foundation_IAsyncAction<D>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(9818): message : see reference to class template instantiation 'winrt::com_ptr<winrt::impl::IContextCallback>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(9590): message : see reference to class template instantiation 'winrt::com_ptr<winrt::impl::IServerSecurity>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(9549): message : see reference to class template instantiation 'std::chrono::time_point<winrt::clock,winrt::Windows::Foundation::TimeSpan>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(2117): message : see reference to class template instantiation 'winrt::com_ptr<To>' being compiled 1> with 1> [ 1> To=winrt::impl::IStaticLifetimeCollection 1> ] 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(8969): message : see reference to function template instantiation 'winrt::com_ptr<To> winrt::Windows::Foundation::IUnknown::as<winrt::impl::IStaticLifetimeCollection>(void) const' being compiled 1> with 1> [ 1> To=winrt::impl::IStaticLifetimeCollection 1> ] 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(8966): message : see reference to class template instantiation 'winrt::com_ptr<winrt::impl::IStaticLifetime>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(5215): message : see reference to class template instantiation 'winrt::com_ptr<winrt::impl::IMarshal>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(2460): message : see reference to class template instantiation 'winrt::com_ptr<To>' being compiled 1> with 1> [ 1> To=winrt::impl::ILanguageExceptionErrorInfo2 1> ] 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(4097): message : see reference to function template instantiation 'winrt::com_ptr<To> winrt::com_ptr<winrt::impl::IRestrictedErrorInfo>::try_as<winrt::impl::ILanguageExceptionErrorInfo2>(void) noexcept const' being compiled 1> with 1> [ 1> To=winrt::impl::ILanguageExceptionErrorInfo2 1> ] 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\base.h(4179): message : see reference to class template instantiation 'winrt::com_ptr<winrt::impl::IRestrictedErrorInfo>' being compiled 1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.24.28314\include\xstring(1743): message : see reference to class template instantiation 'std::basic_string_view<wchar_t,std::char_traits<wchar_t>>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(1001,26): error C2039: 'wait_for': is not a member of 'winrt::impl' 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(100): message : see declaration of 'winrt::impl' 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(1003): message : see reference to class template instantiation 'winrt::impl::consume_Windows_Foundation_IAsyncActionWithProgress<D,TProgress>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(1035,26): error C2039: 'wait_for': is not a member of 'winrt::impl' 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(100): message : see declaration of 'winrt::impl' 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(1037): message : see reference to class template instantiation 'winrt::impl::consume_Windows_Foundation_IAsyncOperationWithProgress<D,TResult,TProgress>' being compiled 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(1054,26): error C2039: 'wait_for': is not a member of 'winrt::impl' 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(100): message : see declaration of 'winrt::impl' 1>C:\...\Windows.UI.Composition-Win32-Samples\cpp\HelloVectors\x64\Debug\Generated Files\winrt\impl\Windows.Foundation.0.h(1056): message : see reference to class template instantiation 'winrt::impl::consume_Windows_Foundation_IAsyncOperation<D,TResult>' being compiled

Is it possible to grab the captured screen "texture" and render it to a BitmapImage in memory?

Here is some context on what I want to do and what I tried already.

I am trying to capture the contents of a WPF Window (not UWP) and convert it to a BitmapImage, specifically a System.Windows.Media.Imaging.BitmapImage (WPF). This BitmapImage then allows me to send the pixel data over a network for display on another PC. I would like to do this in real-time around 60 frames per second.

At the moment, in WPF, I am using the RenderTargetBitmap.Render(Visual) method. It works and is very easy, but it is simply not fast enough to reach 60 fps. If the visual tree becomes even somewhat complex it slows down to below 5 fps.

I found this new UWP composition Screen Capture sample which looks promising, as it is able to capture the same WPF window very smoothly. If I can somehow "grab" the captured image and get the raw pixels, then perhaps I can use this as an alternative to WPF RenderTargetBitmap.

I believe the place to "hook in" would be in the BasicCapture class, where the texture is being copied:

            using (var backBuffer = swapChain.GetBackBuffer<SharpDX.Direct3D11.Texture2D>(0))
            using (var bitmap = Direct3D11Helper.CreateSharpDXTexture2D(frame.Surface))
            {
                d3dDevice.ImmediateContext.CopyResource(bitmap, backBuffer);

                // My addition:
                GetBitmap(bitmap);
            }

The big question now is: how do I convert this Texture2D to a "raw pixel data" or preferably BitmapImage that I can further consume in my WPF app?

With my current "GetBitmap" code, I am able to write the image to a file on disk:

   private void GetBitmap(Texture2D texture)
    {
        // Create texture copy
        var copy = new Texture2D(d3dDevice, new Texture2DDescription
        {
            Width = texture.Description.Width,
            Height = texture.Description.Height,
            MipLevels = 1,
            ArraySize = 1,
            Format = texture.Description.Format,
            Usage = ResourceUsage.Staging,
            SampleDescription = new SampleDescription(1, 0),
            BindFlags = BindFlags.None,
            CpuAccessFlags = CpuAccessFlags.Read,
            OptionFlags = ResourceOptionFlags.None
        });

        // Copy data
        d3dDevice.ImmediateContext.CopyResource(texture, copy);

        var dataBox = d3dDevice.ImmediateContext.MapSubresource(copy, 0, 0, MapMode.Read, MapFlags.None,
            out DataStream stream);
        var rect = new DataRectangle
        {
            DataPointer = stream.DataPointer,
            Pitch = dataBox.RowPitch
        };

        var format = PixelFormat.Format32bppPBGRA;
        Bitmap bmp = new Bitmap(factory, copy.Description.Width, copy.Description.Height, format, rect);

        using (var wic = new WICStream(factory, "test_output.png", NativeFileAccess.ReadWrite))
        using (var encoder = new PngBitmapEncoder(factory, wic))
        using (var frame = new BitmapFrameEncode(encoder))
        {
            frame.Initialize();
            frame.SetSize(bmp.Size.Width, bmp.Size.Height);
            frame.SetPixelFormat(ref format);
            frame.WriteSource(bmp);
            frame.Commit();
            encoder.Commit();
        }

        d3dDevice.ImmediateContext.UnmapSubresource(copy, 0);
        copy.Dispose();
        bmp.Dispose();
    }
}

This addition seems to add about 20-40 ms of delay each frame, which is a bit much, but I'm hopeful a large part of that is actually saving the file. I don't need to save the file to disk, I just need it in memory so I can get the pixel data.

I tried using a InMemoryRandomAccessStream and creating a UWP BitmapImage from that, but there is a threading issue and it does not allow me to create a BitmapImage in this code. Even if I could, I don't see a way to convert the UWP BitmapImage to my WPF BitmapImage which I require.

To clarify, in the end what I need is to use the BitmapSource.CopyPixels method:
https://docs.microsoft.com/en-us/dotnet/api/system.windows.media.imaging.bitmapsource.copypixels?view=netframework-4.8#System_Windows_Media_Imaging_BitmapSource_CopyPixels_System_Windows_Int32Rect_System_IntPtr_System_Int32_System_Int32_

This allows me to copy the pixels on a BitmapSource to the right memory location where it is being send over the network.

If I can somehow get this pixel data without going through a BitmapImage that is fine too, I simply cannot find a way.

Does anyone see a way to make this possible or am I chasing an impossible dream?

Existing code looks like this:

const winrt::Windows::Graphics::Capture::GraphicsCaptureSession session = frame_pool.CreateCaptureSession(item);

We will probably want to do something like this:

if (???) { session.IsCursorCaptureEnabled(cursor_enabled); }

What's the correct test here? Stepping back, is this the correct test for determining if window capture is supported at all?

return winrt::Windows::Foundation::Metadata::ApiInformation::IsApiContractPresent(L"Windows.Foundation.UniversalApiContract", 8);

I am starting from the ScreenCapture WPF sample, and trying to implement a feature that will allow screen capture of the full display excluding the WPF window that i have. I am using the SetWindowDisplayAffinity(handle, WDA_EXCLUDEFROMCAPTURE) in the WPF window to skip the window from screen capture (since i want to skip the window completely, instead of having the black region).

In a regular WPF Window, the approach works fine and I am able to capture the screen skipping the WPF window. However, when the WPF window has AllowTransparenacy enabled, the SetWindowDisplayAffinity returns 0 and the screencapture retains the WPF window.

Is this a known issue? Is there a way to work around this, i.e. to skip a semi transparent WPF window that the application owns from the screen capture?

Thanks in advance,