Giter Club home page Giter Club logo

Comments (8)

robmikh avatar robmikh commented on May 17, 2024 2

When you call bitmap.Save(ms, ImageFormat.Bmp), I'm guessing that's making a copy of bitmap into your MemoryStream. I'm also a bit surprised this works... to my eye you're mixing pixel formats (B8G8R8A8_UNorm vs Format32bppArgb).

Keep in mind that the WPF capture sample you're pointing to doesn't try to access the bits in system memory. Everything is kept in video memory, so it will have different performance characteristics from what you're trying to do.

Does this project have the same issue for you?
https://github.com/robmikh/ManagedScreenshotDemo

from windows.ui.composition-win32-samples.

keytrap-x86 avatar keytrap-x86 commented on May 17, 2024

Okay so I have found a way of converting the backBuffer to Bitmap (or byte array) :

//...
// Get the desktop capture texture
var mapSource = device.ImmediateContext.MapSubresource(screenTexture, 0, MapMode.Read, SharpDX.Direct3D11.MapFlags.None);

// Create Drawing.Bitmap
using(var bitmap = new Bitmap(width, height, PixelFormat.Format32bppArgb)) {
  var boundsRect = new Rectangle(0, 0, width, height);

  // Copy pixels from screen capture Texture to GDI bitmap
  var mapDest = bitmap.LockBits(boundsRect, ImageLockMode.WriteOnly, bitmap.PixelFormat);

  var sourcePtr = mapSource.DataPointer;
  var destPtr = mapDest.Scan0;
  for (int y = 0; y < height; y++) {
    // Copy a single line 
    Utilities.CopyMemory(destPtr, sourcePtr, width * 4);

    // Advance pointers
    sourcePtr = IntPtr.Add(sourcePtr, mapSource.RowPitch);
    destPtr = IntPtr.Add(destPtr, mapDest.Stride);
  }

  // Release source and dest locks
  bitmap.UnlockBits(mapDest);
  device.ImmediateContext.UnmapSubresource(screenTexture, 0);

  using(var ms = new MemoryStream()) {
    bitmap.Save(ms, ImageFormat.Bmp);
    ScreenRefreshed?.Invoke(this, (Bitmap) bitmap.Clone());
    _init = true;
  }
}

The problem is that all this conversion is very costly. My CPU get's up to 30%

Any ways of improving this ?

from windows.ui.composition-win32-samples.

robmikh avatar robmikh commented on May 17, 2024

It looks like you may be doing multiple copies. Here's the approach I would try:

  1. Create a staging texture with the same dimensions as the buffers in the frame pool. Make sure to keep this the same size if you chose to change the buffer size when calling Recreate.
  2. After calling TryGetNextFrame from the FrameArrived event handler, copy the frame to the staging texture.
  3. Map the staging texture.

From here you can copy out each line to either a new allocation or something you reuse (depending on how you're transmitting the frame). Take a look at this comment from another issue/sample for some sample C++ code.

At the end, unmap the texture.

from windows.ui.composition-win32-samples.

keytrap-x86 avatar keytrap-x86 commented on May 17, 2024

@robmihk, thank you for you answer.
Although, for me, what you are advising me to do, is already what I am doing.
Here's the full sample :

public void Start()
        {
            _run = true;
            var factory = new Factory1();
            //Get first adapter
            var adapter = factory.GetAdapter1(0);
            //Get device from adapter
            var device = new SharpDX.Direct3D11.Device(adapter);
            //Get front buffer of the adapter
            var output = adapter.GetOutput(0);
            var output1 = output.QueryInterface<Output1>();

            // Width/Height of desktop to capture
            int width = output.Description.DesktopBounds.Right;
            int height = output.Description.DesktopBounds.Bottom;

            // Create Staging texture CPU-accessible
            var textureDesc = new Texture2DDescription
            {
                CpuAccessFlags = CpuAccessFlags.Read,
                BindFlags = BindFlags.None,
                Format = Format.B8G8R8A8_UNorm,
                Width = width,
                Height = height,
                OptionFlags = ResourceOptionFlags.None,
                MipLevels = 1,
                ArraySize = 1,
                SampleDescription = { Count = 1, Quality = 0 },
                Usage = ResourceUsage.Staging
            };
            var screenTexture = new Texture2D(device, textureDesc);

            _ = Task.Factory.StartNew(() =>
              {
                  // Duplicate the output
                  using (var duplicatedOutput = output1.DuplicateOutput(device))
                  {
                      while (_run)
                      {
                          try
                          {

                              // Try to get duplicated frame within given time is ms
                              duplicatedOutput.AcquireNextFrame(500, out OutputDuplicateFrameInformation duplicateFrameInformation, out SharpDX.DXGI.Resource screenResource);

                              // copy resource into memory that can be accessed by the CPU
                              using (var screenTexture2D = screenResource.QueryInterface<Texture2D>())
                                  device.ImmediateContext.CopyResource(screenTexture2D, screenTexture);

                              // Get the desktop capture texture
                              var mapSource = device.ImmediateContext.MapSubresource(screenTexture, 0, MapMode.Read, SharpDX.Direct3D11.MapFlags.None);

                              // Create Drawing.Bitmap
                              using (var bitmap = new Bitmap(width, height, PixelFormat.Format32bppArgb))
                              {
                                  var boundsRect = new Rectangle(0, 0, width, height);

                                  // Copy pixels from screen capture Texture to GDI bitmap
                                  var mapDest = bitmap.LockBits(boundsRect, ImageLockMode.WriteOnly, bitmap.PixelFormat);

                                  var sourcePtr = mapSource.DataPointer;
                                  var destPtr = mapDest.Scan0;
                                  for (int y = 0; y < height; y++)
                                  {
                                      // Copy a single line 
                                      Utilities.CopyMemory(destPtr, sourcePtr, width * 4);

                                      // Advance pointers
                                      sourcePtr = IntPtr.Add(sourcePtr, mapSource.RowPitch);
                                      destPtr = IntPtr.Add(destPtr, mapDest.Stride);
                                  }

                                  // Release source and dest locks
                                  bitmap.UnlockBits(mapDest);
                                  device.ImmediateContext.UnmapSubresource(screenTexture, 0);

                                  using (var ms = new MemoryStream())
                                  {
                                      bitmap.Save(ms, ImageFormat.Bmp);
                                      ScreenRefreshed?.Invoke(this, (Bitmap)bitmap.Clone());
                                      _init = true;
                                  }
                              }
                              screenResource.Dispose();
                              duplicatedOutput.ReleaseFrame();
                          }
                          catch (SharpDXException e)
                          {
                              if (e.ResultCode.Code != SharpDX.DXGI.ResultCode.WaitTimeout.Result.Code)
                              {
                                  Trace.TraceError(e.Message);
                                  Trace.TraceError(e.StackTrace);
                              }
                          }
                      }
                  }
              });
            while (!_init) ;
        }

This works of course, but as I said, the CPU gets up to 30%.
I'vre tried commenting all ou excep the

var mapSource = device.ImmediateContext.MapSubresource(screenTexture, 0, MapMode.Read, SharpDX.Direct3D11.MapFlags.None);
...
device.ImmediateContext.UnmapSubresource(screenTexture, 0);

But even then, it's quite CPU intensive. Is this the only way of getting the data from the CPU ?

image

from windows.ui.composition-win32-samples.

robmikh avatar robmikh commented on May 17, 2024

It still looks like you're making multiple copies to me. I would try this in C++ and see if you get the same result. I'm not sure what some of those helper methods are doing under the covers.

from windows.ui.composition-win32-samples.

keytrap-x86 avatar keytrap-x86 commented on May 17, 2024

Could you elaborate ? In which way is this making multiple copies ? When I look at SharpDx's example : This is how a single screen capture works (basic screenshot). I'm just trying to make multiples screenshots and convert them to byte array for sending on the wire.

Also, in this example, everything works fine and there's no performance issue. Now of course, for me, there's no point of just showing the captured screen in a form. I don't understand how this works fine and as soon I try to get a Bitmap from it, the cpu goes crazy

from windows.ui.composition-win32-samples.

TKGNET avatar TKGNET commented on May 17, 2024

Since the OP seems to have abandonded the thread, I wanted to leave my regards for robmikH sample solution.
Way faster and definitely more elegant than my own LockBits solution:

  • converting stagingResource -> byte array ~ 20ms per screenshot
  • BitMapEncoder -> file takes ~ 80ms per file on top

I just wish there was a pure in-memory solution to convert stagingResource to drawing.bitmap , other than file export/import. But I guess writing to an InMemoryRandomAccessStream might do the trick.

from windows.ui.composition-win32-samples.

EricBallard avatar EricBallard commented on May 17, 2024

+1 thanks for the demo

from windows.ui.composition-win32-samples.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.