Rationale for creating a copy in makeQImage about pyqtgraph HOT 9 CLOSED

@joshdoe Between #1466 and #1501 (which the two combine to supersede #669) I think we can close this issue out. If there are still performance related issues discussed here that anyone feels should be adopted, I'll re-open.

from pyqtgraph.

The rationale for the default value is that 1) it is the safest option, since the image data may be modified after calling makeQImage, and 2) to preserve backward compatibility. So I don't think it would be a good idea to change the default behavior of makeQImage.

I am in favor, though, of modifying ImageItem to be more clever about whether it asks makeQImage to copy data, and also for adding an option to make this behavior configurable.

from pyqtgraph.

Most calls to makeQImage actually pass an intermediate array to which there is no external reference, either built by makeARGB (ImageItem.render, RawImageWidget.paintEvent), by glReadPixels (GLViewWidget.readQImage) or by hand (ImageExporter.export, SymbolAtlas.getAtlas), so we shouldn't need to worry about whether this array gets modified (and modifications to the user-supplied array won't get reflected into QImage's internal array either). The only place where an externally accessible array is passed to makeQImage is in SymbolAtlas.getAtlas, which sets copy to False anyways (for performance reasons, I guess).

So as long as makeARGB guarantees that the array it returns doesn't share its input's buffer, we should be safe, right? (This does not solve the backwards compatibility issue but you can always use an specify copy=False in all these places instead or define an internal _makeQImage_nocopy function.)

from pyqtgraph.

I am more concerned about applications external to pyqtgraph that may already use makeQImage. Not that I know of any, but I try to minimize making API changes that could result in somebody spending hours chasing a new bug. I would much rather change all of the places internal to pyqtgraph that use makeQImage.

That said: I do not notice any speed difference in VideoSpeedTest.py when I switch to using copy=False in ImageItem (using the latest from devel, which has makeQImage(..., transpose=False) in ImageItem).

Furthermore, using copy=False introduces a memory leak because it tries to make a QImage with write access to the original numpy array, and I still have not worked out how to do this for all versions of PyQt. This is actually a separate issue from copying, though, so perhaps I should add a 'mutable' argument to keep those separate..

from pyqtgraph.

I found this thread while searching for bottleneck in my own code. I have an application that uses gstreamer to stream live video from a camera to a ViewBox. It looks like a lot of the time is coming from calls to ImageItem.render() and

        argb, alpha = fn.makeARGB(image, lut=lut, levels=levels)
        self.qimage = fn.makeQImage(argb, alpha, transpose=False)

Is there a way to not recreate the self.qimage each time, but to just update the data? I'm using np.ndarray(buffer=mapinfo.data,, ...) to use the data directly off of the gstreamer pipeline as my raw input instead of making a copy there.

This hasn't been a problem for video that is only about 1280x1024 (30Hz), but I have a 4K camera, and I'm definitely noticing latency issues on the display. I can capture the data through the pipeline, but the display of the data can't keep up. There is another application we have not using pyqtgraph (Qt5, but C-based) that is not having this latency issue.

I haven't tried using RawImageWidget yet, but when I run the VideoSpeedTest.py example, RawImageWidget is slower that ImageItem.

from pyqtgraph.

Definitely agree we should be able to get much better performance here. If I use just QGraphicsPixmapItem on its own, I can get ~85fps displaying a 4k video stream fullscreen. Using ImageItem, this goes down to ~35fps and it looks more like 3fps.

Here's my benchmark:

from timeit import default_timer as time
from PyQt5.QtWidgets import *
from PyQt5.QtGui import *
import pyqtgraph as pg
import numpy as np

app = QApplication([])

window_size = 1200, 1000
W = 4000
data = np.random.randint(255, size=(10, W, W, 4), dtype='ubyte')
data[...,3] = 255


def runTest(maxPrints=1):
    global data
    dt_avg = None
    last_print = time()
    prints = 0
    while True:
        i = np.random.randint(data.shape[0])
        framestart = time()
        updateImage(data[i])
        app.processEvents()
        dt = time() - framestart
        s = 0.01
        dt_avg = dt if dt_avg is None else (1-s) * dt_avg + s * dt
        if time() - last_print > 2:
            last_print = time()
            print("%0.2f fps" % (1/dt_avg))
            prints += 1
        if prints >= maxPrints:
            break


# Set up the fastest possible Qt display

view = QGraphicsView()
scene = QGraphicsScene()
view.setScene(scene)
view.show()

image = QGraphicsPixmapItem()
scene.addItem(image)

def updateImage(data):
    global image, qimg, pixmap
    #qimg = pg.makeQImage(data)
    qimg = QImage(data.ctypes.data, data.shape[1], data.shape[0], QImage.Format_RGB32)
    pixmap = QPixmap(qimg)
    image.setPixmap(pixmap)

updateImage(data[0])
view.resize(*window_size)
view.fitInView(image)

print("Fast benchmark:")
runTest()
        
view.close()


# Now test pyqtgraph

view = pg.GraphicsLayoutWidget()
view.resize(*window_size)
view.show()
vb = view.addViewBox()
image = pg.ImageItem()
vb.addItem(image)

def updateImage(data):
    global image
    image.setImage(data, copy=False)
    
print("ImageItem benchmark:")
runTest()

from pyqtgraph.

Luke,
Thanks for the benchmark. I copied this and ran it under Spyder (python 3.6.4) and got these results:

Fast benchmark:
191.94 fps
ImageItem benchmark:
1.44 fps

The video data I have can be either RGBA, 8-bit gray scale, or 16-bit grayscale as a Bayer pattern, so I still need to get this into ARGB, but I may be able to do this in the gstreamer pipeline before I get to the display. I'm also using a HistogramLutItem, so I'm thinking I may be forced to use an ImageItem anyway? (I may be just wanting too much.)

from pyqtgraph.

I'd also like to see how to improve the performance of higher throughput videos. VideoSpeedTest at 1080p I get 47fps for mono uint8, but only 17fps for RGB8. RawImageWidget is even slower for me. This is on Windows 7.

from pyqtgraph.

Should link this issue to the #669 PR, even though it's still a WIP, it's increased performance by about 6x for my application.

from pyqtgraph.