Performance of ImageWriter/JpegWriter about scrimage HOT 16 CLOSED

Can you upload an image you are using so I can profile.

from scrimage.

You're also quite a few versions old. You should bump to at least the latest 1.3.x release, although that won't speed you up in this case.

from scrimage.

Sure, here's a test image (1200x1200):

from scrimage.

Actually, I guess this is the image actually being written to bytes (but I get similar results with either):

from scrimage.

So using your image, I wrote a quick benchmark.

object JpegWriterBenchmarks extends App {

  val image = Image(getClass.getResourceAsStream("/issue38.jpeg"))

  println("100 iterations of jpeg writing using scrimage")
  var size = 0
  val result1 = benchmark(100) {
    val bytes = image.write(Format.JPEG)
    size = size + bytes.length
  }
  println(result1)

  println("100 iterations of jpeg writing using ImageIO")
  size = 0
  val result2 = benchmark(100) {
    val bytes = new ByteArrayOutputStream
    ImageIO.write(image.awt, "JPEG", bytes)
    size = size + bytes.toByteArray.length
  }
  println(result2)
}

100 iterations of jpeg writing using scrimage
0 warmup runs
100 benchmarked iterations
Total executed time: 4816 millis
Average time per iteration: 48 millis
20.76 iterations per second

100 iterations of jpeg writing using ImageIO
0 warmup runs
100 benchmarked iterations
Total executed time: 5099 millis
Average time per iteration: 50 millis
19.61 iterations per second

Doesn't look like scrimage is adding any overhead. I'm not sure what else we can do to speed up JPEG writing. Perhaps we can try the apache imaging writer and if that compares better then use that instead.

from scrimage.

For what its worth, the other bottleneck I found in profiling was Image(inputStream) — originally I thought this was on Finagle's end: res.withInputStream[Image](inputStream => Image(inputStream)) but breaking it down further:

profile("withInputStream") { // 63.038ms
  res.withInputStream[Image] {
    inputStream => {
      profile("Image(inputStream)") { Image(inputStream) } // 62.398ms (98.98%)
    }
  }
}

So it sounds like I need to look into (de)serializing these differently. Possibly using the apache imaging writer you mention.

EDIT: I should note that the ~60ms fluctuates quite a bit between ~40ms to ~100ms. 60ms was pretty average. But between this and the original post at the top makes up most of the request time to the image server that doesn't include fetching the source images.

from scrimage.

What library are you using for profiling btw?

On 24 June 2014 23:37, Jim Fleming [email protected] wrote:

For what its worth, the other bottleneck I found in profiling was
Image(inputStream) — originally I thought this was on Finagle's end: res.withInputStream[Image](inputStream
=> Image%28inputStream%29) but breaking it down further:

profile("withInputStream") { // 63.038ms
res.withInputStream[Image] {
inputStream => {
profile("Image(inputStream)") { Image(inputStream) } // 62.398ms (98.98%)
}
}
}

So it sounds like I need to look into (de)serializing these differently.
Possibly using the apache imaging writer you mention.

—
Reply to this email directly or view it on GitHub
#38 (comment).

from scrimage.

Just a simple fn (admittedly discovered on StackOverflow but it seems to work):

def profile[R](name: String)(block: => R): R = {
    val t0: Long = System.nanoTime()
    val result = block
    val t1: Long = System.nanoTime()
    val diff: Float = ((t1 - t0).toFloat / 1000.0f) / 1000.0f
    log.info(s"$name: ${diff}ms")
    result
}

Its not so much profiling as annotating/instrumenting. We're working on getting Zipkin tracing setup.

from scrimage.

Nice.

On 24 June 2014 23:39, Jim Fleming [email protected] wrote:

Just a simple fn (admittedly discovered on StackOverflow but it seems to
work):

def profile[R](name: String)(block: => R): R = {
val t0: Long = System.nanoTime()
val result = block
val t1: Long = System.nanoTime()
val diff: Float = ((t1 - t0).toFloat / 1000.0f) / 1000.0f
log.info(s"$name: ${diff}ms")
result}

—
Reply to this email directly or view it on GitHub
#38 (comment).

from scrimage.

Tried out the Apache Imaging writers and they weren't any faster (actually slower by about 20-50%).

from scrimage.

You could investigate these:

http://www.scimersion.com/jmol-13.0.2/src/org/jmol/util/JpegEncoder.java
https://code.google.com/p/jj2000/
http://www.media.mit.edu/pia/Research/deepview/src/JpegEncoder.java
http://www.cs.mcgill.ca/~zcao7/mutls/release/llvm-gcc-4.2-2.9.source/libjava/classpath/gnu/javax/imageio/jpeg/JPEGDecoder.java

On 30 June 2014 21:00, Jim Fleming [email protected] wrote:

Tried out the Apache Imaging writers and they weren't any faster (actually
slower by about 20-50%).

—
Reply to this email directly or view it on GitHub
#38 (comment).

from scrimage.

Awesome, thanks! I'll try those out.

from scrimage.

If any of them are significantly better, then we can team up and re-write
them in Scala, and add them to Scrimage. They're all basically one file
implementations and byte manipulations so should convert quickly.

On 30 June 2014 21:17, Jim Fleming [email protected] wrote:

Awesome, thanks! I'll try those out.

—
Reply to this email directly or view it on GitHub
#38 (comment).

from scrimage.

Did any of those other readers/writers turn out to be any use ?

from scrimage.

I honestly haven't gotten to them yet. We opted for increasing our varnish cache size in the meantime :)

from scrimage.

ahh ok

On 11 July 2014 22:15, Jim Fleming [email protected] wrote:

I honestly haven't gotten to them yet. We opted for increasing our varnish
cache size in the meantime :)

—
Reply to this email directly or view it on GitHub
#38 (comment).

from scrimage.