Plotting array with multiple curves (request) about holoviews HOT 10 CLOSED

I think this point is addressed in the last point on the FAQ under the question "Q: How do I create a Layout or Overlay object from an arbitrary list?".

In short, the safest way to create an Overlay from a list is as as follows:

hv.Overlay.from_values(list_curves)

If you know the items in the list are all Elements which is true in this case (here you have a list of Curves) then you can do:

hv.Overlay(list_curves)

This won't work if your list contains holomaps which is why the Overlay.from_values classmethod is always the safest approach.

from holoviews.

The method you suggest is extremely slow.

When I plot my data by tranforming my array into a (N, 2) array:

%%timeit
phase_boundaries = {(phi, theta):
                 hv.Curve([(B, mu) for mu_list in mus[phi, theta].T for B, mu in zip(Bs, mu_list)])
                 for theta in range(len(thetas)) 
                 for phi in range(len(phis))}

1 loops, best of 3: 196 ms per loop

But when I use your suggested method:

%%timeit
phase_boundaries = {(phi, theta):
                 hv.Overlay([hv.Curve(zip(Bs, mus[phi, theta, :, i])) for i in range(10)])
                 for theta in range(len(thetas)) 
                 for phi in range(len(phis))}
1 loops, best of 3: 3.85 s per loop

And these are just 10 lines plotted for different angles.

The problem with the first piece of code is that it works in this case, but when I tried to make the example in my initial post, this method didn't work for a non obvious reason.

from holoviews.

Also making a HoloMap with the "slow" method takes 52 seconds, while the "fast" method (that doesn't always work) takes only 21 seconds.

from holoviews.

I agree that those numbers are excessively slow. I am not sure what exactly is causing the slowdown in this case but I do know that Philipp recently pushed a small fix that resulted in a major improvement in performance in certain situations (related to string sanitization).

Are you using the latest development version on master or the release version of HoloViews? If you are using the latter, it may be useful to know if the situation has improved.

from holoviews.

Got a bit faster on the latest master.

%%timeit
phase_boundaries = {(phi, theta):
                 hv.Curve([(B, mu) for mu_list in mus[phi, theta].T for B, mu in zip(Bs, mu_list)])
                 for theta in range(len(thetas)) 
                 for phi in range(len(phis))}
10 loops, best of 3: 161 ms per loop

%%timeit
phase_boundaries = {(phi, theta):
                 hv.Overlay([hv.Curve(zip(Bs, mus[phi, theta, :, i])) for i in range(10)])
                 for theta in range(len(thetas)) 
                 for phi in range(len(phis))}
1 loops, best of 3: 2.52 s per loop

from holoviews.

Good to know there is some improvement!

Assuming the number of thetas and phis stays the constant, the second %%timeit example is creating 10x more curves than the first example. From those numbers, lets say that takes approximately 1.6 seconds leaving the remaining 0.9 seconds for creating the Overlays.

What I would like to know is how many thetas and phis are used here! As you are effectively doing a cross-product, the number of overlays generated could easily be quite large. I would like to know how many overlays are created in those 2.5 seconds before we decide if we should immediately try to optimize performance...

from holoviews.

len(thetas) = 8, len(phis) = 8, and len(Bs) = 100.

I realise that it generates 10 times more curves, but the plots do look same.

from holoviews.

I have spent a little bit of time on optimizing things since the last release. I just profiled the thing and it seems to be sanitize_identifier on the Overlay that's so slow. Here's my timing and profiling output:

%%timeit
phase_boundaries = {(phi, theta):
                 hv.Overlay([hv.Curve(zip(Bs, mus[phi, theta, :, i])) for i in range(10)])
                 for theta in range(thetas) 
                 for phi in range(phis)}
1 loops, best of 3: 1.64 s per loop

And here the two main contributors, the first being sanitize_identifier and the second the param setup method.

Total profiling time 2.7 s
    13056    0.088    0.000    0.980    0.000 util.py:161(__call__)
     3456    0.153    0.000    0.715    0.000 parameterized.py:1376(_setup_params)

If we use an NdOverlay instead it's a lot faster:

%%timeit
phase_boundaries = {(phi, theta):
                 hv.NdOverlay({i: hv.Curve(zip(Bs, mus[phi, theta, :, i]))
                               for i in range(10)})
                 for theta in range(thetas) 
                 for phi in range(phis)}
1 loops, best of 3: 360 ms per loop

So that's my recommendation for now use an NdOverlay, which seems appropriate here anyway. Overlays are best for heterogenous data, while NdOverlays should be used for homogenous data, i.e. multiple observations of the same quantity and sharing the same Element type.

We will still have a look at speeding up HoloViews in general and sanitize_identifier in particular.

from holoviews.

Thanks for the NdOverlay tip! Unfortunately it creates another issue which is related to the following topic I think (posted it there): #56

from holoviews.

Ok, I think we have resolved these issues now, including the new one!

Performance is a general goal that we are fully aware of and working on so I don't feel we need to create a particular issue for that right now. Looks like we are ready to close this one!

from holoviews.