Alpha compositing of ~60000 line plots takes forever

Jae-Joon Lee <lee.j.joon@...83...> writes:

If you're plotting lots of lines, do not use plot but use
LineCollection instead.

http://matplotlib.sourceforge.net/examples/api/collections_demo.html

http://matplotlib.sourceforge.net/api/

collections_api.html#matplotlib.collections.LineCollection

Here is slightly modified version of your code that uses
LineCollection (but I haven't check if the code is correct).
With my not so good macbook, it took me 3 sec for 6000 lines and it
seems like O(n) to me.

Thank you, thank you, thank you.

This is just as convenient, 50% faster even for 1000 series, and runtime does
indeed scale as O(n) up to 10000 series. The projected speedup for 60000 series
was 40x. However, in my actual use case it was at least 400x: Finishing in 2
min 17 sec rather than not getting past halfway in 16 hours.

(The extra difference is probably due to better memory usage. Still,
LineCollection requires O(n) memory, whereas manually updating a bitmap would
only use O(1) memory, where 1 = size of bitmap. However, I hope I never have to
do that...)

May the hours and hours you have saved me be added to your life! :sunglasses:

Jon Olav

Since you are granting extra life blessings, I thought I should add
something to the mix. You should be able to achieve something close
to this using the animation blit API. There is a little hackery at
the end to use the renderer to directly dump a PNG and thereby
circumvent the normal figure.canvas.draw pipeline, but the advantage
is you render directly to the canvas and save no intermediaries. See
the examples and tutorial at

  http://matplotlib.sourceforge.net/examples/animation/index.html
  http://www.scipy.org/Cookbook/Matplotlib/Animations

Here's some example code::

import matplotlib._png as _png
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure()
ax = fig.add_subplot(111)

n = 10000
line, = ax.plot([],[], alpha=1)

x = np.arange(200)

fig.canvas.draw()
ax.axis([0, 200, -1, 1])
for i in range(n):
    if (i%100)==0: print i
    yy = np.sin(x / (2 * np.pi * x[-1] * i))

    line.set_data(x, yy)
    ax.draw_artist(line)

fig.canvas.blit(ax.bbox)

filename = 'test.png'
renderer = fig.canvas.get_renderer()
_png.write_png(renderer._renderer.buffer_rgba(0, 0),
               renderer.width, renderer.height,
               filename, fig.dpi)

JDH

···

On Tue, Mar 16, 2010 at 8:46 AM, Jon Olav Vik <jonovik@...287...> wrote:

Thank you, thank you, thank you.

This is just as convenient, 50% faster even for 1000 series, and runtime does
indeed scale as O(n) up to 10000 series. The projected speedup for 60000 series
was 40x. However, in my actual use case it was at least 400x: Finishing in 2
min 17 sec rather than not getting past halfway in 16 hours.

(The extra difference is probably due to better memory usage. Still,
LineCollection requires O(n) memory, whereas manually updating a bitmap would
only use O(1) memory, where 1 = size of bitmap. However, I hope I never have to
do that...)

May the hours and hours you have saved me be added to your life! :sunglasses: