Plot vs Line Collections

Hi All,

    I am writing an application (wxPython based) which embeds a big
matplotlib figure as a main panel. Basically, this app shows oil well
producers and gas injectors on a 2D map as dots (every dot represents
its surface location), and a bunch of "streamlines" (i.e., straight
lines or simple curves) which connect injectors and producers.
As the numerical simulation continues, more and more streamlines are
added to the plot (because of new wells or because interference
between wells), and actually I end up having 200 dots plus 800-1200
lines. As the simulation progresses, the plots become slower and
slower...
As the lines are usually 2-points straight lines, I was thinking about
using Line Collections; however, every matplotlib line has a linewidth
value that is dependent on the calculated "interference" effect
between wells, which means I have to build a matplotlib line for every
line connecting an injector with a producer. Moreover, every injector
well has its own colour for the streamlines (there are 33 injector
wells).
Will Line Collections save some time in this case? If not, does anyone
have a suggestion on how I could try to speed-up the plotting? I am
not really familiar with some obscure line/axes properties, so I may
have overlooked something.
This is with matplotlib 0.90, numpy 1.0.3, wxPython 2.8.4, Python 2.5,
Windows XP, WxAgg (pure Python implementation).

Thank you for every suggestion, and sorry for the long post.

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."
http://xoomer.virgilio.it/infinity77/

Yes, a line collection will save you a lot of time with upwards of
1000 line segments. This is the use case they were designed to solve:
a bunch of segments of differing widths and colors. One could
optimize it for the special case of simple line segments, ie [(x1,y1),
(x2, y2)] in which case we could use numpy arrays, but currently we
have only the general case of a collection of arbitrary length
segments, and since they are not necessarily the same length, we use a
sequence of segments rather than an array, and this is slower than it
could be.

JDH

···

On 5/31/07, Andrea Gavana <andrea.gavana@...287...> wrote:

Hi All,

    I am writing an application (wxPython based) which embeds a big
matplotlib figure as a main panel. Basically, this app shows oil well
producers and gas injectors on a 2D map as dots (every dot represents
its surface location), and a bunch of "streamlines" (i.e., straight
lines or simple curves) which connect injectors and producers.
As the numerical simulation continues, more and more streamlines are
added to the plot (because of new wells or because interference
between wells), and actually I end up having 200 dots plus 800-1200
lines. As the simulation progresses, the plots become slower and
slower...
As the lines are usually 2-points straight lines, I was thinking about
using Line Collections; however, every matplotlib line has a linewidth
value that is dependent on the calculated "interference" effect
between wells, which means I have to build a matplotlib line for every
line connecting an injector with a producer. Moreover, every injector
well has its own colour for the streamlines (there are 33 injector
wells).
Will Line Collections save some time in this case? If not, does anyone
have a suggestion on how I could try to speed-up the plotting? I am
not really familiar with some obscure line/axes properties, so I may
have overlooked something.

Hi John,

> Hi All,
>
> I am writing an application (wxPython based) which embeds a big
> matplotlib figure as a main panel. Basically, this app shows oil well
> producers and gas injectors on a 2D map as dots (every dot represents
> its surface location), and a bunch of "streamlines" (i.e., straight
> lines or simple curves) which connect injectors and producers.
> As the numerical simulation continues, more and more streamlines are
> added to the plot (because of new wells or because interference
> between wells), and actually I end up having 200 dots plus 800-1200
> lines. As the simulation progresses, the plots become slower and
> slower...
> As the lines are usually 2-points straight lines, I was thinking about
> using Line Collections; however, every matplotlib line has a linewidth
> value that is dependent on the calculated "interference" effect
> between wells, which means I have to build a matplotlib line for every
> line connecting an injector with a producer. Moreover, every injector
> well has its own colour for the streamlines (there are 33 injector
> wells).
> Will Line Collections save some time in this case? If not, does anyone
> have a suggestion on how I could try to speed-up the plotting? I am
> not really familiar with some obscure line/axes properties, so I may
> have overlooked something.

Yes, a line collection will save you a lot of time with upwards of
1000 line segments. This is the use case they were designed to solve:
a bunch of segments of differing widths and colors. One could
optimize it for the special case of simple line segments, ie [(x1,y1),
(x2, y2)] in which case we could use numpy arrays, but currently we
have only the general case of a collection of arbitrary length
segments, and since they are not necessarily the same length, we use a
sequence of segments rather than an array, and this is slower than it
could be.

Thank you for your answer. I am going to try to build these line
collections and see what I can get. I never used them before so it's
like I will get something not very well performing at first...

Thank you for your answer, I am going to look at some examples...

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."

···

On 5/31/07, John Hunter wrote:

On 5/31/07, Andrea Gavana <andrea.gavana@...287...> wrote:

Hi John,

> Hi All,
>
> I am writing an application (wxPython based) which embeds a big
> matplotlib figure as a main panel. Basically, this app shows oil well
> producers and gas injectors on a 2D map as dots (every dot represents
> its surface location), and a bunch of "streamlines" (i.e., straight
> lines or simple curves) which connect injectors and producers.
> As the numerical simulation continues, more and more streamlines are
> added to the plot (because of new wells or because interference
> between wells), and actually I end up having 200 dots plus 800-1200
> lines. As the simulation progresses, the plots become slower and
> slower...
> As the lines are usually 2-points straight lines, I was thinking about
> using Line Collections; however, every matplotlib line has a linewidth
> value that is dependent on the calculated "interference" effect
> between wells, which means I have to build a matplotlib line for every
> line connecting an injector with a producer. Moreover, every injector
> well has its own colour for the streamlines (there are 33 injector
> wells).
> Will Line Collections save some time in this case? If not, does anyone
> have a suggestion on how I could try to speed-up the plotting? I am
> not really familiar with some obscure line/axes properties, so I may
> have overlooked something.

Yes, a line collection will save you a lot of time with upwards of
1000 line segments. This is the use case they were designed to solve:
a bunch of segments of differing widths and colors. One could
optimize it for the special case of simple line segments, ie [(x1,y1),
(x2, y2)] in which case we could use numpy arrays, but currently we
have only the general case of a collection of arbitrary length
segments, and since they are not necessarily the same length, we use a
sequence of segments rather than an array, and this is slower than it
could be.

I managed to get things twice faster than before using Line
Collections (even though some more optimization can be done). However,
I am facing a problem with the legend: as I put all the lines in a
single collection, and I don't want all the lines to be marked in the
legend but only the ones with biggest linewidth for every injector
well, how do I get a particular line of the collection in order to
legend() that line only? I didn't find any method or attribute about
that. Maybe it is just plain impossible.

Thank you for every hint.

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."

···

On 5/31/07, John Hunter wrote:

On 5/31/07, Andrea Gavana <andrea.gavana@...287...> wrote:

Hi John and All,

Hi John,

> > Hi All,
> >
> > I am writing an application (wxPython based) which embeds a big
> > matplotlib figure as a main panel. Basically, this app shows oil well
> > producers and gas injectors on a 2D map as dots (every dot represents
> > its surface location), and a bunch of "streamlines" (i.e., straight
> > lines or simple curves) which connect injectors and producers.
> > As the numerical simulation continues, more and more streamlines are
> > added to the plot (because of new wells or because interference
> > between wells), and actually I end up having 200 dots plus 800-1200
> > lines. As the simulation progresses, the plots become slower and
> > slower...
> > As the lines are usually 2-points straight lines, I was thinking about
> > using Line Collections; however, every matplotlib line has a linewidth
> > value that is dependent on the calculated "interference" effect
> > between wells, which means I have to build a matplotlib line for every
> > line connecting an injector with a producer. Moreover, every injector
> > well has its own colour for the streamlines (there are 33 injector
> > wells).
> > Will Line Collections save some time in this case? If not, does anyone
> > have a suggestion on how I could try to speed-up the plotting? I am
> > not really familiar with some obscure line/axes properties, so I may
> > have overlooked something.
>
> Yes, a line collection will save you a lot of time with upwards of
> 1000 line segments. This is the use case they were designed to solve:
> a bunch of segments of differing widths and colors. One could
> optimize it for the special case of simple line segments, ie [(x1,y1),
> (x2, y2)] in which case we could use numpy arrays, but currently we
> have only the general case of a collection of arbitrary length
> segments, and since they are not necessarily the same length, we use a
> sequence of segments rather than an array, and this is slower than it
> could be.

I managed to get things twice faster than before using Line
Collections (even though some more optimization can be done). However,
I am facing a problem with the legend: as I put all the lines in a
single collection, and I don't want all the lines to be marked in the
legend but only the ones with biggest linewidth for every injector
well, how do I get a particular line of the collection in order to
legend() that line only? I didn't find any method or attribute about
that. Maybe it is just plain impossible.

Thank you for every hint.

I probably solved this thing, with a workaround instead of a real
solution. Instead of definining a single Line Collection for all the
lines, I just build a Line Collection for every injector well (from
which the lines start). That means I have at most 33 Line Collections
to plot instead of 800-1000 calls to axis.plot(), and I can use the
legend() command. It's not optimal, but it works sufficiently fast.
I still have a couple of questions:

1) I am plotting a bunch of points (which represent the well positions
on a 2D map) and next to them the well names as axis.text() instances.
I don't think something like a "Text Collection" exists, but is there
a way to draw a bunch of texts without a loop?
2) I am currently using the pure Python implementation of the WxAgg
backend, as with wxPython 2.8.4 and the pyd backend I always get the
message that "wxPython2.6-unicode can not be found" or something like
that. Is there any plan to adapt matplotlib to use the available
wxPython installation instead of a predefined one? And, in this
respect, will I get any performance improvement for the kind of plots
I am doing right now?

Thank you for your suggestions.

Andrea.

"Imagination Is The Only Weapon In The War Against Reality."

···

On 6/1/07, Andrea Gavana wrote:

On 5/31/07, John Hunter wrote:
> On 5/31/07, Andrea Gavana <andrea.gavana@...287...> wrote:

Is there any plan to adapt matplotlib to use the available
wxPython installation instead of a predefined one?

Yes. The next release of matplotlib will do a better job of selecting the approriate drawing implementation. It will also include support for wxPython 2.8's more efficient drawing methods.

And, in this respect, will I get any performance improvement for the kind of plots
I am doing right now?

Probably not. The focus of optimization has been improving the rate at which Agg images can be drawn by wxPython, rather than improving some specific use case.

Ken

···

On Jun 2, 2007, at 5:51 AM, Andrea Gavana wrote: