Animation with copy_from_bbox / restore_region.

Elan_Pavlov · April 20, 2009, 8:34pm

Hi,

I'm trying to use matplotlib for animating data as it is received from an online source (online in the algorithmic sense not internet:). I'd like the graph plot to be updated with high frequency since the data changes rapidly. I've used the BufferRegion with copy_from_bbox / restore_region and it speeds up the plotting considerably but alas it's still not good enough (with a large number of graphs and a large number of data points in each graph).

What I'd like to do is to utilize the fact that the animation is updated in a predictable fashion (i.e., scrolling off the screen to the left as new data arrives) in order to speed up the animation. The idea would be to copy the right 99% of the graph (or some other fraction) via some kind of function similar to copy_from_bbox, move it 1% to the left and then plot the new 1% of the data. The problem is that as far as I can tell the copy_from_bbox/restore_region does not actually allow changing the area in which it is restored. I've mucked around in the source files a bit but to no avail.

My question is then:
1. Is there some other way to copy everything enclosed in a Bbox? or
2. Is there some way to modify the region in which data reappears when using restore_region?
3. Perhaps I'm missing something?

I'm sure someone else has done this since it seems pretty natural and useful for a variety of applications.

Elan

···

----
"If stupidity got us into this mess, why can't it get us out?"
- Will Rogers

_Jae-Joon_Lee1 · April 23, 2009, 5:21pm

Hi,

I'm trying to use matplotlib for animating data as it is received from an
online source (online in the algorithmic sense not internet:). I'd like
the graph plot to be updated with high frequency since the data changes
rapidly. I've used the BufferRegion with copy_from_bbox / restore_region
and it speeds up the plotting considerably but alas it's still not good
enough (with a large number of graphs and a large number of data points in
each graph).

Well, if you need your plot updated with very high frequency, MPL may
not be your tool of choice.
Anyhow, my first recommendation is not to update the plot frequently.
I mean, do you have to update the plot for every changes? Can you
update it every 10th change, for example?

What I'd like to do is to utilize the fact that the animation is updated
in a predictable fashion (i.e., scrolling off the screen to the left as
new data arrives) in order to speed up the animation. The idea would be to
copy the right 99% of the graph (or some other fraction) via some kind of
function similar to copy_from_bbox, move it 1% to the left and then plot
the new 1% of the data. The problem is that as far as I can tell the
copy_from_bbox/restore_region does not actually allow changing the area in
which it is restored. I've mucked around in the source files a bit but to
no avail.

My question is then:
1. Is there some other way to copy everything enclosed in a Bbox? or
2. Is there some way to modify the region in which data reappears when
using restore_region?
3. Perhaps I'm missing something?

I'm sure someone else has done this since it seems pretty natural and
useful for a variety of applications.

I'm attaching a patch that might do what you want to do.
It implements a "restore_bbox2" method which restores a subset of the
saved background at a specified position.
A small example is also attached (it is based on the gtk backend).

I, personally, am not sure if this kind of feature is useful. If you
shift part of your figure, you have to be careful not to mess up with
the coordinate system. And also be careful about what to be shifted
and what to be not (e.g., ticks).

If others find this useful, I'll commit this to the trunk with some
api improvement.

Regards,

-JJ

restore_region2.diff (3.62 KB)

animation_blit_gtk2.py (1.96 KB)

···

On Mon, Apr 20, 2009 at 4:34 PM, Elan Pavlov <elan@...1166...> wrote:

Elan
----
"If stupidity got us into this mess, why can't it get us out?"
- Will Rogers

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

Elan_Pavlov3 · April 23, 2009, 6:32pm

Hi Jae-Joon,
Thanks a ton! The problem is actually not the frequency of changes.
The current method uses draw_artist on each update. However, the time
for draw_artist is linear in the *number* of points so for graphs with
a large amount of data it is extremely inefficient. Your patch means
that update time is linear in the amount of data *updated* and not in
the total amount of data.
As for the ticks you have a point. In my application I actually want
ticks to be shifted (as time goes on) so indeed this is what I want.
In general ticks that are shifted out can be redrawn using set_data
methods which are efficient since a tick can be defined as it's
endpoints. Basically when first calling plot() I set all of the tick
locators to Null and then just add ticks as a line (with two points)
based on the desired spacing. Changing a tick location (or if desired
redrawing it in the same location) is basically a call to
set_(x|y)data and then draw_artist for a line with two points (or a
line collection will *slightly* improve performance if needed).
Anyway, thanks again. I'll play around with it and let you know what I
run into.

Elan

···

---
I can no other answer make but thanks,
And thanks, and ever thanks.
-- William Shakespeare

On Thu, Apr 23, 2009 at 1:21 PM, Jae-Joon Lee <lee.j.joon@...287...> wrote:

On Mon, Apr 20, 2009 at 4:34 PM, Elan Pavlov <elan@...1166...> wrote:

Hi,

I'm trying to use matplotlib for animating data as it is received from an
online source (online in the algorithmic sense not internet:). I'd like
the graph plot to be updated with high frequency since the data changes
rapidly. I've used the BufferRegion with copy_from_bbox / restore_region
and it speeds up the plotting considerably but alas it's still not good
enough (with a large number of graphs and a large number of data points in
each graph).

Well, if you need your plot updated with very high frequency, MPL may
not be your tool of choice.
Anyhow, my first recommendation is not to update the plot frequently.
I mean, do you have to update the plot for every changes? Can you
update it every 10th change, for example?

What I'd like to do is to utilize the fact that the animation is updated
in a predictable fashion (i.e., scrolling off the screen to the left as
new data arrives) in order to speed up the animation. The idea would be to
copy the right 99% of the graph (or some other fraction) via some kind of
function similar to copy_from_bbox, move it 1% to the left and then plot
the new 1% of the data. The problem is that as far as I can tell the
copy_from_bbox/restore_region does not actually allow changing the area in
which it is restored. I've mucked around in the source files a bit but to
no avail.

My question is then:
1. Is there some other way to copy everything enclosed in a Bbox? or
2. Is there some way to modify the region in which data reappears when
using restore_region?
3. Perhaps I'm missing something?

I'm sure someone else has done this since it seems pretty natural and
useful for a variety of applications.

I'm attaching a patch that might do what you want to do.
It implements a "restore_bbox2" method which restores a subset of the
saved background at a specified position.
A small example is also attached (it is based on the gtk backend).

I, personally, am not sure if this kind of feature is useful. If you
shift part of your figure, you have to be careful not to mess up with
the coordinate system. And also be careful about what to be shifted
and what to be not (e.g., ticks).

If others find this useful, I'll commit this to the trunk with some
api improvement.

Regards,

-JJ

Elan
----
"If stupidity got us into this mess, why can't it get us out?"
- Will Rogers

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
If I knew that a man was coming to my house with the conscious design
of doing me good, I should run for my life.
- Henry David Thoreau

_Jae-Joon_Lee1 · April 24, 2009, 2:53pm

Hi Jae-Joon,
Thanks a ton! The problem is actually not the frequency of changes.
The current method uses draw_artist on each update. However, the time
for draw_artist is linear in the *number* of points so for graphs with
a large amount of data it is extremely inefficient. Your patch means
that update time is linear in the amount of data *updated* and not in
the total amount of data.
As for the ticks you have a point. In my application I actually want
ticks to be shifted (as time goes on) so indeed this is what I want.

I was more concerned about yticks (when images are shifted in
x-direction). Anyhow, it should be not that difficult to handle this.

In general ticks that are shifted out can be redrawn using set_data
methods which are efficient since a tick can be defined as it's
endpoints. Basically when first calling plot() I set all of the tick
locators to Null and then just add ticks as a line (with two points)
based on the desired spacing. Changing a tick location (or if desired
redrawing it in the same location) is basically a call to
set_(x|y)data and then draw_artist for a line with two points (or a
line collection will *slightly* improve performance if needed).
Anyway, thanks again. I'll play around with it and let you know what I
run into.

Yes, please.
If you find this useful, I'll try to push this into the trunk.

Regards,

-JJ

···

On Thu, Apr 23, 2009 at 2:32 PM, Elan Pavlov <epavlov@...287...> wrote:

Elan
---
I can no other answer make but thanks,
And thanks, and ever thanks.
-- William Shakespeare

On Thu, Apr 23, 2009 at 1:21 PM, Jae-Joon Lee <lee.j.joon@...287...> wrote:

On Mon, Apr 20, 2009 at 4:34 PM, Elan Pavlov <elan@...1166...> wrote:

Hi,

I'm trying to use matplotlib for animating data as it is received from an
online source (online in the algorithmic sense not internet:). I'd like
the graph plot to be updated with high frequency since the data changes
rapidly. I've used the BufferRegion with copy_from_bbox / restore_region
and it speeds up the plotting considerably but alas it's still not good
enough (with a large number of graphs and a large number of data points in
each graph).

Well, if you need your plot updated with very high frequency, MPL may
not be your tool of choice.
Anyhow, my first recommendation is not to update the plot frequently.
I mean, do you have to update the plot for every changes? Can you
update it every 10th change, for example?

What I'd like to do is to utilize the fact that the animation is updated
in a predictable fashion (i.e., scrolling off the screen to the left as
new data arrives) in order to speed up the animation. The idea would be to
copy the right 99% of the graph (or some other fraction) via some kind of
function similar to copy_from_bbox, move it 1% to the left and then plot
the new 1% of the data. The problem is that as far as I can tell the
copy_from_bbox/restore_region does not actually allow changing the area in
which it is restored. I've mucked around in the source files a bit but to
no avail.

My question is then:
1. Is there some other way to copy everything enclosed in a Bbox? or
2. Is there some way to modify the region in which data reappears when
using restore_region?
3. Perhaps I'm missing something?

I'm sure someone else has done this since it seems pretty natural and
useful for a variety of applications.

I'm attaching a patch that might do what you want to do.
It implements a "restore_bbox2" method which restores a subset of the
saved background at a specified position.
A small example is also attached (it is based on the gtk backend).

I, personally, am not sure if this kind of feature is useful. If you
shift part of your figure, you have to be careful not to mess up with
the coordinate system. And also be careful about what to be shifted
and what to be not (e.g., ticks).

If others find this useful, I'll commit this to the trunk with some
api improvement.

Regards,

-JJ

Elan
----
"If stupidity got us into this mess, why can't it get us out?"
- Will Rogers

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
If I knew that a man was coming to my house with the conscious design
of doing me good, I should run for my life.
- Henry David Thoreau

_Jae-Joon_Lee1 · May 14, 2009, 6:35am

This is now committed to svn trunk, with slight changes in its api.
An example is added (examples/animation/animation_blit_gtk2.py).

-JJ