Why is flush_events() taking longer than expected?

david.aldrich · July 21, 2017, 1:33pm

Hi

I am using Matplotlib 2.0.2 in a Python 3.6 script on Windows 10 to plot data received over a network connection. New data arrives 4 times per second. I need to update two figures fig_A and fig_B, each containing 8 subplots. The hardware is a fast i7 laptop.

My main execution loop is:

plt.draw()

while (True):
    fig_A.canvas.flush_events()
    fig_B.canvas.flush_events()
    updatePlots() # Polls socket for data and updates scatter collections
    fig_A.canvas.draw_idle()
    fig_B.canvas.draw_idle()

plt.waitforbuttonpress()

I have timed the code and found that the first flush_events() call in the loop takes 350ms. The second flush_events() call takes <1ms.
350ms seems very long and is restricting the update rate.

Why might flush_events() be taking so long?

(I have not changed the backend from the default, which I believe is TkAgg).

Best regards

David

ben.v.root · July 25, 2017, 2:46pm

When you say first and second flush_events(), do you mean the first and
second iterations, or fig_A and fig_B?

Ben Root

···

On Fri, Jul 21, 2017 at 9:33 AM, David Aldrich <David.Aldrich at emea.nec.com> wrote:

Hi

I am using Matplotlib 2.0.2 in a Python 3.6 script on Windows 10 to plot
data received over a network connection. New data arrives 4 times per
second. I need to update two figures fig_A and fig_B, each containing 8
subplots. The hardware is a fast i7 laptop.

My main execution loop is:

plt.draw()

while (True):
    fig_A.canvas.flush_events()
    fig_B.canvas.flush_events()
    updatePlots() # Polls socket for data and
updates scatter collections
    fig_A.canvas.draw_idle()
    fig_B.canvas.draw_idle()

plt.waitforbuttonpress()

I have timed the code and found that the first flush_events() call in the
loop takes 350ms. The second flush_events() call takes <1ms.
350ms seems very long and is restricting the update rate.

Why might flush_events() be taking so long?

(I have not changed the backend from the default, which I believe is
TkAgg).

Best regards

David

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org
Matplotlib-users Info Page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-users/attachments/20170725/a1ce2f04/attachment.html>

david.aldrich · July 25, 2017, 2:49pm

Hi Ben

When you say first and second flush_events(), do you mean the
first and second iterations, or fig_A and fig_B?

The latter:

fig_A.canvas.flush_events() <-- 350ms
fig_B.canvas.flush_events() <-- 1ms

These timings repeat on each iteration.

Best regards

David

···

From: Benjamin Root [mailto:ben.v.root@gmail.com]
Sent: 25 July 2017 15:46
To: David Aldrich <David.Aldrich at EMEA.NEC.COM>
Cc: matplotlib-users at python.org
Subject: Re: [Matplotlib-users] Why is flush_events() taking longer than expected?

When you say first and second flush_events(), do you mean the first and second iterations, or fig_A and fig_B?
Ben Root

On Fri, Jul 21, 2017 at 9:33 AM, David Aldrich <David.Aldrich at emea.nec.com<mailto:David.Aldrich at emea.nec.com>> wrote:
Hi

I am using Matplotlib 2.0.2 in a Python 3.6 script on Windows 10 to plot data received over a network connection. New data arrives 4 times per second. I need to update two figures fig_A and fig_B, each containing 8 subplots. The hardware is a fast i7 laptop.

My main execution loop is:

plt.draw()

while (True):
    fig_A.canvas.flush_events()
    fig_B.canvas.flush_events()
    updatePlots() # Polls socket for data and updates scatter collections
    fig_A.canvas.draw_idle()
    fig_B.canvas.draw_idle()

plt.waitforbuttonpress()

I have timed the code and found that the first flush_events() call in the loop takes 350ms. The second flush_events() call takes <1ms.
350ms seems very long and is restricting the update rate.

Why might flush_events() be taking so long?

(I have not changed the backend from the default, which I believe is TkAgg).

Best regards

David

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org<mailto:Matplotlib-users at python.org>
https://mail.python.org/mailman/listinfo/matplotlib-users

Click here<https://www.mailcontrol.com/sr/HdU0jnqYd33GX2PQPOmvUiOFufhVdryLTKI5fnw3hQXAeY!z4oAmQUowjoeq+mdHW6eN7GuMFmZOzrnJmt4uGw==> to report this email as spam.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-users/attachments/20170725/c6902ebc/attachment.html>

tacaswell · July 31, 2017, 12:53pm

Exactly what `flush_events` is doing depends on the backend, however I
suspect that in most cases the events are handled in a process-level
singleton so both calls are talking to event queue so the first one does
all the work and the second one finds and empty queue, declares victory and
returns. To test this switch the A/B order, I expect the first one to be
slow in either case.

As to why it is going slow, I suspect it is the text on your 16 axes. To
get the speed you want you are probably are going to want to use 'blitting'
which is a way of only re-drawing what has changed (rather than
everything). See
https://matplotlib.org/api/animation_api.html#funcanimation for more
details.

Tom

···

On Tue, Jul 25, 2017 at 11:26 AM David Aldrich <David.Aldrich at emea.nec.com> wrote:

Hi Ben

>When you say first and second flush_events(), do you mean the

>first and second iterations, or fig_A and fig_B?

The latter:

    fig_A.canvas.flush_events() ? 350ms
    fig_B.canvas.flush_events() ? 1ms

These timings repeat on each iteration.

Best regards

David

*From:* Benjamin Root [mailto:ben.v.root at gmail.com]
*Sent:* 25 July 2017 15:46
*To:* David Aldrich <David.Aldrich at EMEA.NEC.COM>
*Cc:* matplotlib-users at python.org
*Subject:* Re: [Matplotlib-users] Why is flush_events() taking longer
than expected?

When you say first and second flush_events(), do you mean the first and
second iterations, or fig_A and fig_B?

Ben Root

On Fri, Jul 21, 2017 at 9:33 AM, David Aldrich <David.Aldrich at emea.nec.com> > wrote:

Hi

I am using Matplotlib 2.0.2 in a Python 3.6 script on Windows 10 to plot
data received over a network connection. New data arrives 4 times per
second. I need to update two figures fig_A and fig_B, each containing 8
subplots. The hardware is a fast i7 laptop.

My main execution loop is:

plt.draw()

while (True):
    fig_A.canvas.flush_events()
    fig_B.canvas.flush_events()
    updatePlots() # Polls socket for data and
updates scatter collections
    fig_A.canvas.draw_idle()
    fig_B.canvas.draw_idle()

plt.waitforbuttonpress()

I have timed the code and found that the first flush_events() call in the
loop takes 350ms. The second flush_events() call takes <1ms.
350ms seems very long and is restricting the update rate.

Why might flush_events() be taking so long?

(I have not changed the backend from the default, which I believe is
TkAgg).

Best regards

David

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org
Matplotlib-users Info Page

Click here
<https://www.mailcontrol.com/sr/HdU0jnqYd33GX2PQPOmvUiOFufhVdryLTKI5fnw3hQXAeY!z4oAmQUowjoeq+mdHW6eN7GuMFmZOzrnJmt4uGw==>
to report this email as spam.
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org
Matplotlib-users Info Page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-users/attachments/20170731/9ed191a9/attachment.html>

david.aldrich · July 31, 2017, 1:00pm

Hi Thomas

Thanks for your reply.

To test this switch the A/B order, I expect the first one to be slow in either case.

Yes, I tried that and the first one is always slower, as you suggested.

To get the speed you want you are probably are going to want to use 'blitting'
which is a way of only re-drawing what has changed (rather than everything).

I understand. However, I expected the redraw work to be done in the plotting calls, not in `flush_events`. Do you think drawing may be done in `flush_events`?

Best regards

David

···

From: Thomas Caswell [mailto:tcaswell@gmail.com]
Sent: 31 July 2017 13:53
To: David Aldrich <David.Aldrich at EMEA.NEC.COM>; Benjamin Root <ben.v.root at gmail.com>
Cc: matplotlib-users at python.org
Subject: Re: [Matplotlib-users] Why is flush_events() taking longer than expected?

Exactly what `flush_events` is doing depends on the backend, however I suspect that in most cases the events are handled in a process-level singleton so both calls are talking to event queue so the first one does all the work and the second one finds and empty queue, declares victory and returns. To test this switch the A/B order, I expect the first one to be slow in either case.

As to why it is going slow, I suspect it is the text on your 16 axes. To get the speed you want you are probably are going to want to use 'blitting' which is a way of only re-drawing what has changed (rather than everything). See https://matplotlib.org/api/animation_api.html#funcanimation for more details.

Tom

On Tue, Jul 25, 2017 at 11:26 AM David Aldrich <David.Aldrich at emea.nec.com<mailto:David.Aldrich at emea.nec.com>> wrote:
Hi Ben

When you say first and second flush_events(), do you mean the
first and second iterations, or fig_A and fig_B?

The latter:

fig_A.canvas.flush_events() <-- 350ms
fig_B.canvas.flush_events() <-- 1ms

These timings repeat on each iteration.

Best regards

David

From: Benjamin Root [mailto:ben.v.root at gmail.com<mailto:ben.v.root@gmail.com>]
Sent: 25 July 2017 15:46
To: David Aldrich <David.Aldrich at EMEA.NEC.COM<mailto:David.Aldrich at EMEA.NEC.COM>>
Cc: matplotlib-users at python.org<mailto:matplotlib-users at python.org>
Subject: Re: [Matplotlib-users] Why is flush_events() taking longer than expected?

When you say first and second flush_events(), do you mean the first and second iterations, or fig_A and fig_B?
Ben Root

On Fri, Jul 21, 2017 at 9:33 AM, David Aldrich <David.Aldrich at emea.nec.com<mailto:David.Aldrich at emea.nec.com>> wrote:
Hi

I am using Matplotlib 2.0.2 in a Python 3.6 script on Windows 10 to plot data received over a network connection. New data arrives 4 times per second. I need to update two figures fig_A and fig_B, each containing 8 subplots. The hardware is a fast i7 laptop.

My main execution loop is:

plt.draw()

while (True):
    fig_A.canvas.flush_events()
    fig_B.canvas.flush_events()
    updatePlots() # Polls socket for data and updates scatter collections
    fig_A.canvas.draw_idle()
    fig_B.canvas.draw_idle()

plt.waitforbuttonpress()

I have timed the code and found that the first flush_events() call in the loop takes 350ms. The second flush_events() call takes <1ms.
350ms seems very long and is restricting the update rate.

Why might flush_events() be taking so long?

(I have not changed the backend from the default, which I believe is TkAgg).

Best regards

David

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org<mailto:Matplotlib-users at python.org>
https://mail.python.org/mailman/listinfo/matplotlib-users

Click here<https://www.mailcontrol.com/sr/HdU0jnqYd33GX2PQPOmvUiOFufhVdryLTKI5fnw3hQXAeY!z4oAmQUowjoeq+mdHW6eN7GuMFmZOzrnJmt4uGw==> to report this email as spam.
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org<mailto:Matplotlib-users at python.org>
Matplotlib-users Info Page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-users/attachments/20170731/df2b38e4/attachment-0001.html>

tacaswell · July 31, 2017, 1:05pm

We have made an effort recently to make drawing as lazy as possible and not
actually do the rendering until it as absolutely required. Ideally delayed
off until the GUI needs the pixel values for the paint event. This lets
you do many `art.update(...); fig.canvas.draw_idle()` and only pay the draw
cost once when you need to show it on the screen (rather than on every
update).

Tom

···

On Mon, Jul 31, 2017 at 9:00 AM David Aldrich <David.Aldrich at emea.nec.com> wrote:

Hi Thomas

Thanks for your reply.

> To test this switch the A/B order, I expect the first one to be slow in
either case.

Yes, I tried that and the first one is always slower, as you suggested.

> To get the speed you want you are probably are going to want to use
'blitting'

> which is a way of only re-drawing what has changed (rather than
everything).

I understand. However, I expected the redraw work to be done in the
plotting calls, not in `flush_events`. Do you think drawing may be done in
`flush_events`?

Best regards

David

*From:* Thomas Caswell [mailto:tcaswell at gmail.com]
*Sent:* 31 July 2017 13:53
*To:* David Aldrich <David.Aldrich at EMEA.NEC.COM>; Benjamin Root <
ben.v.root at gmail.com>

*Cc:* matplotlib-users at python.org
*Subject:* Re: [Matplotlib-users] Why is flush_events() taking longer
than expected?

Exactly what `flush_events` is doing depends on the backend, however I
suspect that in most cases the events are handled in a process-level
singleton so both calls are talking to event queue so the first one does
all the work and the second one finds and empty queue, declares victory and
returns. To test this switch the A/B order, I expect the first one to be
slow in either case.

As to why it is going slow, I suspect it is the text on your 16 axes. To
get the speed you want you are probably are going to want to use 'blitting'
which is a way of only re-drawing what has changed (rather than
everything). See
https://matplotlib.org/api/animation_api.html#funcanimation for more
details.

Tom

On Tue, Jul 25, 2017 at 11:26 AM David Aldrich <David.Aldrich at emea.nec.com> > wrote:

Hi Ben

>When you say first and second flush_events(), do you mean the

>first and second iterations, or fig_A and fig_B?

The latter:

    fig_A.canvas.flush_events() ? 350ms
    fig_B.canvas.flush_events() ? 1ms

These timings repeat on each iteration.

Best regards

David

*From:* Benjamin Root [mailto:ben.v.root at gmail.com]
*Sent:* 25 July 2017 15:46
*To:* David Aldrich <David.Aldrich at EMEA.NEC.COM>
*Cc:* matplotlib-users at python.org
*Subject:* Re: [Matplotlib-users] Why is flush_events() taking longer
than expected?

When you say first and second flush_events(), do you mean the first and
second iterations, or fig_A and fig_B?

Ben Root

On Fri, Jul 21, 2017 at 9:33 AM, David Aldrich <David.Aldrich at emea.nec.com> > wrote:

Hi

I am using Matplotlib 2.0.2 in a Python 3.6 script on Windows 10 to plot
data received over a network connection. New data arrives 4 times per
second. I need to update two figures fig_A and fig_B, each containing 8
subplots. The hardware is a fast i7 laptop.

My main execution loop is:

plt.draw()

while (True):
    fig_A.canvas.flush_events()
    fig_B.canvas.flush_events()
    updatePlots() # Polls socket for data and
updates scatter collections
    fig_A.canvas.draw_idle()
    fig_B.canvas.draw_idle()

plt.waitforbuttonpress()

I have timed the code and found that the first flush_events() call in the
loop takes 350ms. The second flush_events() call takes <1ms.
350ms seems very long and is restricting the update rate.

Why might flush_events() be taking so long?

(I have not changed the backend from the default, which I believe is
TkAgg).

Best regards

David

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org
Matplotlib-users Info Page

Click here
<https://www.mailcontrol.com/sr/HdU0jnqYd33GX2PQPOmvUiOFufhVdryLTKI5fnw3hQXAeY!z4oAmQUowjoeq+mdHW6eN7GuMFmZOzrnJmt4uGw==>
to report this email as spam.

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org
Matplotlib-users Info Page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-users/attachments/20170731/cd1a9405/attachment-0001.html>

david.aldrich · July 31, 2017, 1:07pm

Thanks, I understand.

David

···

From: Thomas Caswell [mailto:tcaswell@gmail.com]
Sent: 31 July 2017 14:06
To: David Aldrich <David.Aldrich at EMEA.NEC.COM>; Benjamin Root <ben.v.root at gmail.com>
Cc: matplotlib-users at python.org
Subject: Re: [Matplotlib-users] Why is flush_events() taking longer than expected?

We have made an effort recently to make drawing as lazy as possible and not actually do the rendering until it as absolutely required. Ideally delayed off until the GUI needs the pixel values for the paint event. This lets you do many `art.update(...); fig.canvas.draw_idle()` and only pay the draw cost once when you need to show it on the screen (rather than on every update).

Tom

On Mon, Jul 31, 2017 at 9:00 AM David Aldrich <David.Aldrich at emea.nec.com<mailto:David.Aldrich at emea.nec.com>> wrote:
Hi Thomas

Thanks for your reply.

To test this switch the A/B order, I expect the first one to be slow in either case.

Yes, I tried that and the first one is always slower, as you suggested.

To get the speed you want you are probably are going to want to use 'blitting'
which is a way of only re-drawing what has changed (rather than everything).

I understand. However, I expected the redraw work to be done in the plotting calls, not in `flush_events`. Do you think drawing may be done in `flush_events`?

Best regards

David

From: Thomas Caswell [mailto:tcaswell at gmail.com<mailto:tcaswell@gmail.com>]
Sent: 31 July 2017 13:53
To: David Aldrich <David.Aldrich at EMEA.NEC.COM<mailto:David.Aldrich at EMEA.NEC.COM>>; Benjamin Root <ben.v.root at gmail.com<mailto:ben.v.root at gmail.com>>

Cc: matplotlib-users at python.org<mailto:matplotlib-users at python.org>
Subject: Re: [Matplotlib-users] Why is flush_events() taking longer than expected?

Exactly what `flush_events` is doing depends on the backend, however I suspect that in most cases the events are handled in a process-level singleton so both calls are talking to event queue so the first one does all the work and the second one finds and empty queue, declares victory and returns. To test this switch the A/B order, I expect the first one to be slow in either case.

As to why it is going slow, I suspect it is the text on your 16 axes. To get the speed you want you are probably are going to want to use 'blitting' which is a way of only re-drawing what has changed (rather than everything). See https://matplotlib.org/api/animation_api.html#funcanimation for more details.

Tom

On Tue, Jul 25, 2017 at 11:26 AM David Aldrich <David.Aldrich at emea.nec.com<mailto:David.Aldrich at emea.nec.com>> wrote:
Hi Ben

When you say first and second flush_events(), do you mean the
first and second iterations, or fig_A and fig_B?

The latter:

fig_A.canvas.flush_events() <-- 350ms
fig_B.canvas.flush_events() <-- 1ms

These timings repeat on each iteration.

Best regards

David

From: Benjamin Root [mailto:ben.v.root at gmail.com<mailto:ben.v.root@gmail.com>]
Sent: 25 July 2017 15:46
To: David Aldrich <David.Aldrich at EMEA.NEC.COM<mailto:David.Aldrich at EMEA.NEC.COM>>
Cc: matplotlib-users at python.org<mailto:matplotlib-users at python.org>
Subject: Re: [Matplotlib-users] Why is flush_events() taking longer than expected?

When you say first and second flush_events(), do you mean the first and second iterations, or fig_A and fig_B?
Ben Root

On Fri, Jul 21, 2017 at 9:33 AM, David Aldrich <David.Aldrich at emea.nec.com<mailto:David.Aldrich at emea.nec.com>> wrote:
Hi

I am using Matplotlib 2.0.2 in a Python 3.6 script on Windows 10 to plot data received over a network connection. New data arrives 4 times per second. I need to update two figures fig_A and fig_B, each containing 8 subplots. The hardware is a fast i7 laptop.

My main execution loop is:

plt.draw()

while (True):
    fig_A.canvas.flush_events()
    fig_B.canvas.flush_events()
    updatePlots() # Polls socket for data and updates scatter collections
    fig_A.canvas.draw_idle()
    fig_B.canvas.draw_idle()

plt.waitforbuttonpress()

I have timed the code and found that the first flush_events() call in the loop takes 350ms. The second flush_events() call takes <1ms.
350ms seems very long and is restricting the update rate.

Why might flush_events() be taking so long?

(I have not changed the backend from the default, which I believe is TkAgg).

Best regards

David

_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org<mailto:Matplotlib-users at python.org>
https://mail.python.org/mailman/listinfo/matplotlib-users

Click here<https://www.mailcontrol.com/sr/HdU0jnqYd33GX2PQPOmvUiOFufhVdryLTKI5fnw3hQXAeY!z4oAmQUowjoeq+mdHW6eN7GuMFmZOzrnJmt4uGw==> to report this email as spam.
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at python.org<mailto:Matplotlib-users at python.org>
Matplotlib-users Info Page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-users/attachments/20170731/4451369e/attachment-0001.html>