Timer and widget callbacks

I have an application using Qt5Agg backend where I have a button widget and a timer. The timer fires every 100ms. The button is clicked by human user. Both timer and button handlers update a shared object as shown in outline below.

My question: I want sharedobj updates from the button click handler when the user has just clicked to not be interleaved in time with updates from the timer waking up and running its handler. In other words, I am looking for matplotlib event handling to guarantee that each handler runs to completion before the other handler gets to run even though the timer is firing repeatedly every 100ms. Is that the case? Thank you!

button.on_clicked(buttonhandler)
timer = fig.canvas.new_timer(interval=100)
timer.add_callback(timerhandler)

def timerhandler():
    sharedobj = …

def buttonhandler():
   sharedobj  = ...

Hi @alfadot

I wonder if your question can be solved by using threading methods, where access to critical shared resources needs to be carefully managed:

http://www.laurentluce.com/posts/python-threads-synchronization-locks-rlocks-semaphores-conditions-events-and-queues/

Hi Laurence,
Yes, that would be one way to solve the problem but an unnecessary complication if matplotlib already provides the guarantee that all event handlers, including the timer event handler, are dispatched such that each handler runs to completion without time overlap with another event handler. So my question is to matplotlib developers or knowledgeable users as to whether such is the case. Thanks.

Ah. I see. A lock at the handler level rather than the shared object level.

I’m no matplotlib expert, but there are empirical ways of answering this question.

Running the test code below confirms that multiple matplotlib timers respect eachother in that way. If you could add a button event to the basic code below that calls its own function and sits for, say, 2 seconds you might be able to empirically prove whether the timers also wait for the Qt5 events to complete.

import matplotlib.pyplot as plt
from datetime import datetime
import time

def timestamp(t,w):
    t1 = datetime.now()
    time.sleep(w)
    t2 = datetime.now()
    print(f"{t}({w}):" , t1 , t2)

fig, ax = plt.subplots()
timer1 = fig.canvas.new_timer(interval=300)
timer1.add_callback(timestamp, 1, 0.3)
timer1.start()
timer2 = fig.canvas.new_timer(interval=450)
timer2.add_callback(timestamp, 2, 0.45)
timer2.start()
plt.show()

> 1(0.3): 2020-01-31 17:45:46.925281 2020-01-31 17:45:47.225704
> 1(0.3): 2020-01-31 17:45:47.227683 2020-01-31 17:45:47.528109
> 2(0.45): 2020-01-31 17:45:47.528948 2020-01-31 17:45:47.979105
> 2(0.45): 2020-01-31 17:45:47.982564 2020-01-31 17:45:48.433146
> 1(0.3): 2020-01-31 17:45:48.433951 2020-01-31 17:45:48.734238

Thanks again Laurence. Empirically, it seems matplotlib event dispatch allows each event handler, including timer event handler, to run to completion without time overlap with another event handler. I used the simple code for testing this.

So all event handlers seem to get serialized through one event dispatch thread (which is how it ought to be). However, it is baffling to see psutil showing 7 threads running on my machine as part of the program below. What are all these threads doing??

It would be great to hear from a matplotlib developer or expert with knowledge of matplotlib internals that the event dispatch for all events is serialized.

import time
import psutil
import numpy as np
from datetime import datetime
import matplotlib.pyplot as plt
from matplotlib.widgets import Button 


fig = plt.figure()
mainax = fig.add_axes([0.1, 0.1, 0.8, 0.8])
x = np.linspace(-3, 3)
mainax.plot(x, x ** 2)


def buttonhandler(event):
    t1 = datetime.now()
    print('button entry: ', t1)
    time.sleep(10)
    t2 = datetime.now()
    print('button exit:  ', t2)
buttonax = plt.axes([0.05, 0.85, 0.05, 0.05])
button = Button(ax=buttonax, label='A')
button.on_clicked(buttonhandler)


def timerhandler():
    t1 = datetime.now()
    p = psutil.Process()
    print(p.num_threads(), 'threads running!')
    print(' timer entry: ', t1)
    mainax.set_title(t1)
    fig.canvas.draw()
    time.sleep(10)
    t2 = datetime.now()
    print(' timer exit:  ', t2)
timer = fig.canvas.new_timer(interval=1000)
timer.add_callback(timerhandler)
timer.start()
plt.show()

No problem.

Incidentally, I found reference in the timer class code that matplotlib specifically creates backend-specific timers to run them through the backend’s native event loop.

e.g. In the case of timer initialisation with the QT5 backend:

self._timer = QtCore.QTimer()

I’ll leave it to the core devs to comment on matplotlib events in general.

So all event handlers seem to get serialized through one event dispatch thread (which is how it ought to be)

The “event dispatch thread” is the main thread. If you are using Qt, then all of the events are managed by the Qt event loop which must be running on the main thread (at Qt’s insistence). Both the user clicks and the timer firing land on the Qt queue and are serviced one at a time in the order they were received. You should aim for the callbacks to be as quick as possible as it is blocking the Qt event loop and your GUI will be un-responsive during that time.

I am, however, hesitant to categorically sign off that the two callbacks will never conflict due to the lack of concrete detail of what the callbacks are doing and a general paranoia around concurrent programming.

You should use draw_idle rather than draw in the call back (which will schedule a re-draw with the GUI framework) rather than forcing the render to happen immediately.

For more details about how python + GUI event loops get integrated see https://github.com/matplotlib/matplotlib/pull/4779 (which I really need to finish and get merged)

As for what the threads are doing, Qt may have some (invisible to Python threads) and we use threading.Timer objects in a few places to warn the user when a task takes longer than expected etc.

1 Like

@tacaswell: thank you for the detailed explanation. That clears things up beautifully.