matplotlib threadsafe?

Community matplotlib-users

Chris_Withers March 17, 2008, 11:43am 1

Hi All,

I'm wondering what work people have done with matplotlib in multi-threaded environments such as your average python web framework.

Is matplotlib threadsafe?

How have people gone about safely using it in a multi-threaded environment?

cheers,

Chris

···

--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk

Michael_Droettboom1 March 18, 2008, 11:19pm 2

At least the Agg backend *looks* to be reasonably threadsafe -- there are no obvious gotchas like global variables etc. Note, though, that multithreading may not gain much in the way of performance since the global interpreter lock is never released around long-running C blocks.

However, I can't speak about this from any experience -- so, maybe it needs some trying. Any patches to help with thread safety and performance are of course welcome

Cheers,
Mike

Chris Withers wrote:

···

Hi All,

I'm wondering what work people have done with matplotlib in multi-threaded environments such as your average python web framework.

Is matplotlib threadsafe?

How have people gone about safely using it in a multi-threaded environment?

cheers,

Chris

Eric_Firing1 March 19, 2008, 1:18am 3

In general, I don't think mpl is threadsafe at all; it uses global variables, such as all the rc parameters, that could easily be modified by one thread while being used by another. I think that great care would be needed if one wanted to have multiple threads making plots. Having one plotting thread and any number of threads doing other things, however, should be OK.

Michael Droettboom wrote:

At least the Agg backend *looks* to be reasonably threadsafe -- there are no obvious gotchas like global variables etc. Note, though, that multithreading may not gain much in the way of performance since the global interpreter lock is never released around long-running C blocks.

Possibly this could be changed. The danger would be accidentally modifying or deleting an array that is being used by C extension code.

However, I can't speak about this from any experience -- so, maybe it needs some trying. Any patches to help with thread safety and performance are of course welcome

At the very least, I think we would have to take all the global state information and put it in a class instance, so there could be multiple plotting machines. Pyplot would then instantiate and use one of these; the OO API would allow one to instantiate any number of them. I have not thought about how easy or hard this would be.

Eric

···

Cheers,
Mike

Chris Withers wrote:

Hi All,

I'm wondering what work people have done with matplotlib in multi-threaded environments such as your average python web framework.

Is matplotlib threadsafe?

How have people gone about safely using it in a multi-threaded environment?

cheers,

Chris

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

Michael_Droettboom1 March 19, 2008, 12:10pm 4

Eric Firing wrote:

In general, I don't think mpl is threadsafe at all; it uses global variables, such as all the rc parameters, that could easily be modified by one thread while being used by another. I think that great care would be needed if one wanted to have multiple threads making plots. Having one plotting thread and any number of threads doing other things, however, should be OK.

Oh yes -- I didn't think of the rcParams.

Michael Droettboom wrote:

At least the Agg backend *looks* to be reasonably threadsafe -- there are no obvious gotchas like global variables etc. Note, though, that multithreading may not gain much in the way of performance since the global interpreter lock is never released around long-running C blocks.

Possibly this could be changed. The danger would be accidentally modifying or deleting an array that is being used by C extension code.

Modifying may be problematic, but deleting would not be likely -- the C function holds a reference to each of the arrays as they are using them.

However, I can't speak about this from any experience -- so, maybe it needs some trying. Any patches to help with thread safety and performance are of course welcome

At the very least, I think we would have to take all the global state information and put it in a class instance, so there could be multiple plotting machines. Pyplot would then instantiate and use one of these; the OO API would allow one to instantiate any number of them. I have not thought about how easy or hard this would be.

Ditto that. Having just come back from PyCon, I'll parrot the standard Python answer to this question which is: "Don't use threads, use multiple processes", which would seem to solve all these issues -- but, I understand that it not always the best solution.

Mike

···

Eric

Cheers,
Mike

Chris Withers wrote:

Hi All,

I'm wondering what work people have done with matplotlib in multi-threaded environments such as your average python web framework.

Is matplotlib threadsafe?

How have people gone about safely using it in a multi-threaded environment?

cheers,

Chris

-------------------------------------------------------------------------

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

--
Michael Droettboom
Science Software Branch
Operations and Engineering Division
Space Telescope Science Institute
Operated by AURA for NASA

Chris_Withers March 20, 2008, 10:55pm 5

Michael Droettboom wrote:

At least the Agg backend *looks* to be reasonably threadsafe -- there are no obvious gotchas like global variables etc. Note, though, that multithreading may not gain much in the way of performance since the global interpreter lock is never released around long-running C blocks.

It's not performance I'm looking for, it's making sure that MPL apps served from multi-threaded wsgi servers don't screw each others charts up

cheers,

Chris

···

--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk

Chris_Withers March 20, 2008, 10:56pm 6

Eric Firing wrote:

In general, I don't think mpl is threadsafe at all; it uses global variables, such as all the rc parameters, that could easily be modified by one thread while being used by another.

Yep, I guessed as much, BFL it is then

I think that great care would be needed if one wanted to have multiple threads making plots. Having one plotting thread and any number of threads doing other things, however, should be OK.

Out of interest, how does one tell MPL to "start a new figure and forget everything that's gone before"?

At the very least, I think we would have to take all the global state information and put it in a class instance, so there could be multiple plotting machines.

Yes.

cheers,

Chris

···

--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk

Eric_Firing1 March 21, 2008, 6:40am 7

Out of interest, how does one tell MPL to "start a new figure and forget everything that's gone before"?

You can minimize the amount of package and module-level state information by using the oo interface: see examples/agg_oo.py. If you change any rcParams dictionary entries, typically using the rc function, then you can restore the dictionary to its default condition with the rcdefaults() function.

Eric

_John_Hunter1 March 22, 2008, 12:14am 8

I think it is comparatively rare to modify rc params within code (and
never necessary since you can achieve the same by setting properties
directly using the API) so if the rc params are the only global
information we are using (and I am not aware of other problems off the
top of my head but you might be well served to post to the antigrain
mailing list to see if agg is thread safe) it shouldn't pose a serious
problem.

JDH

···

On Fri, Mar 21, 2008 at 1:40 AM, Eric Firing <efiring@...202...> wrote:

> Out of interest, how does one tell MPL to "start a new figure and forget
> everything that's gone before"?

You can minimize the amount of package and module-level state
information by using the oo interface: see examples/agg_oo.py. If you
change any rcParams dictionary entries, typically using the rc function,
then you can restore the dictionary to its default condition with the
rcdefaults() function.

Chris_Withers April 9, 2008, 9:15pm 9

Eric Firing wrote:

Out of interest, how does one tell MPL to "start a new figure and forget everything that's gone before"?

You can minimize the amount of package and module-level state information by using the oo interface: see examples/agg_oo.py.

I tried this example, and it generates no output.
Is that to be expected?

cheers,

Chris

···

--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk

Eric_Firing1 April 9, 2008, 9:23pm 10

Chris Withers wrote:

Eric Firing wrote:

Out of interest, how does one tell MPL to "start a new figure and forget everything that's gone before"?

You can minimize the amount of package and module-level state information by using the oo interface: see examples/agg_oo.py.

I tried this example, and it generates no output.
Is that to be expected?

Chris,

It should produce a "test.png"; it is using a non-interactive backend. It works on my machine.

Eric

···

cheers,

Chris