matplotlib FuncAnimation hanging issue

Colin_McAuliffe · May 6, 2013, 12:12am

Hello all,

I'm having an issue with FuncAnimation with ffmpeg. For a small number of frames everything works fine, but for some reason with more than 600 frames the program hangs indefinitely. Is there some kind of frame limit for FuncAmimation or is there a mistake in the arguements I am passing?

ani = ani.FuncAnimation(fig, animate,np.range(1,700), interval=25,blit=True, init_func=init)

Thanks and all the best
Colin

···

--
Colin McAuliffe
PhD Candidate
Columbia University
Department of Civil Engineering and Engineering Mechanics

Tony_S_Yu3 · May 6, 2013, 4:56am

Hi Colin,

Which version of Matplotlib are you running? This sounds similar to a bug
addressed by the following change:

github.com/matplotlib/matplotlib

Animation subprocess bug

matplotlib:master ← tonysyu:animation-subprocess-bug

opened 06:50PM - 07 Jul 12 UTC

tonysyu

+16 -6

Ffmpeg prints a lot of info to stderr. For longer animations, this logging fills… up the (very-small) buffer for subprocess.PIPE and subprocess just hangs (with no error). This PR suppresses logging in ffmpeg to prevent this issue. If matplotlib's verbosity level is set to 'debug*', then ffmpeg's output is piped to stdout. **Edit**: I forgot to post code that reproduces the issue. This is pretty much what I posted to the mailing list. I use `streamplot` mainly because it produces more ffmpeg output so it fails with fewer iterations. The following should hang (this seems to vary on my system, but it's usually close to 200 frames in). ``` python import numpy as np import matplotlib.pyplot as plt import matplotlib.animation as animation fig, ax = plt.subplots() # do nothing function that prints frame def update(frame_num): print frame_num return [] Y, X = np.mgrid[:1:100j, :1:100j] U = np.ones_like(X) V = np.ones_like(X) ax.streamplot(X, Y, U, V, density=1) ani = animation.FuncAnimation(fig, update, save_count=500) ani.save('animation.avi') #plt.show() ```

This change has already made it into the current release (v1.2).

Best,
-Tony

···

On Sun, May 5, 2013 at 7:12 PM, Colin McAuliffe <cjm2176@...1353...>wrote:

Hello all,

I'm having an issue with FuncAnimation with ffmpeg. For a small number
of frames everything works fine, but for some reason with more than
600 frames the program hangs indefinitely. Is there some kind of frame
limit for FuncAmimation or is there a mistake in the arguements I am
passing?

Colin_McAuliffe · May 6, 2013, 12:09pm

Hi Tony, thanks for the reply.

I was using 1.2.0 and just upgraded to 1.2.1 but the problem persists. I ran the example code from the link and it hangs after 350-400 frames. Also, I got an error when running the code as it is posted and had to add:

import matplotlib
matplotlib.use("Agg")

to get it to work. Is this an incorrect setting I'm using?

Colin

Quoting Tony Yu <tsyu80@...287...>:

···

On Sun, May 5, 2013 at 7:12 PM, Colin McAuliffe <cjm2176@...1353...>wrote:

Hello all,

I'm having an issue with FuncAnimation with ffmpeg. For a small number
of frames everything works fine, but for some reason with more than
600 frames the program hangs indefinitely. Is there some kind of frame
limit for FuncAmimation or is there a mistake in the arguements I am
passing?

Hi Colin,

Which version of Matplotlib are you running? This sounds similar to a bug
addressed by the following change:

Animation subprocess bug by tonysyu · Pull Request #989 · matplotlib/matplotlib · GitHub

This change has already made it into the current release (v1.2).

Best,
-Tony

--
Colin McAuliffe
PhD Candidate
Columbia University
Department of Civil Engineering and Engineering Mechanics

Tony_S_Yu3 · May 7, 2013, 3:41am

Hi Tony, thanks for the reply.

I was using 1.2.0 and just upgraded to 1.2.1 but the problem persists. I
ran the example code from the link and it hangs after 350-400 frames. Also,
I got an error when running the code as it is posted and had to add:

import matplotlib
matplotlib.use("Agg")

to get it to work. Is this an incorrect setting I'm using?

Colin

Hmm, that's strange: Your problem sounds too similar to be a different bug.
Could you copy the error message you got? It might be a clue. Also, what
backend are you running?

import matplotlib.pyplot as plt
print plt.rcParams['backend']

Another possibility (longshot) is that your version of `ffmpeg` may not
respect the `-loglevel quiet` flag being passed to suppress output. Maybe
you could try running an `ffmpeg` command with and without that flag to see
if it works.

-Tony

···

On Mon, May 6, 2013 at 7:09 AM, Colin McAuliffe <cjm2176@...1353...>wrote:

Colin_McAuliffe · May 7, 2013, 11:48am

The default backend is macosx but using it leads to the error I mentioned:

Traceback (most recent call last):
   File "./fftest.py", line 24, in <module>
     ani.save('animation.avi')
   File "/Library/Python/2.6/site-packages/matplotlib/animation.py", line 615, in save
     writer.grab_frame()
   File "/Library/Python/2.6/site-packages/matplotlib/animation.py", line 199, in grab_frame
     dpi=self.dpi)
   File "/Library/Python/2.6/site-packages/matplotlib/figure.py", line 1370, in savefig
     self.canvas.print_figure(*args, **kwargs)
   File "/Library/Python/2.6/site-packages/matplotlib/backend_bases.py", line 2015, in print_figure
     print_method = self._get_print_method(format)
   File "/Library/Python/2.6/site-packages/matplotlib/backend_bases.py", line 1956, in _get_print_method
     '%s.' % (format, ', '.join(formats)))
ValueError: Format "rgba" is not supported.
Supported formats: bmp, emf, eps, gif, jpeg, jpg, pdf, pgf, png, ps, raw, rgba, svg, svgz, tif, tiff.

The above error does not occur if I switch to agg.

Also using the ffmpeg command

ffmpeg -f image2 -i t%d.jpg video.avi

on a few images gives the following output, where I put !! next to the lines which are suppressed by including -loglevel quiet

FFmpeg version SVN-r26402, Copyright (c) 2000-2011 the FFmpeg developers
   built on May 2 2013 23:13:41 with llvm_gcc 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00)
   configuration: --enable-libmp3lame --enable-shared --disable-mmx --arch=x86_64
   libavutil 50.36. 0 / 50.36. 0
   libavcore 0.16. 1 / 0.16. 1
   libavcodec 52.108. 0 / 52.108. 0
   libavformat 52.93. 0 / 52.93. 0
   libavdevice 52. 2. 3 / 52. 2. 3
   libavfilter 1.74. 0 / 1.74. 0
   libswscale 0.12. 0 / 0.12. 0
Input #0, image2, from 't%d.jpg':
   Duration: 00:00:00.12, start: 0.000000, bitrate: N/A
     Stream #0.0: Video: mjpeg, yuvj422p, 4272x2848, 25 tbr, 25 tbn, 25 tbc
File 'video.avi' already exists. Overwrite ? [y/N] y
!![buffer @ 0x7fff094014b0] w:4272 h:2848 pixfmt:yuvj422p
!![ffsink @ 0x7fff094016d0] auto-inserting filter 'auto-inserted scaler 0' between the filter 'src' and the filter 'out'
!![scale @ 0x7fff094018f0] w:4272 h:2848 fmt:yuvj422p -> w:4272 h:2848 fmt:yuv420p flags:0x4
!!Output #0, avi, to 'video.avi':
!! Metadata:
!! ISFT : Lavf52.93.0
!! Stream #0.0: Video: mpeg4, yuv420p, 4272x2848, q=2-31, 200 kb/s, 25 tbn, 25 tbc
Stream mapping:
   Stream #0.0 -> #0.0
Press [q] to stop encoding
frame= 3 fps= 2 q=4.0 Lsize= 524kB time=0.12 bitrate=35781.7kbits/s
video:519kB audio:0kB global headers:0kB muxing overhead 1.083673%

Quoting Tony Yu <tsyu80@...287...>:

···

On Mon, May 6, 2013 at 7:09 AM, Colin McAuliffe <cjm2176@...1353...>wrote:

Hi Tony, thanks for the reply.

I was using 1.2.0 and just upgraded to 1.2.1 but the problem persists. I
ran the example code from the link and it hangs after 350-400 frames. Also,
I got an error when running the code as it is posted and had to add:

import matplotlib
matplotlib.use("Agg")

to get it to work. Is this an incorrect setting I'm using?

Colin

Hmm, that's strange: Your problem sounds too similar to be a different bug.
Could you copy the error message you got? It might be a clue. Also, what
backend are you running?

import matplotlib.pyplot as plt
print plt.rcParams['backend']

Another possibility (longshot) is that your version of `ffmpeg` may not
respect the `-loglevel quiet` flag being passed to suppress output. Maybe
you could try running an `ffmpeg` command with and without that flag to see
if it works.

-Tony

--
Colin McAuliffe
PhD Candidate
Columbia University
Department of Civil Engineering and Engineering Mechanics

Colin_McAuliffe · May 12, 2013, 12:08am

To add to the previous email I've tried this with a few different backends with no luck. Have similar issues been found when creating videos with mencoder? If the final result is the same switching away from ffmpeg is fine.

Quoting Colin McAuliffe <cjm2176@...1353...>:

···

The default backend is macosx but using it leads to the error I mentioned:

Traceback (most recent call last):
  File "./fftest.py", line 24, in <module>
    ani.save('animation.avi')
  File "/Library/Python/2.6/site-packages/matplotlib/animation.py",
line 615, in save
    writer.grab_frame()
  File "/Library/Python/2.6/site-packages/matplotlib/animation.py",
line 199, in grab_frame
    dpi=self.dpi)
  File "/Library/Python/2.6/site-packages/matplotlib/figure.py", line
1370, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/matplotlib/backend_bases.py",
line 2015, in print_figure
    print_method = self._get_print_method(format)
  File "/Library/Python/2.6/site-packages/matplotlib/backend_bases.py",
line 1956, in _get_print_method
    '%s.' % (format, ', '.join(formats)))
ValueError: Format "rgba" is not supported.
Supported formats: bmp, emf, eps, gif, jpeg, jpg, pdf, pgf, png, ps,
raw, rgba, svg, svgz, tif, tiff.

The above error does not occur if I switch to agg.

Also using the ffmpeg command

ffmpeg -f image2 -i t%d.jpg video.avi

on a few images gives the following output, where I put !! next to the
lines which are suppressed by including -loglevel quiet

FFmpeg version SVN-r26402, Copyright (c) 2000-2011 the FFmpeg developers
  built on May 2 2013 23:13:41 with llvm_gcc 4.2.1 (Based on Apple
Inc. build 5658) (LLVM build 2336.9.00)
  configuration: --enable-libmp3lame --enable-shared --disable-mmx
--arch=x86_64
  libavutil 50.36. 0 / 50.36. 0
  libavcore 0.16. 1 / 0.16. 1
  libavcodec 52.108. 0 / 52.108. 0
  libavformat 52.93. 0 / 52.93. 0
  libavdevice 52. 2. 3 / 52. 2. 3
  libavfilter 1.74. 0 / 1.74. 0
  libswscale 0.12. 0 / 0.12. 0
Input #0, image2, from 't%d.jpg':
  Duration: 00:00:00.12, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: mjpeg, yuvj422p, 4272x2848, 25 tbr, 25 tbn, 25 tbc
File 'video.avi' already exists. Overwrite ? [y/N] y
!![buffer @ 0x7fff094014b0] w:4272 h:2848 pixfmt:yuvj422p
!![ffsink @ 0x7fff094016d0] auto-inserting filter 'auto-inserted scaler
0' between the filter 'src' and the filter 'out'
!![scale @ 0x7fff094018f0] w:4272 h:2848 fmt:yuvj422p -> w:4272 h:2848
fmt:yuv420p flags:0x4
!!Output #0, avi, to 'video.avi':
!! Metadata:
!! ISFT : Lavf52.93.0
!! Stream #0.0: Video: mpeg4, yuv420p, 4272x2848, q=2-31, 200 kb/s,
25 tbn, 25 tbc
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop encoding
frame= 3 fps= 2 q=4.0 Lsize= 524kB time=0.12 bitrate=35781.7kbits/s
video:519kB audio:0kB global headers:0kB muxing overhead 1.083673%

Quoting Tony Yu <tsyu80@...287...>:

On Mon, May 6, 2013 at 7:09 AM, Colin McAuliffe <cjm2176@...1353...>wrote:

Hi Tony, thanks for the reply.

I was using 1.2.0 and just upgraded to 1.2.1 but the problem persists. I
ran the example code from the link and it hangs after 350-400 frames. Also,
I got an error when running the code as it is posted and had to add:

import matplotlib
matplotlib.use("Agg")

to get it to work. Is this an incorrect setting I'm using?

Colin

Hmm, that's strange: Your problem sounds too similar to be a different bug.
Could you copy the error message you got? It might be a clue. Also, what
backend are you running?

import matplotlib.pyplot as plt
print plt.rcParams['backend']

Another possibility (longshot) is that your version of `ffmpeg` may not
respect the `-loglevel quiet` flag being passed to suppress output. Maybe
you could try running an `ffmpeg` command with and without that flag to see
if it works.

-Tony

--
Colin McAuliffe
PhD Candidate
Columbia University
Department of Civil Engineering and Engineering Mechanics

--
Colin McAuliffe
PhD Candidate
Columbia University
Department of Civil Engineering and Engineering Mechanics

Tony_S_Yu3 · May 13, 2013, 2:14pm

To add to the previous email I've tried this with a few different backends
with no luck. Have similar issues been found when creating videos with
mencoder? If the final result is the same switching away from ffmpeg is
fine.

Sorry for the delayed reply. I don't have mencoder on my system, but it's
probably worth a try. More comments below.

Quoting Colin McAuliffe <cjm2176@...1353...>:

The above error does not occur if I switch to agg.

Also using the ffmpeg command

ffmpeg -f image2 -i t%d.jpg video.avi

on a few images gives the following output, where I put !! next to the
lines which are suppressed by including -loglevel quiet

FFmpeg version SVN-r26402, Copyright (c) 2000-2011 the FFmpeg developers
built on May 2 2013 23:13:41 with llvm_gcc 4.2.1 (Based on Apple
Inc. build 5658) (LLVM build 2336.9.00)

<snip>

!!Output #0, avi, to 'video.avi':

!! Metadata:
!! ISFT : Lavf52.93.0
!! Stream #0.0: Video: mpeg4, yuv420p, 4272x2848, q=2-31, 200 kb/s,
25 tbn, 25 tbc
Stream mapping:
Stream #0.0 -> #0.0
Press [q] to stop encoding
frame= 3 fps= 2 q=4.0 Lsize= 524kB time=0.12
bitrate=35781.7kbits/s
video:519kB audio:0kB global headers:0kB muxing overhead 1.083673%

So the line starting with "frame=" doesn't get suppressed when you run with
`-loglevel quiet`? I actually thought those were the problematic lines
(since more of those get printed out for longer encoding tasks). Maybe this
is an issue with the ffmpeg version?

Unfortunately, I don't have much free time to reply in more depth.

Best,
-Tony

···

On Sat, May 11, 2013 at 7:08 PM, Colin McAuliffe <cjm2176@...1353...>wrote:

Tony_S_Yu3 · May 13, 2013, 2:19pm

Assuming this is the same issue as PR 989, redirecting stdout to the command line will prevent the buffer issues. (You’ll get a lot of debug output printed to the command line, though).

Hope that helps,

-Tony

···

On Mon, May 13, 2013 at 9:14 AM, Tony Yu <tsyu80@…287…> wrote:

Oh, yeah: If you just want to try something to get this working, try setting the verbosity level:

On Sat, May 11, 2013 at 7:08 PM, Colin McAuliffe <cjm2176@…1353…> wrote:

To add to the previous email I’ve tried this with a few different backends with no luck. Have similar issues been found when creating videos with mencoder? If the final result is the same switching away from ffmpeg is fine.

Sorry for the delayed reply. I don’t have mencoder on my system, but it’s probably worth a try. More comments below.

Quoting Colin McAuliffe <cjm2176@…1353…>:

The above error does not occur if I switch to agg.

Also using the ffmpeg command

ffmpeg -f image2 -i t%d.jpg video.avi

on a few images gives the following output, where I put !! next to the

lines which are suppressed by including -loglevel quiet

FFmpeg version SVN-r26402, Copyright (c) 2000-2011 the FFmpeg developers

built on May 2 2013 23:13:41 with llvm_gcc 4.2.1 (Based on Apple

Inc. build 5658) (LLVM build 2336.9.00)

!!Output #0, avi, to ‘video.avi’:

!! Metadata:

!! ISFT : Lavf52.93.0

!! Stream #0.0: Video: mpeg4, yuv420p, 4272x2848, q=2-31, 200 kb/s,

25 tbn, 25 tbc

Stream mapping:

Stream #0.0 → #0.0

Press [q] to stop encoding

frame= 3 fps= 2 q=4.0 Lsize= 524kB time=0.12 bitrate=35781.7kbits/s

video:519kB audio:0kB global headers:0kB muxing overhead 1.083673%

So the line starting with “frame=” doesn’t get suppressed when you run with -loglevel quiet? I actually thought those were the problematic lines (since more of those get printed out for longer encoding tasks). Maybe this is an issue with the ffmpeg version?

Unfortunately, I don’t have much free time to reply in more depth.

Best,

-Tony

import matplotlib

matplotlib.verbose.set_level(‘debug’)