specgram memory problem

David_Craig · February 3, 2012, 12:11pm

Hi, I am using matplotlib to produce some spectrograms for seismic data. I am looking at a 10 day period with a sample rate of 20sps. I would like to have my spectrogram to be composed of 10 minute windows with an overlap of 90%. However when I try and run my script I run out of memory. I can produce the spectrogram for a maximum of 3 days before an error occurs.
I have also tried to produce a spectrogram for each day and stick them together using subplot, but I then get the error given below. Anyone know a way around this??
Thanks,
David

Traceback (most recent call last):
File “/usr/lib/python2.7/site-packages/matplotlib/backends/backend_gtk.py”, line 394, in expose_event
self._render_figure(self._pixmap, w, h)
File “/usr/lib/python2.7/site-packages/matplotlib/backends/backend_gtkagg.py”, line 75, in _render_figure
FigureCanvasAgg.draw(self)
File “/usr/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py”, line 394, in draw
self.figure.draw(self.renderer)
File “/usr/lib/python2.7/site-packages/matplotlib/artist.py”, line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File “/usr/lib/python2.7/site-packages/matplotlib/figure.py”, line 798, in draw
func(*args)
File “/usr/lib/python2.7/site-packages/matplotlib/artist.py”, line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File “/usr/lib/python2.7/site-packages/matplotlib/axes.py”, line 1946, in draw
a.draw(renderer)
File “/usr/lib/python2.7/site-packages/matplotlib/artist.py”, line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File “/usr/lib/python2.7/site-packages/matplotlib/image.py”, line 354, in draw
im = self.make_image(renderer.get_image_magnification())
File “/usr/lib/python2.7/site-packages/matplotlib/image.py”, line 569, in make_image
transformed_viewLim)
File “/usr/lib/python2.7/site-packages/matplotlib/image.py”, line 201, in _get_unsampled_image
x = self.to_rgba(self._A, self._alpha)
File “/usr/lib/python2.7/site-packages/matplotlib/cm.py”, line 194, in to_rgba
x = self.cmap(x, alpha=alpha, bytes=bytes)
File “/usr/lib/python2.7/site-packages/matplotlib/colors.py”, line 551, in call
rgba = np.empty(shape=xa.shape+(4,), dtype=lut.dtype)
MemoryError

Fabrice_Silva2 · February 3, 2012, 3:54pm

It seems that the MemoryError does not occur when computing the
spectrogram, but when rendering it. A quick rule of a thumb tells me
that you have to display an image that is 12000x14400:
        12000 frequencies, as you are using windows with 12000 samples
        (no padded assumed)
        14400 windows, due to the 90% overlap

Having a 4-channel for the RGBA image may throw the MemoryError.
Can you check by trying to "imshow" such an array ?
You can also reduce the overlap, the one you used lead to a spectrum
computation each minute...

···

Le vendredi 03 février 2012 à 12:11 +0000, David Craig a écrit :

Hi, I am using matplotlib to produce some spectrograms for seismic
data. I am looking at a 10 day period with a sample rate of 20sps. I
would like to have my spectrogram to be composed of 10 minute windows
with an overlap of 90%. However when I try and run my script I run out
of memory. I can produce the spectrogram for a maximum of 3 days
before an error occurs.
I have also tried to produce a spectrogram for each day and stick them
together using subplot, but I then get the error given below. Anyone
know a way around this??
Thanks,
David

Fabrice_Silva2 · February 4, 2012, 9:44am

plt.specgram computes the spectrogram and when calls imshow to display
the resulting array into an image

Please tell the shape of Pxx, and try the following

import numpy as np
import matplotlib.pyplot as plt
a = np.empty((12000, 14400), dtype=float)
plt.imshow(a)
plt.show()

···

Le vendredi 03 février 2012 à 17:39 +0000, David Craig a écrit :

sure how to get it to plot the outputs from specgram. I use
specgram as follows,
Pxx, freqs, bins, im = plt.specgram(......)
what am I trying imshow??

Fabrice_Silva2 · February 6, 2012, 10:29am

Please, answer on the mailing list,
It confirms that the troubles lie in the rendering of images. Could you
tell the versions of numpy and matplotlib you are using, and the
characteristics of the computer you are working on ?

···

On Sat, Feb 4, 2012 at 9:44 AM, Fabrice Silva <silva@...1918...> wrote:

        Le vendredi 03 février 2012 à 17:39 +0000, David Craig a écrit :
        > sure how to get it to plot the outputs from specgram. I use
        > specgram as follows,
        > Pxx, freqs, bins, im = plt.specgram(......)
        > what am I trying imshow??

        plt.specgram computes the spectrogram and when calls imshow to display
        the resulting array into an image

        Please tell the shape of Pxx, and try the following

        import numpy as np
        import matplotlib.pyplot as plt
        a = np.empty((12000, 14400), dtype=float)
        plt.imshow(a)
        plt.show()

Le samedi 04 février 2012 à 10:30 +0000, David Craig a écrit :

Pxx has shape (6001, 1430) and when I tried the lines of code it returned the following memory error,

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_gtk.py", line 394, in expose_event
    self._render_figure(self._pixmap, w, h)
  File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_gtkagg.py", line 75, in _render_figure
    FigureCanvasAgg.draw(self)
  File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 394, in draw
    self.figure.draw(self.renderer)
  File "/usr/lib/python2.7/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/matplotlib/figure.py", line 798, in draw
    func(*args)
  File "/usr/lib/python2.7/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/matplotlib/axes.py", line 1946, in draw
    a.draw(renderer)
  File "/usr/lib/python2.7/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/matplotlib/image.py", line 354, in draw
    im = self.make_image(renderer.get_image_magnification())
  File "/usr/lib/python2.7/site-packages/matplotlib/image.py", line 569, in make_image
    transformed_viewLim)
  File "/usr/lib/python2.7/site-packages/matplotlib/image.py", line 201, in _get_unsampled_image
    x = self.to_rgba(self._A, self._alpha)
  File "/usr/lib/python2.7/site-packages/matplotlib/cm.py", line 193, in to_rgba
    x = self.norm(x)
  File "/usr/lib/python2.7/site-packages/matplotlib/colors.py", line 802, in __call__
    val = ma.asarray(value).astype(np.float)
  File "/usr/lib/python2.7/site-packages/numpy/ma/core.py", line 2908, in astype
    output = self._data.astype(newtype).view(type(self))
MemoryError

David_Craig · February 6, 2012, 5:59pm

I'm using a lenovo laptop with fedora 16. It has 2.9 GiB memory and 4 intel core CPUs @ 2.3GHz each. Available disk space is 147.9GiB.
numpy 1.6.0
matplotlib 1.0.1

···

On 6 Feb 2012, at 10:29, Fabrice Silva wrote:

On Sat, Feb 4, 2012 at 9:44 AM, Fabrice Silva <silva@...3954... > mrs.fr> wrote:

        Le vendredi 03 février 2012 à 17:39 +0000, David Craig a écrit :

sure how to get it to plot the outputs from specgram. I use
specgram as follows,
Pxx, freqs, bins, im = plt.specgram(......)
what am I trying imshow??

        plt.specgram computes the spectrogram and when calls imshow to display
        the resulting array into an image

        Please tell the shape of Pxx, and try the following

        import numpy as np
        import matplotlib.pyplot as plt
        a = np.empty((12000, 14400), dtype=float)
        plt.imshow(a)
        plt.show()

Le samedi 04 février 2012 à 10:30 +0000, David Craig a écrit :

Pxx has shape (6001, 1430) and when I tried the lines of code it returned the following memory error,

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_gtk.py", line 394, in expose_event
    self._render_figure(self._pixmap, w, h)
  File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_gtkagg.py", line 75, in _render_figure
    FigureCanvasAgg.draw(self)
  File "/usr/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 394, in draw
    self.figure.draw(self.renderer)
  File "/usr/lib/python2.7/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/matplotlib/figure.py", line 798, in draw
    func(*args)
  File "/usr/lib/python2.7/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/matplotlib/axes.py", line 1946, in draw
    a.draw(renderer)
  File "/usr/lib/python2.7/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/matplotlib/image.py", line 354, in draw
    im = self.make_image(renderer.get_image_magnification())
  File "/usr/lib/python2.7/site-packages/matplotlib/image.py", line 569, in make_image
    transformed_viewLim)
  File "/usr/lib/python2.7/site-packages/matplotlib/image.py", line 201, in _get_unsampled_image
    x = self.to_rgba(self._A, self._alpha)
  File "/usr/lib/python2.7/site-packages/matplotlib/cm.py", line 193, in to_rgba
    x = self.norm(x)
  File "/usr/lib/python2.7/site-packages/matplotlib/colors.py", line 802, in __call__
    val = ma.asarray(value).astype(np.float)
  File "/usr/lib/python2.7/site-packages/numpy/ma/core.py", line 2908, in astype
    output = self._data.astype(newtype).view(type(self))
MemoryError

Please, answer on the mailing list,
It confirms that the troubles lie in the rendering of images. Could you
tell the versions of numpy and matplotlib you are using, and the
characteristics of the computer you are working on ?

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
matplotlib-users List Signup and Options

Benjamin_Root · February 6, 2012, 6:07pm

32-bit or 64-bit OS? Please use ‘uname -a’ to tell us, because you can install a 32-bit OS on a 64-bit machine.

Ben Root

···

On Mon, Feb 6, 2012 at 11:59 AM, David Craig <dcdavemail@…287…> wrote:

I’m using a lenovo laptop with fedora 16. It has 2.9 GiB memory and 4

intel core CPUs @ 2.3GHz each. Available disk space is 147.9GiB.

numpy 1.6.0

matplotlib 1.0.1

David_Craig · February 6, 2012, 7:56pm

uname -a gives,
Linux David 3.2.2-1.fc16.i686 #1 SMP Thu Jan 26 03:38:31 UTC 2012 i686 i686 i386 GNU/Linux

···

On Mon, Feb 6, 2012 at 6:07 PM, Benjamin Root <ben.root@…1304…> wrote:

On Mon, Feb 6, 2012 at 11:59 AM, David Craig <dcdavemail@…1003…7…> wrote:

I’m using a lenovo laptop with fedora 16. It has 2.9 GiB memory and 4

intel core CPUs @ 2.3GHz each. Available disk space is 147.9GiB.

numpy 1.6.0

matplotlib 1.0.1

32-bit or 64-bit OS? Please use ‘uname -a’ to tell us, because you can install a 32-bit OS on a 64-bit machine.

Ben Root