spy ignores negative values?

BTW would you consider changing the definition of spy(2)

    > as shown below, so that one could specify what 'to be a
    > zero' means?

I added these enhancement, and a couple more, and an
examples/spy_demos.py.

On reflection, it might be better to allow the user to simply pass a
sparsity function rather than a precision

def not_zero(Z):
    return Z!=0.

class not_near_zero:
    def __init__(self, precision):
        self.precision = precision
    def __call__(self, Z):
        return absolute(asarray(Z))>self.precision

def spy(Z, element=not_zero):
    mask = element(Z)

Then you could do:

spy(Z, issparse=not_near_zero(1e-6))

The precision implementation you suggested is in svn, but if there is
any consensus that either of these approaches is better, speak up.

JDH

John Hunter wrote:

    > BTW would you consider changing the definition of spy(2)
    > as shown below, so that one could specify what 'to be a
    > zero' means?

I added these enhancement, and a couple more, and an
examples/spy_demos.py.

On reflection, it might be better to allow the user to simply pass a
sparsity function rather than a precision

def not_zero(Z):
    return Z!=0.

class not_near_zero:
    def __init__(self, precision):
        self.precision = precision
    def __call__(self, Z):
        return absolute(asarray(Z))>self.precision

def spy(Z, element=not_zero):
    mask = element(Z)

Then you could do:

spy(Z, issparse=not_near_zero(1e-6))

The precision implementation you suggested is in svn, but if there is
any consensus that either of these approaches is better, speak up.

I was thinking about passing directly a function (or an expression?)
too. But I would not remove the precision argument, since it's usage is
simpler, and is usually all one needs. The best would be to have both
possibilities :slight_smile: (function, if present, taking precedence?)

r.

John,
One small note - we've been bitten in the past by doing:
    x != 0

This assumes a numeric (int/float) quantity. If someone substitutes a different type that looks like a number, this will most likely fail. Python has a __nonzero__ method which can be used by calling 'not' or 'bool()':
    not x == ! bool( x )

All the numeric types implement this correctly (i.e. like a test for x == 0). Of course, I haven't been following this conversion so I'm not sure this applies here...

Ted

···

At 06:08 AM 11/22/2006, John Hunter wrote:

    > BTW would you consider changing the definition of spy(2)
    > as shown below, so that one could specify what 'to be a
    > zero' means?

I added these enhancement, and a couple more, and an
examples/spy_demos.py.

On reflection, it might be better to allow the user to simply pass a
sparsity function rather than a precision

def not_zero(Z):
    return Z!=0.

class not_near_zero:
    def __init__(self, precision):
        self.precision = precision
    def __call__(self, Z):
        return absolute(asarray(Z))>self.precision

def spy(Z, element=not_zero):
    mask = element(Z)

Then you could do:

spy(Z, issparse=not_near_zero(1e-6))

The precision implementation you suggested is in svn, but if there is
any consensus that either of these approaches is better, speak up.

JDH

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Robert Cimrman wrote:

John Hunter wrote:

    > BTW would you consider changing the definition of spy(2)
    > as shown below, so that one could specify what 'to be a
    > zero' means?

I added these enhancement, and a couple more, and an
examples/spy_demos.py.

On reflection, it might be better to allow the user to simply pass a
sparsity function rather than a precision

def not_zero(Z):
    return Z!=0.

class not_near_zero:
    def __init__(self, precision):
        self.precision = precision
    def __call__(self, Z):
        return absolute(asarray(Z))>self.precision

def spy(Z, element=not_zero):
    mask = element(Z)

Then you could do:

spy(Z, issparse=not_near_zero(1e-6))

The precision implementation you suggested is in svn, but if there is
any consensus that either of these approaches is better, speak up.

I was thinking about passing directly a function (or an expression?)
too. But I would not remove the precision argument, since it's usage is
simpler, and is usually all one needs. The best would be to have both
possibilities :slight_smile: (function, if present, taking precedence?)

I agree--and in fact my uninformed view is that even the precision option is taking spy beyond the realm of showing true sparcity. If this is genuinely useful to people, then fine. But if it (mainly the function option) is something that merely might be useful to someone someday, then I suggest it be left out until there is a clear need. (My 2 cents-worth, or less.)

Curiosity questions about implementation:

1) What is the "tocoo" method, and what objects have it?
2) Is there a reason why one shouldn't simply default precision to 0 and use the condition "absolute(asarray(Z)) <= precision"?

One more miscellaneous thought: perhaps spy and spy2 should be consolidated into a single function with a kwarg to select the marker version or the image version? Their purpose is identical (isn't it?), and it would reduce namespace clutter.

Eric

Eric Firing wrote:

Robert Cimrman wrote:

John Hunter wrote:

    > BTW would you consider changing the definition of spy(2)
    > as shown below, so that one could specify what 'to be a
    > zero' means?

I added these enhancement, and a couple more, and an
examples/spy_demos.py.

On reflection, it might be better to allow the user to simply pass a
sparsity function rather than a precision

def not_zero(Z):
    return Z!=0.

class not_near_zero:
    def __init__(self, precision):
        self.precision = precision
    def __call__(self, Z):
        return absolute(asarray(Z))>self.precision

def spy(Z, element=not_zero):
    mask = element(Z)

Then you could do:

spy(Z, issparse=not_near_zero(1e-6))

The precision implementation you suggested is in svn, but if there is
any consensus that either of these approaches is better, speak up.

I was thinking about passing directly a function (or an expression?)
too. But I would not remove the precision argument, since it's usage is
simpler, and is usually all one needs. The best would be to have both
possibilities :slight_smile: (function, if present, taking precedence?)

I agree--and in fact my uninformed view is that even the precision
option is taking spy beyond the realm of showing true sparcity. If this
is genuinely useful to people, then fine. But if it (mainly the
function option) is something that merely might be useful to someone
someday, then I suggest it be left out until there is a clear need. (My
2 cents-worth, or less.)

Well, the precision argument is useful mainly because of unsafe
comparison of floating point values (what is exactly zero?), so it can
be used, in fact, to show the true sparsity, and I for one would use it :slight_smile:

Curiosity questions about implementation:

1) What is the "tocoo" method, and what objects have it?

a scipy sparse matrix method, to make spy compatible with the sparse
matrix module.

2) Is there a reason why one shouldn't simply default precision to 0 and
use the condition "absolute(asarray(Z)) <= precision"?

just a little (full matrix case) and not so little (sparse matrix case)
performance penalty.

One more miscellaneous thought: perhaps spy and spy2 should be
consolidated into a single function with a kwarg to select the marker
version or the image version? Their purpose is identical (isn't it?),
and it would reduce namespace clutter.

that would be good!

r.

Eric Firing wrote:

One more miscellaneous thought: perhaps spy and spy2 should be
consolidated into a single function with a kwarg to select the marker
version or the image version? Their purpose is identical (isn't it?),
and it would reduce namespace clutter.

one more thing here: usually (e.g. in Matlab) the y axis is reversed, so
that one sees the sparsity pattern in the same position as one would see
on paper when writing down the corresponding system of linear equations.
(I use my own spy, with ax.set_ylim( ax.get_ylim()[::-1] ) for this
purpose.)

just my 2 cents, in case of the consolidation,
r.

Robert Cimrman wrote:

Eric Firing wrote:

One more miscellaneous thought: perhaps spy and spy2 should be consolidated into a single function with a kwarg to select the marker version or the image version? Their purpose is identical (isn't it?), and it would reduce namespace clutter.

one more thing here: usually (e.g. in Matlab) the y axis is reversed, so
that one sees the sparsity pattern in the same position as one would see
on paper when writing down the corresponding system of linear equations.
(I use my own spy, with ax.set_ylim( ax.get_ylim()[::-1] ) for this
purpose.)

just my 2 cents, in case of the consolidation.

Good, I was thinking exactly the same thing, so I will take care of that.

Eric

Robert and any other spy users:

I have committed to svn a new axes method, spy3, that combines a modification of the functionality of both spy and spy2. I hope you can check it out. If it looks OK, then I would like to simply replace spy and spy2 with this combined version. In that case the pylab interface, which now gives access to spy and spy2, would have a single spy function which would access the new version. My suspicion is that spy is used almost entirely in interactive mode, and probably not by very many people, so that this changeover can be made quickly with little disruption to present users.

Attached is a script illustrating the difference in the way spy3 displays a matrix (it matches the way it is printed: first index is row number, second is column number, row number increases down) versus what spy and spy2 do (first index is X, second index is Y).

Also attached is the diff for spy3.

Users may want to comment in particular on the default for the "aspect" kwarg. Presently it is set to "equal" so that the shape of the plot is the shape of the array with square cells. This differs from spy and spy2. The rationale is that it gives the best picture of what the array looks like, including its shape.

Eric

Robert Cimrman wrote:

spytest.py (358 Bytes)

spy.diff (3.85 KB)

···

Eric Firing wrote:

One more miscellaneous thought: perhaps spy and spy2 should be consolidated into a single function with a kwarg to select the marker version or the image version? Their purpose is identical (isn't it?), and it would reduce namespace clutter.

one more thing here: usually (e.g. in Matlab) the y axis is reversed, so
that one sees the sparsity pattern in the same position as one would see
on paper when writing down the corresponding system of linear equations.
(I use my own spy, with ax.set_ylim( ax.get_ylim()[::-1] ) for this
purpose.)

just my 2 cents, in case of the consolidation,
r.

Eric Firing wrote:

Robert and any other spy users:

I have committed to svn a new axes method, spy3, that combines a modification of the functionality of both spy and spy2. I hope you can check it out. If it looks OK, then I would like to simply replace spy and spy2 with this combined version. In that case the pylab interface, which now gives access to spy and spy2, would have a single spy function which would access the new version. My suspicion is that spy is used almost entirely in interactive mode, and probably not by very many people, so that this changeover can be made quickly with little disruption to present users.

Attached is a script illustrating the difference in the way spy3 displays a matrix (it matches the way it is printed: first index is row number, second is column number, row number increases down) versus what spy and spy2 do (first index is X, second index is Y).

Also attached is the diff for spy3.

Users may want to comment in particular on the default for the "aspect" kwarg. Presently it is set to "equal" so that the shape of the plot is the shape of the array with square cells. This differs from spy and spy2. The rationale is that it gives the best picture of what the array looks like, including its shape.

Thumbs up :), just add the sparse-full matrix switch to the imshow branch too, if possible. But I am happy with it as it is.

r.

Robert Cimrman wrote:

Eric Firing wrote:

Robert and any other spy users:

I have committed to svn a new axes method, spy3, that combines a modification of the functionality of both spy and spy2. I hope you can check it out. If it looks OK, then I would like to simply replace spy and spy2 with this combined version. In that case the pylab interface, which now gives access to spy and spy2, would have a single spy function which would access the new version. My suspicion is that spy is used almost entirely in interactive mode, and probably not by very many people, so that this changeover can be made quickly with little disruption to present users.

Attached is a script illustrating the difference in the way spy3 displays a matrix (it matches the way it is printed: first index is row number, second is column number, row number increases down) versus what spy and spy2 do (first index is X, second index is Y).

Also attached is the diff for spy3.

Users may want to comment in particular on the default for the "aspect" kwarg. Presently it is set to "equal" so that the shape of the plot is the shape of the array with square cells. This differs from spy and spy2. The rationale is that it gives the best picture of what the array looks like, including its shape.

Thumbs up :), just add the sparse-full matrix switch to the imshow branch too, if possible. But I am happy with it as it is.

Robert,

The sparse-full difference only works with the plot mode; for an image there is no alternative to providing a value for every pixel, so I don't know of any way to optimize it for the case of sparse storage. A polygon collection could be used to achieve the same result in this case. I'm not sure it is a good idea, though, because there would be an advantage of not converting to a regular array only in the case where the array is so large that such a conversion would use a big chunk of memory, and in that case the polygons probably would be less than single-pixel size anyway, so one would be better off using the present symbol-plotting mode.

Anyway, I'm glad it works for you. Thanks for checking.

Eric

Eric Firing wrote:

Robert Cimrman wrote:

Eric Firing wrote:

Robert and any other spy users:

I have committed to svn a new axes method, spy3, that combines a modification of the functionality of both spy and spy2. I hope you can check it out. If it looks OK, then I would like to simply replace spy and spy2 with this combined version. In that case the pylab interface, which now gives access to spy and spy2, would have a single spy function which would access the new version. My suspicion is that spy is used almost entirely in interactive mode, and probably not by very many people, so that this changeover can be made quickly with little disruption to present users.

Attached is a script illustrating the difference in the way spy3 displays a matrix (it matches the way it is printed: first index is row number, second is column number, row number increases down) versus what spy and spy2 do (first index is X, second index is Y).

Also attached is the diff for spy3.

Users may want to comment in particular on the default for the "aspect" kwarg. Presently it is set to "equal" so that the shape of the plot is the shape of the array with square cells. This differs from spy and spy2. The rationale is that it gives the best picture of what the array looks like, including its shape.

Thumbs up :), just add the sparse-full matrix switch to the imshow branch too, if possible. But I am happy with it as it is.

>

The sparse-full difference only works with the plot mode; for an image there is no alternative to providing a value for every pixel, so I don't know of any way to optimize it for the case of sparse storage. A polygon collection could be used to achieve the same result in this case. I'm not sure it is a good idea, though, because there would be an advantage of not converting to a regular array only in the case where the array is so large that such a conversion would use a big chunk of memory, and in that case the polygons probably would be less than single-pixel size anyway, so one would be better off using the present symbol-plotting mode.

Yeah, I have tried to write it myself before I posted this message but found that it probably would not work well - nevertheless I asked :).
What could be done, though, is to raise an exception explaining that sparse matrices and the image mode don't like each other; as it is, the function spy3 just dies on asarray (should be st. like asarray( Z.todense() ))

In [1]:import numpy as nm
In [2]:import scipy.sparse as sp
In [3]:a = nm.array( [[0, 1], [0,1], [2,3]] )
In [4]:b = sp.csr_matrix( a )
In [5]:gca().spy3( b )

···

---------------------------------------------------------------------------
exceptions.TypeError Traceback (most recent call last)

/home/eldaran/<console>

/usr/lib/python2.4/site-packages/matplotlib/axes.py in spy3(self, Z, precision, marker, markersize, aspect, **kwargs)
    4377 """
    4378 if marker is None and markersize is None:
-> 4379 Z = asarray(Z)
    4380 if precision is None: mask = Z!=0.
    4381 else: mask = absolute(Z)>precision

/usr/lib/python2.4/site-packages/numarray/numarraycore.py in asarray(seq, type, typecode, dtype)
     432 if isinstance(seq, _gen.NDArray) and type is None and typecode is None:
     433 return seq
--> 434 return array(seq, type=type, typecode=typecode, copy=0, dtype=dtype)
     435
     436 inputarray = asarray # Obsolete synonym

/usr/lib/python2.4/site-packages/numarray/numarraycore.py in array(sequence, typecode, copy, savespace, type, shape, dtype)
     415 if (hasattr(sequence,'__getitem__')
     416 and hasattr(sequence,'__len__')):
--> 417 return fromlist(sequence,type,shape)
     418
     419 ##SEQUENCE is a scalar or unhandleable

/usr/lib/python2.4/site-packages/numarray/numarraycore.py in fromlist(seq, type, shape, check_overflow, typecode, dtype)
     246 return arr
     247
--> 248 if not len(seq) and type is None:
     249 type = _nt.Long
     250

/home/share/software/usr/lib/python2.4/site-packages/scipy/sparse/sparse.py in __len__(self)
     163 def __len__(self):
     164 # return self.getnnz()
--> 165 raise TypeError, "sparse matrix length is ambiguous; use getnnz()" \
     166 " or shape[0]"
     167

Anyway, I'm glad it works for you. Thanks for checking.

Thanks for your fast help!
r.

Robert Cimrman wrote:
[...]

What could be done, though, is to raise an exception explaining that sparse matrices and the image mode don't like each other; as it is, the function spy3 just dies on asarray (should be st. like asarray( Z.todense() ))

(I think it would be Z.toarray().)

Good point. Alternatives would be to automatically switch it to plot mode, with a warning, or to automatically convert it. Probably the former is the better of these two, and maybe your originally suggested exception is best of all. What do you think?

Eric

Eric Firing wrote:

Robert Cimrman wrote:
[...]

What could be done, though, is to raise an exception explaining that sparse matrices and the image mode don't like each other; as it is, the function spy3 just dies on asarray (should be st. like asarray( Z.todense() ))

(I think it would be Z.toarray().)

Yes.

Good point. Alternatives would be to automatically switch it to plot mode, with a warning, or to automatically convert it. Probably the former is the better of these two, and maybe your originally suggested exception is best of all. What do you think?

I am slightly more for raising the exception: IMHO, people really needing and using sparse matrices convert never to dense ones (they would not fit into memory in most cases); people happy with automatic conversion should use dense matrices from the very beginning - there is no gain in exploiting sparsity for them; people knowing what they are doing can convert explicitly prior to calling spy().

So:
+1 raise an exception
+0 switch to plot mode (ok, but inconsistent - why I got a plot when I wanted an image?)
-10 automatic conversion

r.