[Matplotlib-users] create ListedColormap with different alpha values

Hey John and the rest of the MPL gang:

I've made the changes you suggested, but the problem is looking to be
deeper than it seemed. I'm also moving this conversation to
matplotlib-devel, since that's probably the more appropriate place for
it.

This updated patch allows for the creation of colormaps with various
alphas, but there is likely more work to be done so that mpl can
consistently make use of it (because it seems like all built-in cmaps
are RGB, not RGBA).

In trying to come up with an example that exercises the new
capabilities, I found out that methods like scatter and countourf modify
the colormap you give them and reset all of the alpha values to 1.

I think this is because inside collections, we pass self._alpha, which
is the Artist._alpha, and 1.0 by default, when making calls such
as:
    _colors.colorConverter.to_rgba_array(c, self._alpha)

...Thus resetting all of alpha values.

I was able to get around this by allowing collections to take on an
alpha value of None, and then passing alpha=None to scatter and
countourf, for example. There are probably other places where such a
change should be done, unless someone has a better idea for how do do
this. I updated examples/pylab/plot_scatter.py to show off the new
capability.

Another thing that I was unable to get around is that if you now make a
plot using the same colormap but omit the alpha=None parameter, or set
it to something other than None, it will reset the alpha values on the
previous plot:

    figure(2)
    c = scatter(theta, r, c=colors, s=area,cmap=myColormap,alpha=None)

will do the right thing, but calling scatter without alpha=None

    figure(3)
    d = scatter(theta, r, c=colors, s=area,cmap=myColormap)
or
    d = scatter(theta, r, c=colors, s=area,cmap=myColormap, alpha=.5)

will reset all of the alpha values in myColormap to 1 or .5.
You can do c.cmap._init() to reset its original alpha values, and if you
force a redraw on figure(2) (by panning or zooming on it, for example),
it will look right again. However, if you go and fiddle with figure(3)
(pan/zoom), and come back to figure(2), panning or zooming will
cause all of the alpha values will be reset again.

I'm not sure if it would be worth it to make a copy of the colormap to
prevent this from happening. Anyone have thoughts on this?

(the full example of this is commented with FIXME: in polar_scatter.py)

best,
    Paul Ivanov

John Hunter, on 2008-11-23 07:36, wrote:

rgba_colormap_v2.diff (6.89 KB)

···

On Sun, Nov 23, 2008 at 2:01 AM, Paul Ivanov <pivanov314@...149...> wrote:

I took a stab at it, how does this look?

I also took the liberty of adding alpha to LinearSegmentedColormap and
updated its docstring changing two somewhat ambiguous uses of the word
'entry' with 'key' and 'value'.

Hey Paul,

Thanks for taking this on. I haven't tested this but I read the patch
and have some inline comments below. Some additional comments:

  * the patch should include a section in the CHANGELOG and
API_CHANGES letting people know what is different.

  * you should run examples/tests/backend_driver.py and make sure all
the examples still run, checking the output of some of the mappable
types (images, scaltter, pcolor...)

  * it would be nice to have an example in the examples dir which
exercises the new capabilities.

See also, in case you haven't,
http://matplotlib.sourceforge.net/devel/coding_guide.html, which
covers some of this in more detail.

Thanks again! Comments below:

    Index: lib/matplotlib/colors.py
    ===================================================================
    --- lib/matplotlib/colors.py (revision 6431)
    +++ lib/matplotlib/colors.py (working copy)
    @@ -452,7 +452,7 @@
             self._isinit = False

    - def __call__(self, X, alpha=1.0, bytes=False):
    + def __call__(self, X, alpha=None, bytes=False):
             """
             *X* is either a scalar or an array (of any dimension).
             If scalar, a tuple of rgba values is returned, otherwise
    @@ -466,9 +466,10 @@
             """
You need to document what alpha can be here: what does None mean, can
it be an array, scalar, etc...

             if not self._isinit: self._init()
    - alpha = min(alpha, 1.0) # alpha must be between 0 and 1
    - alpha = max(alpha, 0.0)
    - self._lut[:-3, -1] = alpha
    + if alpha:

I prefer to explicitly use "if alpha is None", since there are other
things that would test False (0, , '') that you probably don't mean.

    + alpha = min(alpha, 1.0) # alpha must be between 0 and 1
    + alpha = max(alpha, 0.0)

You should be able to use np.clip(alpha, 0, 1) here, but we should
consider instead raising for illegal alpha values since this will be
more helpful to the user. I realize some of this is inherited code
from before your changes, but we can improve it while making this
patch.

    + self._lut[:-3, -1] = alpha
             mask_bad = None
             if not cbook.iterable(X):
                 vtype = 'scalar'
    @@ -558,9 +559,10 @@
         def __init__(self, name, segmentdata, N=256):
             """Create color map from linear mapping segments

    - segmentdata argument is a dictionary with a red, green and blue
    - entries. Each entry should be a list of *x*, *y0*, *y1* tuples,
    - forming rows in a table.
    + segmentdata argument is a dictionary with red, green and blue
    + keys. An optional alpha key is also supported. Each value
    + should be a list of *x*, *y0*, *y1* tuples, forming rows in a
    + table.

             Example: suppose you want red to increase from 0 to 1 over
             the bottom half, green to do the same over the middle half,
    @@ -606,6 +608,8 @@
             self._lut[:-3, 0] = makeMappingArray(self.N,
self._segmentdata['red'])
             self._lut[:-3, 1] = makeMappingArray(self.N,
self._segmentdata['green'])
             self._lut[:-3, 2] = makeMappingArray(self.N,
self._segmentdata['blue'])
    + if self._segmentdata.has_key('alpha'):
    + self._lut[:-3, 3] = makeMappingArray(self.N,
self._segmentdata['blue'])

Is this what you meant? I think you would use 'alpha' rather than
'blue' here, no?

             self._isinit = True
             self._set_extremes()

    @@ -664,11 +668,10 @@

         def _init(self):
    - rgb = np.array([colorConverter.to_rgb(c)
    + rgba = np.array([colorConverter.to_rgba(c)
                         for c in self.colors], np.float)
             self._lut = np.zeros((self.N + 3, 4), np.float)
    - self._lut[:-3, :-1] = rgb
    - self._lut[:-3, -1] = 1
    + self._lut[:-3] = rgba
             self._isinit = True
             self._set_extremes()

Hi all,

thanks for the efforts put in this topic. A few hours after my initial post, I had a working quick&dirty cut&paste solution with minor source code changes without really knowing what I'm doing...

I'm glad that someone with a deeper understanding of the mpl internals is working on a mpl conform solution. I will subscribe to matplotlib-devel to keep track of the changes and help to test patches.

Regards
Simon

Paul Ivanov wrote:

···

Hey John and the rest of the MPL gang:

I've made the changes you suggested, but the problem is looking to be
deeper than it seemed. I'm also moving this conversation to
matplotlib-devel, since that's probably the more appropriate place for
it.

This updated patch allows for the creation of colormaps with various
alphas, but there is likely more work to be done so that mpl can
consistently make use of it (because it seems like all built-in cmaps
are RGB, not RGBA).

In trying to come up with an example that exercises the new
capabilities, I found out that methods like scatter and countourf modify
the colormap you give them and reset all of the alpha values to 1.

I think this is because inside collections, we pass self._alpha, which
is the Artist._alpha, and 1.0 by default, when making calls such
as:
    _colors.colorConverter.to_rgba_array(c, self._alpha)

...Thus resetting all of alpha values.

I was able to get around this by allowing collections to take on an
alpha value of None, and then passing alpha=None to scatter and
countourf, for example. There are probably other places where such a
change should be done, unless someone has a better idea for how do do
this. I updated examples/pylab/plot_scatter.py to show off the new
capability.

Another thing that I was unable to get around is that if you now make a
plot using the same colormap but omit the alpha=None parameter, or set
it to something other than None, it will reset the alpha values on the
previous plot:

    figure(2)
    c = scatter(theta, r, c=colors, s=area,cmap=myColormap,alpha=None)

will do the right thing, but calling scatter without alpha=None

    figure(3)
    d = scatter(theta, r, c=colors, s=area,cmap=myColormap)
or
    d = scatter(theta, r, c=colors, s=area,cmap=myColormap, alpha=.5)

will reset all of the alpha values in myColormap to 1 or .5.
You can do c.cmap._init() to reset its original alpha values, and if you
force a redraw on figure(2) (by panning or zooming on it, for example),
it will look right again. However, if you go and fiddle with figure(3)
(pan/zoom), and come back to figure(2), panning or zooming will
cause all of the alpha values will be reset again.

I'm not sure if it would be worth it to make a copy of the colormap to
prevent this from happening. Anyone have thoughts on this?

(the full example of this is commented with FIXME: in polar_scatter.py)

best,
    Paul Ivanov

John Hunter, on 2008-11-23 07:36, wrote:

On Sun, Nov 23, 2008 at 2:01 AM, Paul Ivanov <pivanov314@...149...> wrote:

I took a stab at it, how does this look?

I also took the liberty of adding alpha to LinearSegmentedColormap and
updated its docstring changing two somewhat ambiguous uses of the word
'entry' with 'key' and 'value'.

Hey Paul,

Thanks for taking this on. I haven't tested this but I read the patch
and have some inline comments below. Some additional comments:

  * the patch should include a section in the CHANGELOG and
API_CHANGES letting people know what is different.

  * you should run examples/tests/backend_driver.py and make sure all
the examples still run, checking the output of some of the mappable
types (images, scaltter, pcolor...)

  * it would be nice to have an example in the examples dir which
exercises the new capabilities.

See also, in case you haven't,
http://matplotlib.sourceforge.net/devel/coding_guide.html, which
covers some of this in more detail.

Thanks again! Comments below:

    Index: lib/matplotlib/colors.py
    ===================================================================
    --- lib/matplotlib/colors.py (revision 6431)
    +++ lib/matplotlib/colors.py (working copy)
    @@ -452,7 +452,7 @@
             self._isinit = False

    - def __call__(self, X, alpha=1.0, bytes=False):
    + def __call__(self, X, alpha=None, bytes=False):
             """
             *X* is either a scalar or an array (of any dimension).
             If scalar, a tuple of rgba values is returned, otherwise
    @@ -466,9 +466,10 @@
             """
You need to document what alpha can be here: what does None mean, can
it be an array, scalar, etc...

             if not self._isinit: self._init()
    - alpha = min(alpha, 1.0) # alpha must be between 0 and 1
    - alpha = max(alpha, 0.0)
    - self._lut[:-3, -1] = alpha
    + if alpha:

I prefer to explicitly use "if alpha is None", since there are other
things that would test False (0, , '') that you probably don't mean.

    + alpha = min(alpha, 1.0) # alpha must be between 0 and 1
    + alpha = max(alpha, 0.0)

You should be able to use np.clip(alpha, 0, 1) here, but we should
consider instead raising for illegal alpha values since this will be
more helpful to the user. I realize some of this is inherited code
from before your changes, but we can improve it while making this
patch.

    + self._lut[:-3, -1] = alpha
             mask_bad = None
             if not cbook.iterable(X):
                 vtype = 'scalar'
    @@ -558,9 +559,10 @@
         def __init__(self, name, segmentdata, N=256):
             """Create color map from linear mapping segments

    - segmentdata argument is a dictionary with a red, green and blue
    - entries. Each entry should be a list of *x*, *y0*, *y1* tuples,
    - forming rows in a table.
    + segmentdata argument is a dictionary with red, green and blue
    + keys. An optional alpha key is also supported. Each value
    + should be a list of *x*, *y0*, *y1* tuples, forming rows in a
    + table.

             Example: suppose you want red to increase from 0 to 1 over
             the bottom half, green to do the same over the middle half,
    @@ -606,6 +608,8 @@
             self._lut[:-3, 0] = makeMappingArray(self.N,
self._segmentdata['red'])
             self._lut[:-3, 1] = makeMappingArray(self.N,
self._segmentdata['green'])
             self._lut[:-3, 2] = makeMappingArray(self.N,
self._segmentdata['blue'])
    + if self._segmentdata.has_key('alpha'):
    + self._lut[:-3, 3] = makeMappingArray(self.N,
self._segmentdata['blue'])

Is this what you meant? I think you would use 'alpha' rather than
'blue' here, no?

             self._isinit = True
             self._set_extremes()

    @@ -664,11 +668,10 @@

         def _init(self):
    - rgb = np.array([colorConverter.to_rgb(c)
    + rgba = np.array([colorConverter.to_rgba(c)
                         for c in self.colors], np.float)
             self._lut = np.zeros((self.N + 3, 4), np.float)
    - self._lut[:-3, :-1] = rgb
    - self._lut[:-3, -1] = 1
    + self._lut[:-3] = rgba
             self._isinit = True
             self._set_extremes()