Request for feedback regarding creation of a plot-type object.

John_Hunter1 · November 2, 2005, 4:24am

Before you go off in this direction, could you outline how

    > you think colormaps should work? From the sounds of it, you
    > are addressing mostly cases where users have specifically
    > picked colors and you would like colorbar() to work with
    > the selected colors. Generally speaking, you seem to be

Isn't this closely related to the idea we've tslked about a number of
times (mostly off list) to supplant the colormap infrastructure with a
"DiscreteColormap" or something along those lines, which mapped data
to a set of discrete colors, using nearest neighbor or what have
you. Then you would have the best of both worlds: your favorite
colors and consistency with the mpl colorbar/colormapping API. Would
this work?

JDH

Jordan_Dawe · November 3, 2005, 5:55am

Isn't this closely related to the idea we've tslked about a number of
times (mostly off list) to supplant the colormap infrastructure with a
"DiscreteColormap" or something along those lines, which mapped data
to a set of discrete colors, using nearest neighbor or what have
you. Then you would have the best of both worlds: your favorite
colors and consistency with the mpl colorbar/colormapping API. Would
this work?

I don't quite understand the idea here, but the colorbar mapping is really only part of this. If you make classes for each plot type, you could do things like make legend() a call to the PlotClass.Legend() method, and each plot could make it's own kind of legend.

I know this is a large architecture change, but it could be implimented incrimentally and I think it would give a lot of benefits in regards to what you could do with customizing different plot behaviours. But, as I said before, I don't really have nearly the grasp of the matplotlib codebase that the devs do. It doesn't look too difficult to me, but there are probably issues I am not aware of. Is there any reason each plot type shouldn't have it's own class?

Jordan

_Perry_Greenfield · November 3, 2005, 3:17pm

I get the feeling that two different ideas are being discussed here. A discrete color map still would require someone to define a custom one if they have a favorite set of colors they want to use for each contour level. That involves some level of inconvenience. I gather from what Jordan is saying is that he wants to be able to automatically construct a colorbar from the colors assigned to each contour level without having to construct a colormap in the first place. That is, after the contour is done with its specific color/level assignments, then a request for a colorbar would show a linear relationship between data levels and the discrete colors chosen. Do I understand correctly?

Perry

···

On Nov 3, 2005, at 12:55 AM, Jordan Dawe wrote:

Isn't this closely related to the idea we've tslked about a number of
times (mostly off list) to supplant the colormap infrastructure with a
"DiscreteColormap" or something along those lines, which mapped data
to a set of discrete colors, using nearest neighbor or what have
you. Then you would have the best of both worlds: your favorite
colors and consistency with the mpl colorbar/colormapping API. Would
this work?

I don't quite understand the idea here, but the colorbar mapping is really only part of this. If you make classes for each plot type, you could do things like make legend() a call to the PlotClass.Legend() method, and each plot could make it's own kind of legend.

I know this is a large architecture change, but it could be implimented incrimentally and I think it would give a lot of benefits in regards to what you could do with customizing different plot behaviours. But, as I said before, I don't really have nearly the grasp of the matplotlib codebase that the devs do. It doesn't look too difficult to me, but there are probably issues I am not aware of. Is there any reason each plot type shouldn't have it's own class?

Eric_Firing2 · November 3, 2005, 6:22pm

Perry Greenfield wrote:

Isn't this closely related to the idea we've tslked about a number of
times (mostly off list) to supplant the colormap infrastructure with a
"DiscreteColormap" or something along those lines, which mapped data
to a set of discrete colors, using nearest neighbor or what have
you. Then you would have the best of both worlds: your favorite
colors and consistency with the mpl colorbar/colormapping API. Would
this work?

I don't quite understand the idea here, but the colorbar mapping is really only part of this. If you make classes for each plot type, you could do things like make legend() a call to the PlotClass.Legend() method, and each plot could make it's own kind of legend.

I know this is a large architecture change, but it could be implimented incrimentally and I think it would give a lot of benefits in regards to what you could do with customizing different plot behaviours. But, as I said before, I don't really have nearly the grasp of the matplotlib codebase that the devs do. It doesn't look too difficult to me, but there are probably issues I am not aware of. Is there any reason each plot type shouldn't have it's own class?

I get the feeling that two different ideas are being discussed here. A discrete color map still would require someone to define a custom one if they have a favorite set of colors they want to use for each contour level. That involves some level of inconvenience. I gather from what Jordan is saying is that he wants to be able to automatically construct a colorbar from the colors assigned to each contour level without having to construct a colormap in the first place. That is, after the contour is done with its specific color/level assignments, then a request for a colorbar would show a linear relationship between data levels and the discrete colors chosen. Do I understand correctly?

Automatic colorbar generation when the contourf (for example) colors are given explicitly is certainly one of the things Jordan is talking about, and it is something I have been intending to get to. I think that a first working version, for contour and contourf, can be done with little modification to the present code, and I am inclined to do that at least as an interim measure ASAP--within a few days--unless a better idea becomes clear.

That brings us to the larger framework, and here I think there are two or three general ideas rattling around, overlapping but not mutually exclusive.

1) The ScalarMappable class could be extended to include additional mapping methods. Here are four possibilities:
   - boundaries: this is what contourf already does; N-1 colors are mapped to the regions defined by N boundaries.
   - nearest-neighbor: similar but N-to-N
   - dictionary: maps hashable object to color, given an existing dictionary
   - indexed array: Matlab has this, and I use it extensively in Matlab to deal with exactly the problem that motivated Jordan's message; but it might not be necessary for mpl.

2) When I changed contourf to output a ContourSet object, John suggested that this approach might be used for other types of plot (certainly pcolor, for example), and this is getting closer to what Jordan is talking about as a framework, I think. The idea is to package all useful information in an object, so that either its methods can be called later, or it can be passed to a function (such as colorbar) that understands what to do with it.

3) Jordan's suggestion seems to be going a little farther; I think the idea is to design a class hierarchy for plot types to accomplish the goals of (2) in a more systematic way, rather than dealing with each plot type ad hoc as the demand arises.

Eric

···

On Nov 3, 2005, at 12:55 AM, Jordan Dawe wrote:

Jordan_Dawe · November 3, 2005, 6:23pm

I get the feeling that two different ideas are being discussed here. A discrete color map still would require someone to define a custom one if they have a favorite set of colors they want to use for each contour level. That involves some level of inconvenience. I gather from what Jordan is saying is that he wants to be able to automatically construct a colorbar from the colors assigned to each contour level without having to construct a colormap in the first place. That is, after the contour is done with its specific color/level assignments, then a request for a colorbar would show a linear relationship between data levels and the discrete colors chosen. Do I understand correctly?

Yes, that's it exactly.

Jordan

Eric_Firing2 · November 27, 2005, 8:40pm

John,

I have committed to CVS a set of changes that I think will address recent requests by Jordan and Gerald, and will be of more general use as well. From the CHANGELOG:

2005-11-27 Multiple changes in cm.py, colors.py, figure.py, image.py,
            contour.py, contour_demo.py; new _cm.py, examples/image_masked.py.
            1) Separated the color table data from cm.py out into
            a new file, _cm.py, to make it easier to find the actual
            code in cm.py and to add new colormaps. Also added
            some line breaks to the color data dictionaries. Everything
            from _cm.py is imported by cm.py, so the split should be
            transparent.
            2) Enabled automatic generation of a colormap from
            a list of colors in contour; see modified
            examples/contour_demo.py.
            3) Support for imshow of a masked array, with the
            ability to specify colors (or no color at all) for
            masked regions, and for regions that are above or
            below the normally mapped region. See
            examples/image_masked.py.
            4) In support of the above, added two new classes,
            ListedColormap, and no_norm, to colors.py, and modified
            the Colormap class to include common functionality. Added
            a clip kwarg to the normalize class. Reworked color
            handling in contour.py, especially in the ContourLabeller
            mixin.

With one very subtle exception, I don't think any default behaviors have been changed, so the changes should be entirely non-disruptive.

That one exception is that in the original color mapping scheme, the last color was essentially never used. Consider an extreme case: a colormap with two entries. Suppose the image data ranged from 0 to 1. All values except exactly 1.0 were mapped to the first color, and only that upper limit value was mapped to the second color. As I have changed colors.py, values from 0.0 up to 0.5 will get the first color, and values from 0.5 through 1.0 will get the second. I think this is what a user would expect, and the only reason it hasn't mattered is that with 256 colors in a continuous range, one can't see the difference.

The request from Jordan that is addressed here is the desire for precise color control in filled contouring. Now one can specify a list of colors in the "colors" kwarg, and they will be used to generate a colormap, which will then be used by colorbar. This is shown in a second plot added to examples/contourf_demo.py. (Jordan suggested a more extensive refactoring, which may indeed be a good idea; but I wanted to make these simpler changes before trying to think about anything more drastic.)

The request from Gerald was for easy handling of masked arrays in imshow. It was in the context of basemap, which I have not tested yet; but at least for pylab.imshow, the situation is now easy to handle. If changes to basemap are needed, they should be very simple, and I can do them later as needed. I think that the changes to colormapping that I made to support this will be useful much more widely, but I have made no attempt to track down the places where changes will be in order. I may make additional changes to contour, and I know I will need to change colorbar to fully support this. I think colorbar needs some more refactoring anyway, but I can't do it immediately, and I did not want to delay getting the other changes out for testing and, hopefully, productive use.

Eric

Eric_Firing2 · November 27, 2005, 9:02pm

John,

I think that the following change, from API_CHANGES, is causing trouble:

made pos=None the default for tick formatters rather than 0 to
indicate "not supplied"

The symptom is that running (for example) contour_demo.py from the command line with gtkagg backend, I often, but not always, get a blast of errors like this:

Traceback (most recent call last):
   File "/usr/lib/python2.4/site-packages/matplotlib/backends/backend_gtk.py", line 188, in motion_notify_event
     FigureCanvasBase.motion_notify_event(self, x, y)
   File "/usr/lib/python2.4/site-packages/matplotlib/backend_bases.py", line 797, in motion_notify_event
     func(event)
   File "/usr/lib/python2.4/site-packages/matplotlib/backend_bases.py", line 1085, in mouse_move
     try: s = event.inaxes.format_coord(event.xdata, event.ydata)
   File "/usr/lib/python2.4/site-packages/matplotlib/axes.py", line 612, in format_coord
     ys = self.format_ydata(y)
   File "/usr/lib/python2.4/site-packages/matplotlib/axes.py", line 605, in format_ydata
     val = func(y)
   File "/usr/lib/python2.4/site-packages/matplotlib/ticker.py", line 152, in format_data
     return self.__call__(value)
   File "/usr/lib/python2.4/site-packages/matplotlib/ticker.py", line 178, in __call__
     else: return self.seq[pos]
TypeError: list indices must be integers

pos=None would certainly cause this...but I haven't checked it.

(System is stock Mandriva 2006; mpl is CVS.)

Eric

Jeff_Whitaker1 · November 27, 2005, 11:17pm

Eric Firing wrote:

John,

I have committed to CVS a set of changes that I think will address recent requests by Jordan and Gerald, and will be of more general use as well. From the CHANGELOG:

2005-11-27 Multiple changes in cm.py, colors.py, figure.py, image.py,
           contour.py, contour_demo.py; new _cm.py, examples/image_masked.py.
           1) Separated the color table data from cm.py out into
           a new file, _cm.py, to make it easier to find the actual
           code in cm.py and to add new colormaps. Also added
           some line breaks to the color data dictionaries. Everything
           from _cm.py is imported by cm.py, so the split should be
           transparent.
           2) Enabled automatic generation of a colormap from
           a list of colors in contour; see modified
           examples/contour_demo.py.
           3) Support for imshow of a masked array, with the
           ability to specify colors (or no color at all) for
           masked regions, and for regions that are above or
           below the normally mapped region. See
           examples/image_masked.py.
           4) In support of the above, added two new classes,
           ListedColormap, and no_norm, to colors.py, and modified
           the Colormap class to include common functionality. Added
           a clip kwarg to the normalize class. Reworked color
           handling in contour.py, especially in the ContourLabeller
           mixin.

With one very subtle exception, I don't think any default behaviors have been changed, so the changes should be entirely non-disruptive.

That one exception is that in the original color mapping scheme, the last color was essentially never used. Consider an extreme case: a colormap with two entries. Suppose the image data ranged from 0 to 1. All values except exactly 1.0 were mapped to the first color, and only that upper limit value was mapped to the second color. As I have changed colors.py, values from 0.0 up to 0.5 will get the first color, and values from 0.5 through 1.0 will get the second. I think this is what a user would expect, and the only reason it hasn't mattered is that with 256 colors in a continuous range, one can't see the difference.

The request from Jordan that is addressed here is the desire for precise color control in filled contouring. Now one can specify a list of colors in the "colors" kwarg, and they will be used to generate a colormap, which will then be used by colorbar. This is shown in a second plot added to examples/contourf_demo.py. (Jordan suggested a more extensive refactoring, which may indeed be a good idea; but I wanted to make these simpler changes before trying to think about anything more drastic.)

The request from Gerald was for easy handling of masked arrays in imshow. It was in the context of basemap, which I have not tested yet; but at least for pylab.imshow, the situation is now easy to handle. If changes to basemap are needed, they should be very simple, and I can do them later as needed. I think that the changes to colormapping that I made to support this will be useful much more widely, but I have made no attempt to track down the places where changes will be in order. I may make additional changes to contour, and I know I will need to change colorbar to fully support this. I think colorbar needs some more refactoring anyway, but I can't do it immediately, and I did not want to delay getting the other changes out for testing and, hopefully, productive use.

Eric

Eric: This is fantastic - thanks! I'll try imshow with masked arrays and let you know if there are any problems. Should be very useful for projections which have some parts hidden, such as orthographic.

-Jeff

···

--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@...236...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : Jeffrey S. Whitaker: NOAA Physical Sciences Laboratory

_Gerald_John_M_Manip · November 29, 2005, 2:46pm

Eric,

I played with the changes you've made and the results look great!

Thanks,

Gerald

Eric Firing wrote:

···

John,

I have committed to CVS a set of changes that I think will address recent requests by Jordan and Gerald, and will be of more general use as well. From the CHANGELOG:

2005-11-27 Multiple changes in cm.py, colors.py, figure.py, image.py,
           contour.py, contour_demo.py; new _cm.py, examples/image_masked.py.
           1) Separated the color table data from cm.py out into
           a new file, _cm.py, to make it easier to find the actual
           code in cm.py and to add new colormaps. Also added
           some line breaks to the color data dictionaries. Everything
           from _cm.py is imported by cm.py, so the split should be
           transparent.
           2) Enabled automatic generation of a colormap from
           a list of colors in contour; see modified
           examples/contour_demo.py.
           3) Support for imshow of a masked array, with the
           ability to specify colors (or no color at all) for
           masked regions, and for regions that are above or
           below the normally mapped region. See
           examples/image_masked.py.
           4) In support of the above, added two new classes,
           ListedColormap, and no_norm, to colors.py, and modified
           the Colormap class to include common functionality. Added
           a clip kwarg to the normalize class. Reworked color
           handling in contour.py, especially in the ContourLabeller
           mixin.

With one very subtle exception, I don't think any default behaviors have been changed, so the changes should be entirely non-disruptive.

That one exception is that in the original color mapping scheme, the last color was essentially never used. Consider an extreme case: a colormap with two entries. Suppose the image data ranged from 0 to 1. All values except exactly 1.0 were mapped to the first color, and only that upper limit value was mapped to the second color. As I have changed colors.py, values from 0.0 up to 0.5 will get the first color, and values from 0.5 through 1.0 will get the second. I think this is what a user would expect, and the only reason it hasn't mattered is that with 256 colors in a continuous range, one can't see the difference.

The request from Jordan that is addressed here is the desire for precise color control in filled contouring. Now one can specify a list of colors in the "colors" kwarg, and they will be used to generate a colormap, which will then be used by colorbar. This is shown in a second plot added to examples/contourf_demo.py. (Jordan suggested a more extensive refactoring, which may indeed be a good idea; but I wanted to make these simpler changes before trying to think about anything more drastic.)

The request from Gerald was for easy handling of masked arrays in imshow. It was in the context of basemap, which I have not tested yet; but at least for pylab.imshow, the situation is now easy to handle. If changes to basemap are needed, they should be very simple, and I can do them later as needed. I think that the changes to colormapping that I made to support this will be useful much more widely, but I have made no attempt to track down the places where changes will be in order. I may make additional changes to contour, and I know I will need to change colorbar to fully support this. I think colorbar needs some more refactoring anyway, but I can't do it immediately, and I did not want to delay getting the other changes out for testing and, hopefully, productive use.

Eric

Jeff_Whitaker1 · November 30, 2005, 1:43pm

Eric Firing wrote:

John,

I have committed to CVS a set of changes that I think will address recent requests by Jordan and Gerald, and will be of more general use as well. From the CHANGELOG:

2005-11-27 Multiple changes in cm.py, colors.py, figure.py, image.py,
           contour.py, contour_demo.py; new _cm.py, examples/image_masked.py.
           1) Separated the color table data from cm.py out into
           a new file, _cm.py, to make it easier to find the actual
           code in cm.py and to add new colormaps. Also added
           some line breaks to the color data dictionaries. Everything
           from _cm.py is imported by cm.py, so the split should be
           transparent.
           2) Enabled automatic generation of a colormap from
           a list of colors in contour; see modified
           examples/contour_demo.py.
           3) Support for imshow of a masked array, with the
           ability to specify colors (or no color at all) for
           masked regions, and for regions that are above or
           below the normally mapped region. See
           examples/image_masked.py.
           4) In support of the above, added two new classes,
           ListedColormap, and no_norm, to colors.py, and modified
           the Colormap class to include common functionality. Added
           a clip kwarg to the normalize class. Reworked color
           handling in contour.py, especially in the ContourLabeller
           mixin.

With one very subtle exception, I don't think any default behaviors have been changed, so the changes should be entirely non-disruptive.

That one exception is that in the original color mapping scheme, the last color was essentially never used. Consider an extreme case: a colormap with two entries. Suppose the image data ranged from 0 to 1. All values except exactly 1.0 were mapped to the first color, and only that upper limit value was mapped to the second color. As I have changed colors.py, values from 0.0 up to 0.5 will get the first color, and values from 0.5 through 1.0 will get the second. I think this is what a user would expect, and the only reason it hasn't mattered is that with 256 colors in a continuous range, one can't see the difference.

The request from Jordan that is addressed here is the desire for precise color control in filled contouring. Now one can specify a list of colors in the "colors" kwarg, and they will be used to generate a colormap, which will then be used by colorbar. This is shown in a second plot added to examples/contourf_demo.py. (Jordan suggested a more extensive refactoring, which may indeed be a good idea; but I wanted to make these simpler changes before trying to think about anything more drastic.)

The request from Gerald was for easy handling of masked arrays in imshow. It was in the context of basemap, which I have not tested yet; but at least for pylab.imshow, the situation is now easy to handle. If changes to basemap are needed, they should be very simple, and I can do them later as needed. I think that the changes to colormapping that I made to support this will be useful much more widely, but I have made no attempt to track down the places where changes will be in order. I may make additional changes to contour, and I know I will need to change colorbar to fully support this. I think colorbar needs some more refactoring anyway, but I can't do it immediately, and I did not want to delay getting the other changes out for testing and, hopefully, productive use.

Eric

Eric: Tried the masked imshow with basemap and it seems to work fine (without any modifications to basemap). I've added a new example, plotmap_masked.py, that shows how to use mask out the oceans on a map using imshow and pcolor. Thanks again!

-Jeff

···

--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@...236...
325 Broadway Office : Skaggs Research Cntr 1D-124
Boulder, CO, USA 80303-3328 Web : Jeffrey S. Whitaker: NOAA Physical Sciences Laboratory