Matplotlib's new default colormap

Please use this thread to discuss the best choice for a new default matplotlib colormap.

This follows on from a discussion on the matplotlib-devel mailing list entitled “How to move beyond JET as the default matplotlib colormap”.

It is accepted that there can never be a best colormap for all data, so some documentation on choosing an appropriate colormap for specific data should always be sought. Nonetheless, matplotlib does need a default, and it has been shown just how damaging the Jet (matplotlib’s current default) colormap really is, so we need to come up with a genuine alternative.

To keep this thread as useful as possible please avoid posting “+1” type messages. If you’d like to suggest a colormap for consideration as matplotlib’s new default please try to keep to reference-able/demonstrable fact.

Thanks,

Phil

I remember reading a (peer-reviewed, I think) article about how "jet" was a
very unfortunate choice of default. I can't find the exact article now, but
I did find some other useful ones:

http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html
http://www.sandia.gov/~kmorel/documents/ColorMaps/
http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf

Darren

···

On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pelson.pub@...149...> wrote:

Please use this thread to discuss the best choice for a new *default*
matplotlib colormap.

This follows on from a discussion on the matplotlib-devel mailing list
entitled "How to move beyond JET as the default matplotlib colormap".

Those are good articles. There's a lot of literature on the problems
with "jet", and lots of links in the matplotlib issue [1]. For those
trying to get up to speed quickly, MathWorks recently put together a
nice review of the literature [2]. One particularly striking paper
they cite studied a group of medical students and found that (a) they
were used to/practiced at using jet, (b) when given a choice of
colormaps they said that they preferred jet, (c) they nonetheless made
more *medical diagnostic errors* when using jet than with better
designed colormaps (Borkin et al, 2011).

I won't suggest a specific colormap, but I do propose that whatever we
chose satisfy the following criteria:

- it should be a sequential colormap, because diverging colormaps are
really misleading unless you know where the "center" of the data is,
and for a default colormap we generally won't.

- it should be perceptually uniform, i.e., human subjective judgements
of how far apart nearby colors are should correspond as linearly as
possible to the difference between the numerical values they
represent, at least locally. There's lots of research on how to
measure perceptual distance -- a colleague and I happen to have
recently implemented a state-of-the-art model of this for another
project, in case anyone wants to play with it [3], or just using
good-old-L*a*b* is a reasonable quick-and-dirty approximation.

- it should have a perceptually uniform luminance ramp, i.e. if you
convert to greyscale it should still be uniform. This is useful both
in practical terms (greyscale printers are still a thing!) and because
luminance is a very strong and natural cue to magnitude.

- it should also have some kind of variation in hue, because hue
variation is a really helpful additional cue to perception, having two
cues is better than one, and there's no reason not to do it.

- the hue variation should be chosen to produce reasonable results
even for viewers with the more common types of colorblindness. (Which
rules out things like red-to-green.)

And, for bonus points, it would be nice to choose a hue ramp that
still works if you throw away the luminance variation, because then we
could use the version with varying luminance for 2d plots, and the
version with just hue variation for 3d plots. (In 3d plots you really
want to reserve the luminance channel for lighting/shading, because
your brain is *really* good at extracting 3d shape from luminance
variation. If the 3d surface itself has massively varying luminance
then this screws up the ability to see shape.)

Do these seem like good requirements?

-n

[1] Replace "jet" as the default colormap · Issue #875 · matplotlib/matplotlib · GitHub
[2] Rainbow Color Map Critiques: An Overview and Annotated Bibliography - MATLAB & Simulink
[3] GitHub - njsmith/pycam02ucs: (Don't use this - use njsmith/colorspacious or matplotlib/viscm or both) ; install (or just run out
of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute
the perceptual distance between two RGB colors. It's also possible to
use the underlying color model stuff to do things like generate colors
with evenly spaced luminance and hues, or draw 3d plots of the shape
of the human color space. It's pretty fun to play with :slight_smile:

···

On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsdale24@...149...> wrote:

On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pelson.pub@...149...> wrote:

Please use this thread to discuss the best choice for a new default
matplotlib colormap.

This follows on from a discussion on the matplotlib-devel mailing list
entitled "How to move beyond JET as the default matplotlib colormap".

I remember reading a (peer-reviewed, I think) article about how "jet" was a
very unfortunate choice of default. I can't find the exact article now, but
I did find some other useful ones:

http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html
http://www.sandia.gov/~kmorel/documents/ColorMaps/
http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf

--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh

Please use this thread to discuss the best choice for a new default
matplotlib colormap.

This follows on from a discussion on the matplotlib-devel mailing list
entitled "How to move beyond JET as the default matplotlib colormap".

I remember reading a (peer-reviewed, I think) article about how "jet" was a
very unfortunate choice of default. I can't find the exact article now, but
I did find some other useful ones:

http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html
http://www.sandia.gov/~kmorel/documents/ColorMaps/
http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf

Those are good articles. There's a lot of literature on the problems
with "jet", and lots of links in the matplotlib issue [1]. For those
trying to get up to speed quickly, MathWorks recently put together a
nice review of the literature [2]. One particularly striking paper
they cite studied a group of medical students and found that (a) they
were used to/practiced at using jet, (b) when given a choice of
colormaps they said that they preferred jet, (c) they nonetheless made
more *medical diagnostic errors* when using jet than with better
designed colormaps (Borkin et al, 2011).

I won't suggest a specific colormap, but I do propose that whatever we
chose satisfy the following criteria:

- it should be a sequential colormap, because diverging colormaps are
really misleading unless you know where the "center" of the data is,
and for a default colormap we generally won't.

- it should be perceptually uniform, i.e., human subjective judgements
of how far apart nearby colors are should correspond as linearly as
possible to the difference between the numerical values they
represent, at least locally. There's lots of research on how to
measure perceptual distance -- a colleague and I happen to have
recently implemented a state-of-the-art model of this for another
project, in case anyone wants to play with it [3], or just using
good-old-L*a*b* is a reasonable quick-and-dirty approximation.

- it should have a perceptually uniform luminance ramp, i.e. if you
convert to greyscale it should still be uniform. This is useful both
in practical terms (greyscale printers are still a thing!) and because
luminance is a very strong and natural cue to magnitude.

- it should also have some kind of variation in hue, because hue
variation is a really helpful additional cue to perception, having two
cues is better than one, and there's no reason not to do it.

- the hue variation should be chosen to produce reasonable results
even for viewers with the more common types of colorblindness. (Which
rules out things like red-to-green.)

And, for bonus points, it would be nice to choose a hue ramp that
still works if you throw away the luminance variation, because then we
could use the version with varying luminance for 2d plots, and the
version with just hue variation for 3d plots. (In 3d plots you really
want to reserve the luminance channel for lighting/shading, because
your brain is *really* good at extracting 3d shape from luminance
variation. If the 3d surface itself has massively varying luminance
then this screws up the ability to see shape.)

Do these seem like good requirements?

Goals, yes, though I wouldn't put much weight on the "bonus" criterion. I would add that it should be aesthetically pleasing, or at least comfortable, to most people. Perfection might not be attainable, and some tradeoffs may be required. Is anyone set up to produce test images and/or metrics for judging existing colormaps, or newly designed ones, on all of these criteria?

Eric

···

On 2014/11/21, 4:42 PM, Nathaniel Smith wrote:

On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsdale24@...149...> wrote:

On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pelson.pub@...149...> wrote:

-n

[1] Replace "jet" as the default colormap · Issue #875 · matplotlib/matplotlib · GitHub
[2] Rainbow Color Map Critiques: An Overview and Annotated Bibliography - MATLAB & Simulink
[3] GitHub - njsmith/pycam02ucs: (Don't use this - use njsmith/colorspacious or matplotlib/viscm or both) ; install (or just run out
of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute
the perceptual distance between two RGB colors. It's also possible to
use the underlying color model stuff to do things like generate colors
with evenly spaced luminance and hues, or draw 3d plots of the shape
of the human color space. It's pretty fun to play with :slight_smile:

There was a talk by Kristen Thyng at scipy2014 that might be a good backgrounder for this:
http://pyvideo.org/video/2769/perceptions-of-matplotlib-colormaps

At the end she references this site http://mycarta.wordpress.com/ of Matteo Niccoli which is full of good content. I wonder if it’s worth contacting Kristen or Matteo to let them know there’s a discussion going on here that they might be interested in?

···

On 22 November 2014 at 09:53, Eric Firing <efiring@…229…> wrote:

On 2014/11/21, 4:42 PM, Nathaniel Smith wrote:

On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsdale24@…149…> wrote:

On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pelson.pub@…149…> wrote:

Please use this thread to discuss the best choice for a new default

matplotlib colormap.

This follows on from a discussion on the matplotlib-devel mailing list

entitled “How to move beyond JET as the default matplotlib colormap”.

I remember reading a (peer-reviewed, I think) article about how “jet” was a

very unfortunate choice of default. I can’t find the exact article now, but

I did find some other useful ones:

http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html

http://www.sandia.gov/~kmorel/documents/ColorMaps/

http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf

Those are good articles. There’s a lot of literature on the problems

with “jet”, and lots of links in the matplotlib issue [1]. For those

trying to get up to speed quickly, MathWorks recently put together a

nice review of the literature [2]. One particularly striking paper

they cite studied a group of medical students and found that (a) they

were used to/practiced at using jet, (b) when given a choice of

colormaps they said that they preferred jet, (c) they nonetheless made

more medical diagnostic errors when using jet than with better

designed colormaps (Borkin et al, 2011).

I won’t suggest a specific colormap, but I do propose that whatever we

chose satisfy the following criteria:

  • it should be a sequential colormap, because diverging colormaps are

really misleading unless you know where the “center” of the data is,

and for a default colormap we generally won’t.

  • it should be perceptually uniform, i.e., human subjective judgements

of how far apart nearby colors are should correspond as linearly as

possible to the difference between the numerical values they

represent, at least locally. There’s lots of research on how to

measure perceptual distance – a colleague and I happen to have

recently implemented a state-of-the-art model of this for another

project, in case anyone wants to play with it [3], or just using

good-old-Lab* is a reasonable quick-and-dirty approximation.

  • it should have a perceptually uniform luminance ramp, i.e. if you

convert to greyscale it should still be uniform. This is useful both

in practical terms (greyscale printers are still a thing!) and because

luminance is a very strong and natural cue to magnitude.

  • it should also have some kind of variation in hue, because hue

variation is a really helpful additional cue to perception, having two

cues is better than one, and there’s no reason not to do it.

  • the hue variation should be chosen to produce reasonable results

even for viewers with the more common types of colorblindness. (Which

rules out things like red-to-green.)

And, for bonus points, it would be nice to choose a hue ramp that

still works if you throw away the luminance variation, because then we

could use the version with varying luminance for 2d plots, and the

version with just hue variation for 3d plots. (In 3d plots you really

want to reserve the luminance channel for lighting/shading, because

your brain is really good at extracting 3d shape from luminance

variation. If the 3d surface itself has massively varying luminance

then this screws up the ability to see shape.)

Do these seem like good requirements?

Goals, yes, though I wouldn’t put much weight on the “bonus” criterion.

I would add that it should be aesthetically pleasing, or at least

comfortable, to most people. Perfection might not be attainable, and

some tradeoffs may be required. Is anyone set up to produce test images

and/or metrics for judging existing colormaps, or newly designed ones,

on all of these criteria?

Eric

-n

[1] https://github.com/matplotlib/matplotlib/issues/875

[2] http://uk.mathworks.com/company/newsletters/articles/rainbow-color-map-critiques-an-overview-and-annotated-bibliography.html

[3] https://github.com/njsmith/pycam02ucs ; install (or just run out

of the source tree) and then use pycam02ucs.deltaEp_sRGB to compute

the perceptual distance between two RGB colors. It’s also possible to

use the underlying color model stuff to do things like generate colors

with evenly spaced luminance and hues, or draw 3d plots of the shape

of the human color space. It’s pretty fun to play with :slight_smile:


Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server

from Actuate! Instantly Supercharge Your Business Reports and Dashboards

with Interactivity, Sharing, Native Excel Exports, App Integration & more

Get technology previously reserved for billion-dollar corporations, FREE

http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk


Matplotlib-devel mailing list

Matplotlib-devel@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

I had some time on a plane today, so I wrote a little script for
visualizing colormaps (esp. WRT perceptual uniformity and
colorblindness). To try it:

git clone https://github.com/njsmith/pycam02ucs.git cd pycam02ucs
$ ipython
In [1]: %matplotlib
In [2]: from pycam02ucs.viscm import viscm
In [3]: viscm("jet")

(Or substitute your favorite built-in colormap, or pass a matplotlib
colormap object, i.e. a callable that takes an array of values in the
range [0, 1] and returns an array of RGBA values with shape (n, 4).)

I'm attaching an example, plus an annotated example explaining what
the different bits show.

It's a bit crude, but has definitely reached the
fun-to-play-around-with stage :-). If anyone makes improvements send
me a PR!

Hidden feature: you can pass show_gamut=True to get a crude
approximation of the space of possible sRGB colors drawn onto the 3d
plot at the bottom. The idea is if trying to design a better colormap
it's useful to have a sense of what potential colors are available to
use. It's pretty crude and somewhat distracting though so I left it
off by default for now.

-n

YlGnBu_r.pdf (182 KB)

···

On Fri, Nov 21, 2014 at 10:53 PM, Eric Firing <efiring@...229...> wrote:

On 2014/11/21, 4:42 PM, Nathaniel Smith wrote:

On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsdale24@...149...> wrote:

On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pelson.pub@...149...> wrote:

Please use this thread to discuss the best choice for a new default
matplotlib colormap.

This follows on from a discussion on the matplotlib-devel mailing list
entitled "How to move beyond JET as the default matplotlib colormap".

I remember reading a (peer-reviewed, I think) article about how "jet" was a
very unfortunate choice of default. I can't find the exact article now, but
I did find some other useful ones:

http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html
http://www.sandia.gov/~kmorel/documents/ColorMaps/
http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf

Those are good articles. There's a lot of literature on the problems
with "jet", and lots of links in the matplotlib issue [1]. For those
trying to get up to speed quickly, MathWorks recently put together a
nice review of the literature [2]. One particularly striking paper
they cite studied a group of medical students and found that (a) they
were used to/practiced at using jet, (b) when given a choice of
colormaps they said that they preferred jet, (c) they nonetheless made
more *medical diagnostic errors* when using jet than with better
designed colormaps (Borkin et al, 2011).

I won't suggest a specific colormap, but I do propose that whatever we
chose satisfy the following criteria:

- it should be a sequential colormap, because diverging colormaps are
really misleading unless you know where the "center" of the data is,
and for a default colormap we generally won't.

- it should be perceptually uniform, i.e., human subjective judgements
of how far apart nearby colors are should correspond as linearly as
possible to the difference between the numerical values they
represent, at least locally. There's lots of research on how to
measure perceptual distance -- a colleague and I happen to have
recently implemented a state-of-the-art model of this for another
project, in case anyone wants to play with it [3], or just using
good-old-L*a*b* is a reasonable quick-and-dirty approximation.

- it should have a perceptually uniform luminance ramp, i.e. if you
convert to greyscale it should still be uniform. This is useful both
in practical terms (greyscale printers are still a thing!) and because
luminance is a very strong and natural cue to magnitude.

- it should also have some kind of variation in hue, because hue
variation is a really helpful additional cue to perception, having two
cues is better than one, and there's no reason not to do it.

- the hue variation should be chosen to produce reasonable results
even for viewers with the more common types of colorblindness. (Which
rules out things like red-to-green.)

And, for bonus points, it would be nice to choose a hue ramp that
still works if you throw away the luminance variation, because then we
could use the version with varying luminance for 2d plots, and the
version with just hue variation for 3d plots. (In 3d plots you really
want to reserve the luminance channel for lighting/shading, because
your brain is *really* good at extracting 3d shape from luminance
variation. If the 3d surface itself has massively varying luminance
then this screws up the ability to see shape.)

Do these seem like good requirements?

Goals, yes, though I wouldn't put much weight on the "bonus" criterion.
  I would add that it should be aesthetically pleasing, or at least
comfortable, to most people. Perfection might not be attainable, and
some tradeoffs may be required. Is anyone set up to produce test images
and/or metrics for judging existing colormaps, or newly designed ones,
on all of these criteria?

--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh

Neat stuff! Just a quick note about the 3D plot. By default, the scatter markers are shaded to give an illusion of depth. This is often what we want, but I think in this case, it might make sense to not do that. Add depthshade=False to the scatter call to turn it off. I think that was added for mpl version 1.3.

Ben Root

···

On Tue, Dec 23, 2014 at 4:16 AM, Nathaniel Smith <njs@…503…> wrote:

On Fri, Nov 21, 2014 at 10:53 PM, Eric Firing <efiring@…930…> wrote:

On 2014/11/21, 4:42 PM, Nathaniel Smith wrote:

On Fri, Nov 21, 2014 at 5:46 PM, Darren Dale <dsdale24@…149…> wrote:

On Fri, Nov 21, 2014 at 12:32 PM, Phil Elson <pelson.pub@…149…> wrote:

Please use this thread to discuss the best choice for a new default

matplotlib colormap.

This follows on from a discussion on the matplotlib-devel mailing list

entitled “How to move beyond JET as the default matplotlib colormap”.

I remember reading a (peer-reviewed, I think) article about how “jet” was a

very unfortunate choice of default. I can’t find the exact article now, but

I did find some other useful ones:

http://cresspahl.blogspot.com/2012/03/expanded-control-of-octaves-colormap.html

http://www.sandia.gov/~kmorel/documents/ColorMaps/

http://www.sandia.gov/~kmorel/documents/ColorMaps/ColorMapsExpanded.pdf

Those are good articles. There’s a lot of literature on the problems

with “jet”, and lots of links in the matplotlib issue [1]. For those

trying to get up to speed quickly, MathWorks recently put together a

nice review of the literature [2]. One particularly striking paper

they cite studied a group of medical students and found that (a) they

were used to/practiced at using jet, (b) when given a choice of

colormaps they said that they preferred jet, (c) they nonetheless made

more medical diagnostic errors when using jet than with better

designed colormaps (Borkin et al, 2011).

I won’t suggest a specific colormap, but I do propose that whatever we

chose satisfy the following criteria:

  • it should be a sequential colormap, because diverging colormaps are

really misleading unless you know where the “center” of the data is,

and for a default colormap we generally won’t.

  • it should be perceptually uniform, i.e., human subjective judgements

of how far apart nearby colors are should correspond as linearly as

possible to the difference between the numerical values they

represent, at least locally. There’s lots of research on how to

measure perceptual distance – a colleague and I happen to have

recently implemented a state-of-the-art model of this for another

project, in case anyone wants to play with it [3], or just using

good-old-Lab* is a reasonable quick-and-dirty approximation.

  • it should have a perceptually uniform luminance ramp, i.e. if you

convert to greyscale it should still be uniform. This is useful both

in practical terms (greyscale printers are still a thing!) and because

luminance is a very strong and natural cue to magnitude.

  • it should also have some kind of variation in hue, because hue

variation is a really helpful additional cue to perception, having two

cues is better than one, and there’s no reason not to do it.

  • the hue variation should be chosen to produce reasonable results

even for viewers with the more common types of colorblindness. (Which

rules out things like red-to-green.)

And, for bonus points, it would be nice to choose a hue ramp that

still works if you throw away the luminance variation, because then we

could use the version with varying luminance for 2d plots, and the

version with just hue variation for 3d plots. (In 3d plots you really

want to reserve the luminance channel for lighting/shading, because

your brain is really good at extracting 3d shape from luminance

variation. If the 3d surface itself has massively varying luminance

then this screws up the ability to see shape.)

Do these seem like good requirements?

Goals, yes, though I wouldn’t put much weight on the “bonus” criterion.

I would add that it should be aesthetically pleasing, or at least

comfortable, to most people. Perfection might not be attainable, and

some tradeoffs may be required. Is anyone set up to produce test images

and/or metrics for judging existing colormaps, or newly designed ones,

on all of these criteria?

I had some time on a plane today, so I wrote a little script for

visualizing colormaps (esp. WRT perceptual uniformity and

colorblindness). To try it:

$ git clone https://github.com/njsmith/pycam02ucs.git

$ cd pycam02ucs

$ ipython

In [1]: %matplotlib

In [2]: from pycam02ucs.viscm import viscm

In [3]: viscm(“jet”)

(Or substitute your favorite built-in colormap, or pass a matplotlib

colormap object, i.e. a callable that takes an array of values in the

range [0, 1] and returns an array of RGBA values with shape (n, 4).)

I’m attaching an example, plus an annotated example explaining what

the different bits show.

It’s a bit crude, but has definitely reached the

fun-to-play-around-with stage :-). If anyone makes improvements send

me a PR!

Hidden feature: you can pass show_gamut=True to get a crude

approximation of the space of possible sRGB colors drawn onto the 3d

plot at the bottom. The idea is if trying to design a better colormap

it’s useful to have a sense of what potential colors are available to

use. It’s pretty crude and somewhat distracting though so I left it

off by default for now.

-n

Nathaniel J. Smith

Postdoctoral researcher - Informatics - University of Edinburgh

http://vorpus.org


Dive into the World of Parallel Programming! The Go Parallel Website,

sponsored by Intel and developed in partnership with Slashdot Media, is your

hub for all things parallel software development, from weekly thought

leadership blogs to news, videos, case studies, tutorials and more. Take a

look and join the conversation now. http://goparallel.sourceforge.net


Matplotlib-devel mailing list

Matplotlib-devel@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Happy new year everyone!

Apologies for the long silence. I was snowed in with work before Christmas and then mostly cut off from the internet for the past two weeks. Fortunately, I had a chance over the holidays to flesh out the GUI which I mentioned in my previous email. You can find it here:

https://github.com/maxalbert/colormap-selector

Basically, it allows you to pick the start/end color of a colormap from two cross sections in CIELab space and interpolates those colors linearly (see the README file for more details). Currently there is one scatterplot to illustrate the resulting colormap but it can be trivially extended to show more interesting sample plots. There are still a few things missing that I’d like to add but at least it’s in a state where it can be used and I’d be grateful for feedback, especially with regard to the colormaps generated with it (I do have some opinions myself but it would be interesting to hear others’ first).

Regarding our ongoing discussion, I had a very useful chat with two colleagues before Christmas which spurred more thoughts. But I guess it’s best to discuss them in a separate email when I’m less tired. :wink:

Best wishes,

Max

Nice job.

I find your GUI a little bit confusing (new to colormap stuff) but I
like the idea, basically I find it overkill, I would replace the gui
by a plot and a couple of slider widgets something simpler to
integrate without new dependencies.
Do you really need the third 3d plot on the right?

···

On Mon, Jan 5, 2015 at 9:37 PM, Maximilian Albert <maximilian.albert@...149...> wrote:

Happy new year everyone!

Apologies for the long silence. I was snowed in with work before Christmas
and then mostly cut off from the internet for the past two weeks.
Fortunately, I had a chance over the holidays to flesh out the GUI which I
mentioned in my previous email. You can find it here:

   GitHub - maxalbert/colormap-selector: GUI to facilitate defining perceptually linear colormaps in CIELab space

Basically, it allows you to pick the start/end color of a colormap from two
cross sections in CIELab space and interpolates those colors linearly (see
the README file for more details). Currently there is one scatterplot to
illustrate the resulting colormap but it can be trivially extended to show
more interesting sample plots. There are still a few things missing that I'd
like to add but at least it's in a state where it can be used and I'd be
grateful for feedback, especially with regard to the colormaps generated
with it (I do have some opinions myself but it would be interesting to hear
others' first).

Regarding our ongoing discussion, I had a very useful chat with two
colleagues before Christmas which spurred more thoughts. But I guess it's
best to discuss them in a separate email when I'm less tired. :wink:

Best wishes,
Max

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
matplotlib-devel List Signup and Options

--
Y yo que culpa tengo de que ellas se crean todo lo que yo les digo?

-- Antonio Alducin --

There's a downside to this approach for the kinds of colormaps we've
been talking about in this thread, where we want both a large
lightness range plus a colorful result. The problem is that the way
color space is shaped, you can't simultaneously have both high
saturation (colorfulness) *and* high/low lightness. So if you pick
your extreme points to be near black and white, then they can only
have a slight tinting of color, and then if you linearly interpolate
between these, then you end up with slightly tinted greyscale.

Colormaps like YlGnBu or cubehelix or parula are designed to start out
with low saturation, then as they move into the middle of the
lightness scale they arc outwards, then arc back in again.

This is a lot easier to visualize (e.g. by playing with the script I
posted upthread) than it is to explain in text :-). Like, if you do
viscm(YlGnBu_r) and look at the plot in the lower-right then it's
clear that it's not a simple straight line in (J'/K, a', b') space
(which is a higher-tech analogue to L* a* b* space).

···

On Tue, Jan 6, 2015 at 2:37 AM, Maximilian Albert <maximilian.albert@...149...> wrote:

Happy new year everyone!

Apologies for the long silence. I was snowed in with work before Christmas
and then mostly cut off from the internet for the past two weeks.
Fortunately, I had a chance over the holidays to flesh out the GUI which I
mentioned in my previous email. You can find it here:

   GitHub - maxalbert/colormap-selector: GUI to facilitate defining perceptually linear colormaps in CIELab space

Basically, it allows you to pick the start/end color of a colormap from two
cross sections in CIELab space and interpolates those colors linearly (see
the README file for more details).

--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh

Hi Federico,

Thanks for trying it out and for the feedback!

Indeed, I started out writing a simple IPython notebook along the lines you suggested, with just a couple of sliders and plots, but it quickly became too slow and unwieldy for quick explorations, hence the slightly more elaborate GUI.

I agree that the reason for the 3D plot on the right may not be obvious at the moment. Personally, I find it useful to get a feel for what the representable colors in CIELab space (and the cross sections for L=const) look like, but when simply using a linear interpolation between two colors (as I’m doing at the moment) it may not be needed to visualise it in 3D.

The reason I added it is that while playing around with the GUI I got the impression that my initial suggestion of using a simple linear interpolation between two colors may not result in the best-looking colormaps (this is confirmed by Nathaniel’s reply). I’m currently toying with the option to use curved interpolations, and for thee it would be very useful IMHO to see what they look like in 3D.

Btw, I have refactored my code a bit and it should be easy to write a simpler UI (e.g. in an IPython notebook) which doesn’t need the other dependencies (also, I could drop the wxpython dependency because some conflict with Vispy which I had experienced seems to have disappeared). If you like, feel free to give it a shot to write a UI the way you imagine it. It’s always good to have more options for exploration. :slight_smile:

Best wishes,

Max

···

2015-01-08 17:44 GMT+00:00 Federico Ariza <ariza.federico@…714…>:

Nice job.

I find your GUI a little bit confusing (new to colormap stuff) but I

like the idea, basically I find it overkill, I would replace the gui

by a plot and a couple of slider widgets something simpler to

integrate without new dependencies.

Do you really need the third 3d plot on the right?

On Mon, Jan 5, 2015 at 9:37 PM, Maximilian Albert

<maximilian.albert@…716…> wrote:

Happy new year everyone!

Apologies for the long silence. I was snowed in with work before Christmas

and then mostly cut off from the internet for the past two weeks.

Fortunately, I had a chance over the holidays to flesh out the GUI which I

mentioned in my previous email. You can find it here:

https://github.com/maxalbert/colormap-selector

Basically, it allows you to pick the start/end color of a colormap from two

cross sections in CIELab space and interpolates those colors linearly (see

the README file for more details). Currently there is one scatterplot to

illustrate the resulting colormap but it can be trivially extended to show

more interesting sample plots. There are still a few things missing that I’d

like to add but at least it’s in a state where it can be used and I’d be

grateful for feedback, especially with regard to the colormaps generated

with it (I do have some opinions myself but it would be interesting to hear

others’ first).

Regarding our ongoing discussion, I had a very useful chat with two

colleagues before Christmas which spurred more thoughts. But I guess it’s

best to discuss them in a separate email when I’m less tired. :wink:

Best wishes,

Max


Dive into the World of Parallel Programming! The Go Parallel Website,

sponsored by Intel and developed in partnership with Slashdot Media, is your

hub for all things parallel software development, from weekly thought

leadership blogs to news, videos, case studies, tutorials and more. Take a

look and join the conversation now. http://goparallel.sourceforge.net


Matplotlib-devel mailing list

Matplotlib-devel@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Y yo que culpa tengo de que ellas se crean todo lo que yo les digo?

– Antonio Alducin –

Hi Nathaniel,

> Basically, it allows you to pick the start/end color of a colormap from
two
> cross sections in CIELab space and interpolates those colors linearly
(see
> the README file for more details).

There's a downside to this approach for the kinds of colormaps we've
been talking about in this thread, where we want both a large
lightness range plus a colorful result. The problem is that the way
color space is shaped, you can't simultaneously have both high
saturation (colorfulness) *and* high/low lightness. So if you pick
your extreme points to be near black and white, then they can only
have a slight tinting of color, and then if you linearly interpolate
between these, then you end up with slightly tinted greyscale.

You raise an excellent point here. It explains nicely what I have
experienced while playing with my GUI. Indeed, I found that a simple linear
interpolation did not result in totally satisfactory colormaps (see my
previous reply to Federico). I couldn't quite explain why, but your
explanation makes this clear.

One exception I encountered is an interpolation between dark blue and
yellow as in the attached screenshot (which I hope makes it through to the
mailing list) - probably because it mostly avoids the low-saturation
(grey-ish) region of the color space. But I agree with you that using a
curved, rather than linear, interpolation can probably yield better results.

It sounds like you have a good deal of experience with various color spaces
and colormaps. Do you have an idea for a good "recipe" how to pick a curve
in a given colorspace that leads to a satisfactory colormap? My first idea
was to change the interpolating line to a circular arc passing through an
"intermediate" color, but it's not clear to me whether there is any
preferred "direction" for nudging the line into an arc. Also, most other
colormaps, such as the examples "YlGnBu" and "cubehelix" which you
mentioned, use more complicated curves than mere circular arcs (btw, kudos
for your script - it's a great way of visualising these colormaps). I don't
have enough knowledge yet to decide whether either approach is better. I've
started toying with curved interpolations in my code but this is not quite
ready to be pushed to Github yet. Anyway, if you have any suggestions I'd
love to hear them.

I also found a few more blog posts and papers which I hadn't seen before
and which look extremely useful:

(i) "Subtleties of color"

A series of six blog posts with an excellent introduction to color theory
and the issues around choosing colormaps. Well worth a read! It also
suggests that CIE L*c*h* space (which uses the three variables lightness,
chroma (saturation) and hue), may be a better choice than CIE L*a*b*, which
I have been using so far.

(ii) "How To Avoid Equidistant HSV Colors"
http://vis4.net/blog/posts/avoid-equidistant-hsv-colors/

Blog post with some interactive tools to visualise sections of CIE L*a*b*
space and HCL (Hue-Chroma-Lightness) space.

Here is a nice standalone version of the second tool:

(iii) "Generating Color Palettes using Intuitive Parameters"
http://magnaview.nl/documents/MagnaView-M_Wijffelaars-Generating_color_palettes_using_intuitive_parameters.pdf

Excellent-looking paper on the subject. I haven't read it in full yet but
it looks like a great resource which might answer some of my questions
above.

At this stage I'm wondering how best to proceed. There seems to be huge
number of resources and information, but we don't really have a clear path
forward. I agree with Phil Elson's assessment when I talked to him at the
Open Source Day: what we need is for someone to make a suggestion for a
colormap and list a number of reasons why this particular one should be
chosen. Then we have a basis for discussion and can argue about it. If
anybody has such a suggestion yet, it would be great to hear about it (even
if it is not perfect). Otherwise I'll try to make one once I have explored
various options a bit more (although it may take a little while as my spare
time is rather limited at the moment).

Best wishes,
Max

Hi Nathaniel,

> Basically, it allows you to pick the start/end color of a colormap from
> two
> cross sections in CIELab space and interpolates those colors linearly
> (see
> the README file for more details).

There's a downside to this approach for the kinds of colormaps we've
been talking about in this thread, where we want both a large
lightness range plus a colorful result. The problem is that the way
color space is shaped, you can't simultaneously have both high
saturation (colorfulness) *and* high/low lightness. So if you pick
your extreme points to be near black and white, then they can only
have a slight tinting of color, and then if you linearly interpolate
between these, then you end up with slightly tinted greyscale.

You raise an excellent point here. It explains nicely what I have
experienced while playing with my GUI. Indeed, I found that a simple linear
interpolation did not result in totally satisfactory colormaps (see my
previous reply to Federico). I couldn't quite explain why, but your
explanation makes this clear.

One exception I encountered is an interpolation between dark blue and yellow
as in the attached screenshot (which I hope makes it through to the mailing
list) - probably because it mostly avoids the low-saturation (grey-ish)
region of the color space.

I guess this probably also has to do with another weird feature of how
the colorspace is shaped. You'll often see pictures in books that
illustrate it like two cones:
   http://www.tvtechnology.com/BE_Files/uploads/2013/05/ColorTopCones-305be18.jpg
which does capture the general idea that your range of saturations is
widest when lightness is in the middle, and narrows down when you move
towards black or white. But it's actually a bit more complicated than
that -- the actual shape is sorta lumpy, more like the picture here:
    http://www.gamutvision.com/
In particular, you can have pretty-saturated blues even at very low
lightnesses, and pretty-saturated yellows even at high lightnesses.
E.g. there literally does not exist a dark red that's as intense as
the most intense dark blue.

So this makes dark-blue-to-light-yellow the obvious way to go if you
want a dark-to-light colormap that is also colorful.

I don't think it's a coincidence that parula does exactly this :slight_smile:

There is an obvious degree of freedom here though -- the color wheel
is, like, a wheel, so if you want to go from blue to yellow you can do
that either clockwise or counterclockwise, i.e., you can go through
green or you can go through red. Parula goes via green (and so does
matplotlib's YlGnBu, for that matter). If we want to have a
distinctive colormap that people won't confuse with Matlab(R)(TM) then
maybe we should make a blue-purple-red-yellow one.

And in fact, this is probably theoretically optimal! As another weird
quirk of how color works, the 4 focal colors (red/green/blue/yellow)
are not actually at right angles to each other on the hue circle --
see the lower diagram on this figure:

    Measuring Colour - R. W. G. Hunt, M. R. Pointer - Google Books

From yellow-to-blue via red is a ~213 degree angle, while

yellow-to-blue-via-green is only a ~147 degree angle (in a space where
we define our "hue angle" based on perceptual
just-noticeable-differences). So a blue-purple-red-yellow colormap
should theoretically have higher perceptual resolution than a
blue-green-yellow colormap.

But I agree with you that using a curved, rather
than linear, interpolation can probably yield better results.

It sounds like you have a good deal of experience with various color spaces
and colormaps. Do you have an idea for a good "recipe" how to pick a curve
in a given colorspace that leads to a satisfactory colormap?

I haven't tried it yet, but my first idea would be to say that I want
a linear ramp in lightness (CIECAM02's "J"), and a linear ramp in hue
(CIECAM02's "h"), that starts at blue and ends at yellow, and then run
an optimizer to try and find a set of colorfulness values (CIECAM02's
"M") that maximize some criteria, i.e.:
  -- need to stay within the sRGB gamut
  -- the points should be as close to equidistant as possible
(measured in CAM02-UCS space)
  -- the total arc should be as long as possible (measured in
CAM02-UCS space) (this forces it to use the large colorfulness values
when available)
  -- and maybe some sort of wiggliness penalty (integral of squared
third derivative or something?) to smooth it out a bit

Then it just becomes an optimization problem -- given any proposed set
of JMh values we can convert into sRGB to check the gamut, convert in
CAM02-UCS to check the distances, etc., and compute an objective
function.

My first idea
was to change the interpolating line to a circular arc passing through an
"intermediate" color, but it's not clear to me whether there is any
preferred "direction" for nudging the line into an arc. Also, most other
colormaps, such as the examples "YlGnBu" and "cubehelix" which you
mentioned, use more complicated curves than mere circular arcs (btw, kudos
for your script - it's a great way of visualising these colormaps). I don't
have enough knowledge yet to decide whether either approach is better. I've
started toying with curved interpolations in my code but this is not quite
ready to be pushed to Github yet. Anyway, if you have any suggestions I'd
love to hear them.

I also found a few more blog posts and papers which I hadn't seen before and
which look extremely useful:

(i) "Subtleties of color"

Elegant Figures - Subtleties of Color (Part 1 of 6)

A series of six blog posts with an excellent introduction to color theory
and the issues around choosing colormaps. Well worth a read! It also
suggests that CIE L*c*h* space (which uses the three variables lightness,
chroma (saturation) and hue), may be a better choice than CIE L*a*b*, which
I have been using so far.

They're kinda the same thing -- c*h* is just the polar coordinates
version of a*b*, so you can switch back and forth depending on which
way of thinking about things feels more natural for a given task. Of
course if you do linear interpolation in polar coordinates you get
some sort of funky curve, so I guess it would make a difference that
way :-). (And the Mh that I talk about above are also conceptually
just a polar coordinates version of a and b -- the CIECAM02
calculations literally have intermediate values called a and b that
play the same role as CIEL*a*b*'s a* and b*.)

-n

···

On Thu, Jan 8, 2015 at 10:30 PM, Maximilian Albert <maximilian.albert@...149...> wrote:

--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh