marker color handling

So: please either restore your patch with additional changes

    > to fix the problem Stefan found, or revert all of it cleanly
    > until this can be sorted out.

Norbert -- can you describe briefly the inconsistencies in the format
arg color handling you were trying to address in your patch? I see
this --
http://sourceforge.net/tracker/index.php?func=detail&aid=1504387&group_id=80706&atid=560722
-- but not much detail about the problems the changes fix.

Also, for all developers, when making API changes

  * post here with a brief descriptions of changes and rationale

  * make a note in API_CHANGES

  * run backend_driver.py. It is a good habit to not only run the
    script to make sure no exceptions are raised, but also to view
    the output PNG etc files in your favorite image viewer to make
    sure the output is correct

Thanks!
JDH

OK, I found the problem and committed a temporary fix. The real problem,
however is rooted a bit deeper.

First an explanation of the intended change:

It used to be that marker colors were partly automatic, but not
completely. I.e.

    plot(x,y,'-or')

would set both, line color and marker color to red. However

    plot(x,y,color='r')

would set only the line color and leave marker color to the default.
My change was to introduce a special value 'auto' for markeredgecolor and
markerfacecolor. This special value would cause the marker color to
always follow
the line color. (Which is, in 99% of the cases, what you want.) Most of
the special logic in axes.py could therefore go away. mfc and mec would
simply be left at 'auto' unless explicitely assigned another color. The
handling of the special value would then happen in lines.py at time of
plotting. (including the effect that for filled markers, the edge would
default to black)

The problem in r2790: I changed the default value in matplotlibrc to
'auto' and everything worked fine for me. I forgot that, of course,
anybody updating from an older version, would still have the values
'blue' and 'black' in their matplotlibrc, which would not be overridden
by the '.r' option that Stefan used.

The (temporary) solution in r2800: I deactivated the
rcfile-configurability of markeredgecolor and markerfacecolor. Assuming
that hardly anybody would want to change the 'auto' behavior in their
rcfile, this should be a good solution until we have solved the core
problem.

The core problem: The matplotlibrc file distributed with matplotlib
contains all the default values in non-commented lines. This file is
usually copied to the home-directory of any user, making it impossible
to simply change any default value in later versions. It is not possible
to find out which values in the users matplotlibrc were set on purpose
and which were just left untouched from the distributed file.

The better solution: place '#' at the beginning of every line in
matplotlibrc.template (except for 'backend' and 'numerix' which carry
important information) Any user who explicitely wants to change a value,
can simply uncomment the line and set a value. Otherwise, the default
value from matplotlib/__init__.py will remain active, even if changed in
an update. Of course, this would only make sense, if users were informed
and encouraged to replace their personal matplotlibrc

The ultimate solution: The file matplotlib.template should probably be
dropped completely and be auto-created from the information in
matplotlib/__init__.py - this would remove quite some redundancy and
potential for inconsistencies.

Comments on this?

Greetings,
Norbert

John Hunter wrote:

···

            
    > So: please either restore your patch with additional changes
    > to fix the problem Stefan found, or revert all of it cleanly
    > until this can be sorted out.

Norbert -- can you describe briefly the inconsistencies in the format
arg color handling you were trying to address in your patch? I see
this --
http://sourceforge.net/tracker/index.php?func=detail&aid=1504387&group_id=80706&atid=560722
-- but not much detail about the problems the changes fix.

Also, for all developers, when making API changes

  * post here with a brief descriptions of changes and rationale

  * make a note in API_CHANGES

  * run backend_driver.py. It is a good habit to not only run the
    script to make sure no exceptions are raised, but also to view
    the output PNG etc files in your favorite image viewer to make
    sure the output is correct

Thanks!
JDH

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Norbert Nemec wrote:

OK, I found the problem and committed a temporary fix. The real problem,
however is rooted a bit deeper.

First an explanation of the intended change:

It used to be that marker colors were partly automatic, but not
completely. I.e.

    plot(x,y,'-or')

would set both, line color and marker color to red. However

    plot(x,y,color='r')

would set only the line color and leave marker color to the default.
My change was to introduce a special value 'auto' for markeredgecolor and
markerfacecolor. This special value would cause the marker color to
always follow
the line color. (Which is, in 99% of the cases, what you want.) Most of
the special logic in axes.py could therefore go away. mfc and mec would
simply be left at 'auto' unless explicitely assigned another color. The
handling of the special value would then happen in lines.py at time of
plotting. (including the effect that for filled markers, the edge would
default to black)

In your explanation above, it is not clear what happens in each of the 4 cases: mec auto or non-auto, and mfc auto or non-auto.

In looking at your original patch, I also wondered what is the reason for supporting 3 different ways of specifying "_draw_nothing"? (I had not previously noticed that there was any such thing at all, and I guess I don't understand what it is for.)
class Line2D(Artist):
     _lineStyles = {
         '-' : '_draw_solid',
         '--' : '_draw_dashed',
         '-.' : '_draw_dash_dot',
         ':' : '_draw_dotted',
         'steps': '_draw_steps',
         'None' : '_draw_nothing',
         ' ' : '_draw_nothing',
         '' : '_draw_nothing',
     }

I was also a little uncomfortable with pushing some of the color decision logic all the way down into the draw method, together with a default value, although maybe there is no better way to get the desired behavior:

         if self._marker is not None:
             gc = renderer.new_gc()
             if (is_string_like(self._markeredgecolor) and
                 self._markeredgecolor == 'auto'):
                 if self._marker in self.filled_markers:
                     gc.set_foreground('k')
                 else:
                     gc.set_foreground(self._color)
             else:
                 gc.set_foreground(self._markeredgecolor)

The problem in r2790: I changed the default value in matplotlibrc to
'auto' and everything worked fine for me. I forgot that, of course,
anybody updating from an older version, would still have the values
'blue' and 'black' in their matplotlibrc, which would not be overridden
by the '.r' option that Stefan used.

This is not the first time matplotlibrc has bitten us, and it won't be the last...

But *shouldn't* '.r' override a setting in matplotlibrc, regardless of what that setting is? I think it should have set the mfc, or preferably both the mfc and the mec.

The (temporary) solution in r2800: I deactivated the
rcfile-configurability of markeredgecolor and markerfacecolor. Assuming
that hardly anybody would want to change the 'auto' behavior in their
rcfile, this should be a good solution until we have solved the core
problem.

The core problem: The matplotlibrc file distributed with matplotlib
contains all the default values in non-commented lines. This file is
usually copied to the home-directory of any user, making it impossible
to simply change any default value in later versions. It is not possible
to find out which values in the users matplotlibrc were set on purpose
and which were just left untouched from the distributed file.

The better solution: place '#' at the beginning of every line in
matplotlibrc.template (except for 'backend' and 'numerix' which carry
important information) Any user who explicitely wants to change a value,
can simply uncomment the line and set a value. Otherwise, the default
value from matplotlib/__init__.py will remain active, even if changed in
an update. Of course, this would only make sense, if users were informed
and encouraged to replace their personal matplotlibrc

This seems like a good idea, and one that is consistent with the way many other configurable systems are often handled. I think that regardless of what else is done, this would reduce pain during updates; it would also make it easier for the user to see what changes to the defaults he/she has made.

The ultimate solution: The file matplotlib.template should probably be
dropped completely and be auto-created from the information in
matplotlib/__init__.py - this would remove quite some redundancy and
potential for inconsistencies.

Reducing redundancy is appealing, but I don't know if it would be worth the effort of implementing your auto-generation idea--which might add clutter and complexity to __init__.py.

Eric

Eric Firing wrote:

Norbert Nemec wrote:
  

OK, I found the problem and committed a temporary fix. The real problem,
however is rooted a bit deeper.

First an explanation of the intended change:

It used to be that marker colors were partly automatic, but not
completely. I.e.

    plot(x,y,'-or')

would set both, line color and marker color to red. However

    plot(x,y,color='r')

would set only the line color and leave marker color to the default.
My change was to introduce a special value 'auto' for markeredgecolor and
markerfacecolor. This special value would cause the marker color to
always follow
the line color. (Which is, in 99% of the cases, what you want.) Most of
the special logic in axes.py could therefore go away. mfc and mec would
simply be left at 'auto' unless explicitely assigned another color. The
handling of the special value would then happen in lines.py at time of
plotting. (including the effect that for filled markers, the edge would
default to black)

In your explanation above, it is not clear what happens in each of the 4
cases: mec auto or non-auto, and mfc auto or non-auto.
  
mec and mfc are handled independently.

markeredges are drawn using:
mec, if mec!='auto'
color, if mec=='auto' and marker not in filled_markers
black, if mec=='auto' and marker in filled_marker

markerfaces are drawn using:
mfc, if mfc!='auto'
color, if mfc=='auto'

Since non-filled markers are considered 'edges' without filling, this
logic is necessary to get the correct behavior.

In looking at your original patch, I also wondered what is the reason
for supporting 3 different ways of specifying "_draw_nothing"? (I had
not previously noticed that there was any such thing at all, and I guess
I don't understand what it is for.)
class Line2D(Artist):
     _lineStyles = {
         '-' : '_draw_solid',
         '--' : '_draw_dashed',
         '-.' : '_draw_dash_dot',
         ':' : '_draw_dotted',
         'steps': '_draw_steps',
         'None' : '_draw_nothing',
         ' ' : '_draw_nothing',
         '' : '_draw_nothing',
     }
  

The idea was to be able to say something like
plot(x,y,line='',marker='o',color=(.2,.5,.8))
which seemed a bit more intuitive to me than line='None' when I created
the patch. By now, I wonder about it myself...

If there is an objection against this detail, I can revert it. I
probably should have split the patch in parts in the first place.
Unfortunately, the different parts had become more interdependent than I
would have liked.

I was also a little uncomfortable with pushing some of the color
decision logic all the way down into the draw method, together with a
default value, although maybe there is no better way to get the desired
behavior:

         if self._marker is not None:
             gc = renderer.new_gc()
             if (is_string_like(self._markeredgecolor) and
                 self._markeredgecolor == 'auto'):
                 if self._marker in self.filled_markers:
                     gc.set_foreground('k')
                 else:
                     gc.set_foreground(self._color)
             else:
                 gc.set_foreground(self._markeredgecolor)
  

This was the cleanest solution. If the decision is made earlier, one
always has to store not only the value of mec, but also, whether it was
set explicitely or automatically. Otherwise, the marker color is not
updated when set_color is called on an existing graph. (Which was the
problem that started my whole effort)

The problem in r2790: I changed the default value in matplotlibrc to
'auto' and everything worked fine for me. I forgot that, of course,
anybody updating from an older version, would still have the values
'blue' and 'black' in their matplotlibrc, which would not be overridden
by the '.r' option that Stefan used.
    
This is not the first time matplotlibrc has bitten us, and it won't be
the last...

But *shouldn't* '.r' override a setting in matplotlibrc, regardless of
what that setting is? I think it should have set the mfc, or preferably
both the mfc and the mec.
  

OK, that would be an alternative solution: set both mfc and mec to
'auto', whenever the color is specified using a format string. However,
this would mean that the rcfile options markeredgecolor and
markerfacecolor are often ignored, even if they were set on purpose. If
that is the case, one could just as well deactivate them completely and
prevent some confusion.

The (temporary) solution in r2800: I deactivated the
rcfile-configurability of markeredgecolor and markerfacecolor. Assuming
that hardly anybody would want to change the 'auto' behavior in their
rcfile, this should be a good solution until we have solved the core
problem.

The core problem: The matplotlibrc file distributed with matplotlib
contains all the default values in non-commented lines. This file is
usually copied to the home-directory of any user, making it impossible
to simply change any default value in later versions. It is not possible
to find out which values in the users matplotlibrc were set on purpose
and which were just left untouched from the distributed file.

The better solution: place '#' at the beginning of every line in
matplotlibrc.template (except for 'backend' and 'numerix' which carry
important information) Any user who explicitely wants to change a value,
can simply uncomment the line and set a value. Otherwise, the default
value from matplotlib/__init__.py will remain active, even if changed in
an update. Of course, this would only make sense, if users were informed
and encouraged to replace their personal matplotlibrc
    
This seems like a good idea, and one that is consistent with the way
many other configurable systems are often handled. I think that
regardless of what else is done, this would reduce pain during updates;
it would also make it easier for the user to see what changes to the
defaults he/she has made.
  

So, should we simply do that? The only problem that I see is, that
matplotlibrc.template will probably soon be out of sync with
defaultParams in __init__.py, once there is no necessity for developers
to update it.

The ultimate solution: The file matplotlib.template should probably be
dropped completely and be auto-created from the information in
matplotlib/__init__.py - this would remove quite some redundancy and
potential for inconsistencies.
    
Reducing redundancy is appealing, but I don't know if it would be worth
the effort of implementing your auto-generation idea--which might add
clutter and complexity to __init__.py.
  

Not necessarily. I was thinking of moving defaultParams out of
__init__.py to a separate file, which can be imported by setup.py to
write matplotlibrc. This would even reduce the complexity of
__init__.py. However, it will need some cleanup first to reduce
dependencies. I've started on that, but it will take some more time.

In any case, this auto-generation would solve the problem of a
increasingly outdated matplotlibrc.template.

Norbert,

I am splitting this thread because I think different issues are involved in different parts.

In looking at your original patch, I also wondered what is the reason for supporting 3 different ways of specifying "_draw_nothing"? (I had not previously noticed that there was any such thing at all, and I guess I don't understand what it is for.)
class Line2D(Artist):
     _lineStyles = {
         '-' : '_draw_solid',
         '--' : '_draw_dashed',
         '-.' : '_draw_dash_dot',
         ':' : '_draw_dotted',
         'steps': '_draw_steps',
         'None' : '_draw_nothing',
         ' ' : '_draw_nothing',
         '' : '_draw_nothing',
     }
  

The idea was to be able to say something like
plot(x,y,line='',marker='o',color=(.2,.5,.8))
which seemed a bit more intuitive to me than line='None' when I created
the patch. By now, I wonder about it myself...

It raises larger API questions. From the standpoint of user-level code readability, the present array of marker and line identifiers (inherited from Matlab) is not good. For example, why should '-' mean a solid line, but '_' means a horizontal line marker? What is the difference between '^' and '2', and how on earth is anyone reading the code supposed to know what '2' means? The only justification I can see for the 1- and 2-character codes is their convenience in interactive use for things like "plot(x,y,'g.'". I would not want this to go away, but for non-interactive coding I think longer names, typically corresponding to the name of the function that generates the marker (in the case of markers) should be available and used. Further, it may be that only the set of unique descriptive names should be available at the level of lines.py, and the shorthand versions should be relegated to support code for the plot command, which is at a higher level. Going even further, maybe the short format strings should be limited to a subset of available lines and markers. Is it really important to let people write "plot(x,y,'g2')" instead of "plot(x,y, color='g', marker='tri_up')"? (I still don't know what is the difference between "tri_up" and "triangle_up".)

This needs either some broader discussion, or a command decision by John along the lines of "Forget it; it's OK the way it is, or it is too late to make any change in this part of the API."

Eric

Norbert,

  

The problem in r2790: I changed the default value in matplotlibrc to
'auto' and everything worked fine for me. I forgot that, of course,
anybody updating from an older version, would still have the values
'blue' and 'black' in their matplotlibrc, which would not be overridden
by the '.r' option that Stefan used.
    

This is not the first time matplotlibrc has bitten us, and it won't be the last...

But *shouldn't* '.r' override a setting in matplotlibrc, regardless of what that setting is? I think it should have set the mfc, or preferably both the mfc and the mec.
  

OK, that would be an alternative solution: set both mfc and mec to
'auto', whenever the color is specified using a format string. However,
this would mean that the rcfile options markeredgecolor and
markerfacecolor are often ignored, even if they were set on purpose. If
that is the case, one could just as well deactivate them completely and
prevent some confusion.

Either I am not understanding you correctly, or we have fundamentally different views of the role of matplotlibrc values. The way I see it, function args and kwargs *always* override matplotlibrc values, which in turn *always* override built-in defaults. So in the example above, if the user writes "plot(x,y,'r.')", red dots should be plotted no matter what is in matplotlibrc. It should not depend on whether something is set to 'auto'.

Eric

Norbert,

The core problem: The matplotlibrc file distributed with matplotlib
contains all the default values in non-commented lines. This file is
usually copied to the home-directory of any user, making it impossible
to simply change any default value in later versions. It is not possible
to find out which values in the users matplotlibrc were set on purpose
and which were just left untouched from the distributed file.

The better solution: place '#' at the beginning of every line in
matplotlibrc.template (except for 'backend' and 'numerix' which carry
important information) Any user who explicitely wants to change a value,
can simply uncomment the line and set a value. Otherwise, the default
value from matplotlib/__init__.py will remain active, even if changed in
an update. Of course, this would only make sense, if users were informed
and encouraged to replace their personal matplotlibrc
    

This seems like a good idea, and one that is consistent with the way many other configurable systems are often handled. I think that regardless of what else is done, this would reduce pain during updates; it would also make it easier for the user to see what changes to the defaults he/she has made.
  

So, should we simply do that? The only problem that I see is, that
matplotlibrc.template will probably soon be out of sync with
defaultParams in __init__.py, once there is no necessity for developers
to update it.

The ultimate solution: The file matplotlib.template should probably be
dropped completely and be auto-created from the information in
matplotlib/__init__.py - this would remove quite some redundancy and
potential for inconsistencies.
    

Reducing redundancy is appealing, but I don't know if it would be worth the effort of implementing your auto-generation idea--which might add clutter and complexity to __init__.py.
  

Not necessarily. I was thinking of moving defaultParams out of
__init__.py to a separate file, which can be imported by setup.py to
write matplotlibrc. This would even reduce the complexity of
__init__.py. However, it will need some cleanup first to reduce
dependencies. I've started on that, but it will take some more time.

In any case, this auto-generation would solve the problem of a
increasingly outdated matplotlibrc.template.

Before going too far with all this, it would be good to hear from John and others. It all sounds good to me, but others might have objections or good alternatives that should be considered.

To summarize, your two proposals are:

1) generate matplotlibrc with almost everything commented out by default

2) eliminate matplotlibrc.template by having setup.py autogenerate matplotlibrc based on the rcParams-related code that is presently in __init__.py.

Eric

Eric Firing wrote:

Norbert,
  

The problem in r2790: I changed the default value in matplotlibrc to
'auto' and everything worked fine for me. I forgot that, of course,
anybody updating from an older version, would still have the values
'blue' and 'black' in their matplotlibrc, which would not be overridden
by the '.r' option that Stefan used.
    

This is not the first time matplotlibrc has bitten us, and it won't be
the last...

But *shouldn't* '.r' override a setting in matplotlibrc, regardless of
what that setting is? I think it should have set the mfc, or preferably
both the mfc and the mec.
  

OK, that would be an alternative solution: set both mfc and mec to
'auto', whenever the color is specified using a format string. However,
this would mean that the rcfile options markeredgecolor and
markerfacecolor are often ignored, even if they were set on purpose. If
that is the case, one could just as well deactivate them completely and
prevent some confusion.

Either I am not understanding you correctly, or we have fundamentally
different views of the role of matplotlibrc values. The way I see it,
function args and kwargs *always* override matplotlibrc values, which in
turn *always* override built-in defaults. So in the example above, if
the user writes "plot(x,y,'r.')", red dots should be plotted no matter
what is in matplotlibrc. It should not depend on whether something is
set to 'auto'.
  
I fully agree with you.

Guess, the simple solution to the dilemma is to drop the idea of
rcfile-configurability of markerfacecolor and markeredgecolor. That way,
markers would always have the same color as the line, unless explicitely
set differently by the kwargs markeredgecolor/markerfacecolor.

This is, what is wanted in 99% of the cases and for every other case, a
rcfile-option will not help anyway.

Greetings,
Norbert

Norbert,

Either I am not understanding you correctly, or we have fundamentally different views of the role of matplotlibrc values. The way I see it, function args and kwargs *always* override matplotlibrc values, which in turn *always* override built-in defaults. So in the example above, if the user writes "plot(x,y,'r.')", red dots should be plotted no matter what is in matplotlibrc. It should not depend on whether something is set to 'auto'.

I fully agree with you.

Guess, the simple solution to the dilemma is to drop the idea of
rcfile-configurability of markerfacecolor and markeredgecolor. That way,
markers would always have the same color as the line, unless explicitely
set differently by the kwargs markeredgecolor/markerfacecolor.

This is, what is wanted in 99% of the cases and for every other case, a
rcfile-option will not help anyway.

This sounds ideal to me--it makes everything simpler, both in the code and in explaining what the behavior is.

Thanks!

Eric