Sequential Calls to create bar charts jumbles output on second. first always ok.

I extract data out of a database (via Django and Python). I'm drawing two bar charts. One is a stacked bar, and one is a simple bar.
The code for each graph is in their own function. I call the function to draw the graph sequencially (in function "CreateAllWebSite() below).

Problem is that when I do this, the graph called second comes out all jumbled. See attached copies of png files. If I swap the function calls, always first is ok and the the second one is jumbled. If I call them one at a time, they both work just fine (which is how i developed the second). once put together, the second is wrong. For example, the legend from the first graph gets put on the second graph . The x-axis for the second is all jumbled at the bottom left corner of the graph instead of along the x-axis properly.

I'm using Matplotlib 0.98.5.3 on OS X. I will upgrade to 1.0 of Matplot lib soon.

I suspect some sort of memory issue; perhaps caused by how I import matplotlib in the global section of the python program; but far as I can tell this is correct, or is it? I suspect memory as how else would legend from the first graph get put on the second graph? Is there some sort of reset function to call to clear out the old graph? I sort of thought that calling a completely different function would cover that.

Here's the code extract with portions I think no relevant snipped out. From debugging I'm confident the number arrays holding the data is ok. As I said, when producing the graphs one at a time, it work fine. So that code snipped out below.

#!/usr/bin/env python
# −*− coding: UTF−8 −*−

import matplotlib
matplotlib.use('Agg') # before importing pyplot per docs
import matplotlib.pyplot as plt
import numpy as np

[snipped the import of Python and Django libraries]

memberstat_barchart.png

memberstat_barchart_cat.png

···

###########################################################
def CreateAllWebSite():
  # here is the one function that calls the two graphing functions sequentially.
  # whichever of these is called first is done fine. Second flawed. calling one at at time, each works ok
  CreateMemberStatFigure()
  CreateMemberStatCategoryFigure()
###############################################
def CreateMemberStatFigure():
# this function creates a stacked bar chart
[snipped out portions where arrays created]
  corp1=[]
  corp2=[]
  corp3=[]
  ordm=[]
  ordr=[]
  ordh=[]
  month=[]
  for i in memberstatsp:
      month.append(None2Zero(i['month'])) # None2Zero simply converts any "None" data to a number zero.
      corp1.append(None2Zero(i['CORP1']))
      corp2.append(None2Zero(i['CORP2']))
      corp3.append(None2Zero(i['CORP3']))
      ordm.append(None2Zero(i['ORM']))
      ordr.append(None2Zero(i['ORMR']))
      ordh.append(None2Zero(i['HON']))
  N=len(month)
  corp1=np.array(corp1)
  corp2=np.array(corp2)
  corp3=np.array(corp3)
  ordm=np.array(ordm)
  ordr=np.array(ordr)
  ordh=np.array(ordh)

  # create new bases for each bar
  base=np.zeros(N,int)
  base1=base+ordh
  base2=base1+corp3
  base3=base2+corp2
  base4=base3+corp1
  base5=base4+ordm

  width = 24 # the width of the bars: can also be len(x) sequence

  p1 = plt.bar(month, ordh, width, color=barchartcolor[0],linewidth=0)
  p2 = plt.bar(month, corp3, width, color=barchartcolor[1],bottom=base1,linewidth=0)
  p3 = plt.bar(month, corp2, width, color=barchartcolor[2],bottom=base2,linewidth=0)
  p4 = plt.bar(month, corp1, width, color=barchartcolor[3],bottom=base3,linewidth=0)
  p5 = plt.bar(month, ordm, width, color=barchartcolor[4],bottom=base4,linewidth=0)
  p6 = plt.bar(month, ordr, width, color=barchartcolor[5],bottom=base5,linewidth=0)

  plt.yticks(np.arange(0,200,20))
  plt.title(ORG_NAME+' Membership by Month')
  plt.ylabel('Count')
  plt.legend( (p1[0], p2[0], p3[0], p4[0], p5[0], p6[0]),mtypes,loc='best')
  plt.show()

  matplotlib.rcParams['figure.figsize']=(3,2)

  fn=TEMP_DIR+SOC_MEMBER_STAT_BARCHART_FIGURE
  plt.savefig(fn)

  return

###############################################
def CreateMemberStatCategoryFigure():
  membercategories=Membercategory.objects.all() # gets the categories out of the database
  mcats=[]
  cnt=[]
  for i in membercategories:
      mcats.append(i.category)
      cnt.append(Member.Active_objects.filter(membercategory__category=i.category).count())
  xlocations = np.array(range(len(mcats)))+0.5
  p1 = plt.bar(xlocations,cnt)
  plt.figtext(0.15,0.85,dt)
  plt.xticks(xlocations+0.4, mcats,rotation=17)
  plt.title(ORG_NAME+' Membership by Member Category')
  plt.ylabel('Count')
  plt.show()

  matplotlib.rcParams['figure.figsize']=(3,2)

  fn=TEMP_DIR+SOC_MEMBER_STAT_BARCHART_CAT_FIGURE
  plt.savefig(fn)

  return

################################################

Hi Rob,

It would be helpful if you could provide a runnable example. I know it's a pain, but you should simplify your example so that it's runnable with only matplotlib and numpy You should create some fake data to replace these portions that you've cut out. Plus there are some (Django?) classes/functions/globals that are undefined.

Best,
-Tony

···

On Aug 11, 2010, at 6:56 AM, Rob Schneider wrote:

I extract data out of a database (via Django and Python). I'm drawing two bar charts. One is a stacked bar, and one is a simple bar.
The code for each graph is in their own function. I call the function to draw the graph sequencially (in function "CreateAllWebSite() below).

Problem is that when I do this, the graph called second comes out all jumbled. See attached copies of png files. If I swap the function calls, always first is ok and the the second one is jumbled. If I call them one at a time, they both work just fine (which is how i developed the second). once put together, the second is wrong. For example, the legend from the first graph gets put on the second graph . The x-axis for the second is all jumbled at the bottom left corner of the graph instead of along the x-axis properly.

I'm using Matplotlib 0.98.5.3 on OS X. I will upgrade to 1.0 of Matplot lib soon.

I suspect some sort of memory issue; perhaps caused by how I import matplotlib in the global section of the python program; but far as I can tell this is correct, or is it? I suspect memory as how else would legend from the first graph get put on the second graph? Is there some sort of reset function to call to clear out the old graph? I sort of thought that calling a completely different function would cover that.

Here's the code extract with portions I think no relevant snipped out. From debugging I'm confident the number arrays holding the data is ok. As I said, when producing the graphs one at a time, it work fine. So that code snipped out below.

#!/usr/bin/env python
# −*− coding: UTF−8 −*−

import matplotlib
matplotlib.use('Agg') # before importing pyplot per docs
import matplotlib.pyplot as plt
import numpy as np

[snipped the import of Python and Django libraries]

###########################################################
def CreateAllWebSite():
# here is the one function that calls the two graphing functions sequentially.
# whichever of these is called first is done fine. Second flawed. calling one at at time, each works ok
CreateMemberStatFigure()
CreateMemberStatCategoryFigure()
###############################################
def CreateMemberStatFigure():
# this function creates a stacked bar chart
[snipped out portions where arrays created]

Agreed. The only thing I can think of is that the second figure is
reusing the first. You can try calling plt.figure() at the beginning
of the functions to create a new figure, or call plt.figure() in
between the calls to CreateMemberStatCategoryFigure() and
CreateMemberStatFigure(). I can't be sure since you didn't include the
code that actually calls these functions.

Ryan

···

On Wed, Aug 11, 2010 at 8:43 AM, Tony S Yu <tsyu80@...287...> wrote:

On Aug 11, 2010, at 6:56 AM, Rob Schneider wrote:

I extract data out of a database (via Django and Python). I'm drawing two bar charts. One is a stacked bar, and one is a simple bar.
The code for each graph is in their own function. I call the function to draw the graph sequencially (in function "CreateAllWebSite() below).

Problem is that when I do this, the graph called second comes out all jumbled. See attached copies of png files. If I swap the function calls, always first is ok and the the second one is jumbled. If I call them one at a time, they both work just fine (which is how i developed the second). once put together, the second is wrong. For example, the legend from the first graph gets put on the second graph . The x-axis for the second is all jumbled at the bottom left corner of the graph instead of along the x-axis properly.

I'm using Matplotlib 0.98.5.3 on OS X. I will upgrade to 1.0 of Matplot lib soon.

I suspect some sort of memory issue; perhaps caused by how I import matplotlib in the global section of the python program; but far as I can tell this is correct, or is it? I suspect memory as how else would legend from the first graph get put on the second graph? Is there some sort of reset function to call to clear out the old graph? I sort of thought that calling a completely different function would cover that.

Here's the code extract with portions I think no relevant snipped out. From debugging I'm confident the number arrays holding the data is ok. As I said, when producing the graphs one at a time, it work fine. So that code snipped out below.

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

import matplotlib
matplotlib.use('Agg') # before importing pyplot per docs
import matplotlib.pyplot as plt
import numpy as np

[snipped the import of Python and Django libraries]

###########################################################
def CreateAllWebSite():
# here is the one function that calls the two graphing functions sequentially.
# whichever of these is called first is done fine. Second flawed. calling one at at time, each works ok
CreateMemberStatFigure()
CreateMemberStatCategoryFigure()
###############################################
def CreateMemberStatFigure():
# this function creates a stacked bar chart
[snipped out portions where arrays created]

Hi Rob,

It would be helpful if you could provide a runnable example. I know it's a pain, but you should simplify your example so that it's runnable with only matplotlib and numpy You should create some fake data to replace these portions that you've cut out. Plus there are some (Django?) classes/functions/globals that are undefined.

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Agreed. The only thing I can think of is that the second figure is
reusing the first. You can try calling plt.figure() at the beginning
of the functions to create a new figure, or call plt.figure() in
between the calls to CreateMemberStatCategoryFigure() and
CreateMemberStatFigure(). I can't be sure since you didn't include the
code that actually calls these functions.

Ryan

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

-

Ryan,

Thanks. Indeed I did provide the code that actually calls these functions and the functions show all plotting-related lines. Yes, I did not provide the single line call that calls the function that calls the two plotting functions, nor did I provide the call which starts the Python program. And I did not provide runnable code. Everyone's attention was on what code I did or did not provide. It was all there. Naively I thought by code inspection by experts who read this mailing list they could quickly spot my flaw.

And a flaw it turned out to be.

I took your suggestion to put a plt.figure() call at the beginning of each function that generates a plot and the problem disappeared. Now calling each plotting function sequentially within the same instance of a Python program the second plot does not get scrambled. This is good as it now provides the automation I was hoping for.

I've now also found http://matplotlib.sourceforge.net/api/pyplot_api.html#matplotlib.pyplot.figure and http://matplotlib.sourceforge.net/api/pyplot_api.html#matplotlib.pyplot.close which look to be important. I was not aware of the need for these functions before. Oversight on my part. Using them now.

I'm now off doing more productive things. thanks for the pointer!!

2010/8/14 Rob Schneider <rmschne@...3231...>:

Agreed. The only thing I can think of is that the second figure is
reusing the first. You can try calling plt.figure() at the beginning
of the functions to create a new figure, or call plt.figure() in
between the calls to CreateMemberStatCategoryFigure() and
CreateMemberStatFigure(). I can't be sure since you didn't include the
code that actually calls these functions.

Still, there shouldn't be artists rendered outside of the axes. I
often replot things and do not create a new figure. I believe there
is something under the hood! It *should* be unnecessary to create a
new fig, although it surely fixes the thing, and is an easy enough
fix.

Thanks. Indeed I did provide the code that actually calls these functions and the functions show all plotting-related lines. Yes, I did not provide the single line call that calls the function that calls the two plotting functions, nor did I provide the call which starts the Python program. And I did not provide runnable code. Everyone's attention was on what code I did or did not provide. It was all there. Naively I thought by code inspection by experts who read this mailing list they could quickly spot my flaw.

You know, we are all volunteering our time here on the list to help,
and I think blaming us to not be able to read hundred lines of code is
not productive at all. It's not our side of the task to track whole
parts of programs down. And, usually one has to play with the code
and to run it to see if it fixes the problem!

You can help improving matplotlib by providing the things the core
developers or experienced Python users like me need to track the error
down inside of matplotlib.

Friedrich

P.S.: Try to not think in experts and customers terms.

One bar chart was categorical, the other based on dates. That threw
the axes scaling completely off. There weren't any misdrawn artists,
unless you're referring to the jumble of text. These are actually
tightly packed ticklabels for x-axis. Since the scaling is fubar-ed,
it looks messed up.

Not sure what could be reasonably expected in such a case.

Ryan

···

On Wed, Aug 18, 2010 at 3:32 PM, Friedrich Romstedt <friedrichromstedt@...287...> wrote:

2010/8/14 Rob Schneider <rmschne@...3231...>:

Agreed. The only thing I can think of is that the second figure is
reusing the first. You can try calling plt.figure() at the beginning
of the functions to create a new figure, or call plt.figure() in
between the calls to CreateMemberStatCategoryFigure() and
CreateMemberStatFigure(). I can't be sure since you didn't include the
code that actually calls these functions.

Still, there shouldn't be artists rendered outside of the axes. I
often replot things and do not create a new figure. I believe there
is something under the hood! It *should* be unnecessary to create a
new fig, although it surely fixes the thing, and is an easy enough
fix.

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

I think the graphs were composed correctly, else why would they ever work? They were fixed by calling the figure() and close() functions. i'm reporting that to the list with the hope that others can value from it.

···

On Wed, Aug 18, 2010 at 3:32 PM, Friedrich Romstedt > <friedrichromstedt@...287...> wrote:

2010/8/14 Rob Schneider <rmschne@...3231...>:

Agreed. The only thing I can think of is that the second figure is
reusing the first. You can try calling plt.figure() at the beginning
of the functions to create a new figure, or call plt.figure() in
between the calls to CreateMemberStatCategoryFigure() and
CreateMemberStatFigure(). I can't be sure since you didn't include the
code that actually calls these functions.

Still, there shouldn't be artists rendered outside of the axes. I
often replot things and do not create a new figure. I believe there
is something under the hood! It *should* be unnecessary to create a
new fig, although it surely fixes the thing, and is an easy enough
fix.

One bar chart was categorical, the other based on dates. That threw
the axes scaling completely off. There weren't any misdrawn artists,
unless you're referring to the jumble of text. These are actually
tightly packed ticklabels for x-axis. Since the scaling is fubar-ed,
it looks messed up.

Not sure what could be reasonably expected in such a case.

Ryan

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

You know, we are all volunteering our time here on the list to help,
and I think blaming us to not be able to read hundred lines of code is
not productive at all. It's not our side of the task to track whole
parts of programs down. And, usually one has to play with the code
and to run it to see if it fixes the problem!

You can help improving matplotlib by providing the things the core
developers or experienced Python users like me need to track the error
down inside of matplotlib.

Friedrich

P.S.: Try to not think in experts and customers terms.

I'm sorry you took anything I said as "blame". Not intended that way. Just stating that using figure() and close() resolved the issue. As I look back on the material I've used to learn how to use Matplotlib, they sometimes call these functions and sometimes not. I've concluded that it is required to call these functions. The message to this list is that it *appears* at least to me that it is required to call these functions. Hopefully that will help someone.

The graphs *were* composed just fine, that wasn't my point. The
problem was that the original code tried (by accident) to take two
fundamentally different graphs and put them on the same axes.
Matplotlib assumes that unless you tell it otherwise [by creating a
new figure or calling hold(false)], everything should be on the same
figure. Hence, adding a call to figure() fixes the problem by putting
the second graph on its own figure. Now, if there is not an existing
figure, the commands from pyplot will create one for you, but that's
only in the case where there are no existing (live, not closed)
figures.

Ryan

···

On Thu, Aug 19, 2010 at 6:49 AM, Rob Schneider <rmschne@...3231...> wrote:

On Wed, Aug 18, 2010 at 3:32 PM, Friedrich Romstedt >> <friedrichromstedt@...287...> wrote:

2010/8/14 Rob Schneider <rmschne@...3231...>:

Agreed. The only thing I can think of is that the second figure is
reusing the first. You can try calling plt.figure() at the beginning
of the functions to create a new figure, or call plt.figure() in
between the calls to CreateMemberStatCategoryFigure() and
CreateMemberStatFigure(). I can't be sure since you didn't include the
code that actually calls these functions.

Still, there shouldn't be artists rendered outside of the axes. I
often replot things and do not create a new figure. I believe there
is something under the hood! It *should* be unnecessary to create a
new fig, although it surely fixes the thing, and is an easy enough
fix.

One bar chart was categorical, the other based on dates. That threw
the axes scaling completely off. There weren't any misdrawn artists,
unless you're referring to the jumble of text. These are actually
tightly packed ticklabels for x-axis. Since the scaling is fubar-ed,
it looks messed up.

Not sure what could be reasonably expected in such a case.

Ryan

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

I think the graphs were composed correctly, else why would they ever work? They were fixed by calling the figure() and close() functions. i'm reporting that to the list with the hope that others can value from it.

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

2010/8/14 Rob Schneider <rmschne@…3231…>:

Agreed. The only thing I can think of is that the second figure is

reusing the first. You can try calling plt.figure() at the beginning

of the functions to create a new figure, or call plt.figure() in

between the calls to CreateMemberStatCategoryFigure() and

CreateMemberStatFigure(). I can’t be sure since you didn’t include the

code that actually calls these functions.

Still, there shouldn’t be artists rendered outside of the axes. I

often replot things and do not create a new figure. I believe there

is something under the hood! It should be unnecessary to create a

new fig, although it surely fixes the thing, and is an easy enough

fix.

One bar chart was categorical, the other based on dates. That threw

the axes scaling completely off. There weren’t any misdrawn artists,

unless you’re referring to the jumble of text. These are actually

tightly packed ticklabels for x-axis. Since the scaling is fubar-ed,

it looks messed up.

Not sure what could be reasonably expected in such a case.

Ryan

Ryan May

Graduate Research Assistant

School of Meteorology

University of Oklahoma

I think the graphs were composed correctly, else why would they ever work? They were fixed by calling the figure() and close() functions. i’m reporting that to the list with the hope that others can value from it.

Just to add my 2 cents…

In python, explicit is preferred over implicit. So, it would be a good idea to call figure() or any other figure-handling function when you mean to use them. However, because we strive to cater to those who come from Matlab, we need to make matplotlib a little bit more robust and allow it to implicitly make assumptions when the programmer does not explicitly state their intentions.

So, while the following code example works just fine:

from pylab import *
scatter([0.0, 0.5, 2.4, 1.2], [1.4, 0.7, 2.1, 0.3])

title(“This is a scatter plot”)
show()

The “better”, more robust way would be this:

import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.gca()
ax.scatter([0.0, 0.5, 2.4, 1.2], [1.4, 0.7, 2.1, 0.3])
ax.set_title(“This is a scatter plot”)
plt.show()

This way, you explicitly state that you want to do a particular action on a particular axes that belongs in a particular figure. With this approach, you can make functions that take a ‘ax’ parameter and make specialty graphs on whatever axes you tell it. You can have multiple figures made simultaneously with multiple axes (think subplots) and be able to open and close whichever ones you want whenever you like.

In the documentation, you see a variety of styles because of this flexibility. Whether this is wise or not, that is probably a different question.

I hope this clears up any confusion you and other future matplotlib-ers have! I hope you continue to use and enjoy matplotlib!

Ben Root

···

On Thu, Aug 19, 2010 at 6:49 AM, Rob Schneider <rmschne@…3244…> wrote:

On Wed, Aug 18, 2010 at 3:32 PM, Friedrich Romstedt > > > <friedrichromstedt@…120…287…> wrote:

2010/8/19 Rob Schneider <rmschne@...3231...>:

I'm sorry you took anything I said as "blame". Not intended that way. Just stating that using figure() and close() resolved the issue. As I look back on the material I've used to learn how to use Matplotlib, they sometimes call these functions and sometimes not. I've concluded that it is required to call these functions. The message to this list is that it *appears* at least to me that it is required to call these functions. Hopefully that will help someone.

Ok, I took it wrongly. My mistake. Sorry.

I want to second Ben, with hoping that you will be a vivid member of
the matplotlib community.

Friedrich