Matplotlib INSISTS on using scientific notation, how do I make it STOP!?

I'm charting financial data, so scientific notation is unwanted in ALL cases.
Sometimes if I pass it data with just a few trades right near each other, it
scales so the y-axis get set to some bizare exponent, like 1.7321e1. By the
way, why would anyone ever want a plot in scientific notation where the
exponent is not a power of 10? It makes absolutely no sense whatsoever that
it would do that, it is a bug.

So, here is the code I'm trying to use to force matplotlib to NOT use
scientific notation on the y-axis, unsuccesfully:

   formatter = ScalarFormatter()
   formatter.set_scientific(False)
   formatter.set_powerlimits((-10000,10000))
   pl.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
   pl.yaxis.set_major_formatter(formatter)
   pl.yaxis.set_minor_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
   pl.yaxis.set_minor_formatter(formatter)
   pl.set_autoscaley_on(False)
   yfm = pl.yaxis.get_major_formatter()
   yfm.set_powerlimits([ -10000, 10000])

None of these attempts to turn off scientific notation is working. Any
suggestions? Please?

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

I'm charting financial data, so scientific notation is unwanted in ALL cases.
Sometimes if I pass it data with just a few trades right near each other, it
scales so the y-axis get set to some bizare exponent, like 1.7321e1. By the
way, why would anyone ever want a plot in scientific notation where the
exponent is not a power of 10? It makes absolutely no sense whatsoever that
it would do that, it is a bug.

So, here is the code I'm trying to use to force matplotlib to NOT use
scientific notation on the y-axis, unsuccesfully:

    formatter = ScalarFormatter()
    formatter.set_scientific(False)
    formatter.set_powerlimits((-10000,10000))
    pl.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
    pl.yaxis.set_major_formatter(formatter)
    pl.yaxis.set_minor_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
    pl.yaxis.set_minor_formatter(formatter)
    pl.set_autoscaley_on(False)
    yfm = pl.yaxis.get_major_formatter()
    yfm.set_powerlimits([ -10000, 10000])

None of these attempts to turn off scientific notation is working. Any
suggestions? Please?

Please provide a minimal script illustrating the problem, that is, triggering the unwanted axis handling.

Eric

···

On 2013/01/26 5:16 PM, Todamont wrote:

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnnow-d2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

This is the relevant code:

import sys, shutil
import matplotlib
from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg
from mhd_scipy import load_mhd
from datetime import datetime
import matplotlib.dates as dates
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.font_manager
from matplotlib.ticker import ScalarFormatter, FormatStrFormatter,
FuncFormatter

...

   fig=Figure(figsize=(5.5,2.4))
   pl=fig.add_subplot(111)
  
pl.plot_date(zip(graphdate),zip(price),'-',color='white',lw='0.5',alpha=0.25)
   pl.set_ylim([min(price),max(price)])
   ax2=pl.twinx()
   ax2.set_ylim([min(price),max(price)])

# here is my attempt to force NON-SCIENTIFIC axes
   formatter = ScalarFormatter()
   formatter.set_scientific(False)
   formatter.set_powerlimits((-1000000000,10000))
   pl.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
   pl.yaxis.set_major_formatter(formatter)
   pl.yaxis.set_minor_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
   pl.yaxis.set_minor_formatter(formatter)
   pl.set_autoscaley_on(False)
   yfm = pl.yaxis.get_major_formatter()
   yfm.set_powerlimits([ -100000000000, 10000])

#finally, render...
   canvas=FigureCanvasAgg(fig) bigName = pyArgs[4] + "_big.png" # create
image name string..
   canvas.print_figure(bigName,dpi=200) # create date/timestamped file
   print "Large line-graph Created"

So, if I pass this thing price data that includes only 1 trade, or just a
few trades that are within a cent or two of each other, that's when the
bizarre axis scaling happens...

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40322.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

This is the relevant code:

Thanks, but what would really help is not what you think is the relevant code, but a completely self-contained *minimal* script, so that I can run it as-is, then modify it (probably only slightly), and return the modified version.

I suspect that using plot_date and a few fake data points, you can reproduce the problem. I doubt that it matters whether you use pyplot with an interactive backend, or use the OO form with FigureCanvasAgg.

Eric

···

On 2013/01/26 5:33 PM, Todamont wrote:

import sys, shutil
import matplotlib
from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg
from mhd_scipy import load_mhd
from datetime import datetime
import matplotlib.dates as dates
import matplotlib.pyplot as plt
import numpy as np
import matplotlib.font_manager
from matplotlib.ticker import ScalarFormatter, FormatStrFormatter,
FuncFormatter

...

    fig=Figure(figsize=(5.5,2.4))
    pl=fig.add_subplot(111)

pl.plot_date(zip(graphdate),zip(price),'-',color='white',lw='0.5',alpha=0.25)
    pl.set_ylim([min(price),max(price)])
    ax2=pl.twinx()
    ax2.set_ylim([min(price),max(price)])

# here is my attempt to force NON-SCIENTIFIC axes
    formatter = ScalarFormatter()
    formatter.set_scientific(False)
    formatter.set_powerlimits((-1000000000,10000))
    pl.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
    pl.yaxis.set_major_formatter(formatter)
    pl.yaxis.set_minor_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
    pl.yaxis.set_minor_formatter(formatter)
    pl.set_autoscaley_on(False)
    yfm = pl.yaxis.get_major_formatter()
    yfm.set_powerlimits([ -100000000000, 10000])

#finally, render...
    canvas=FigureCanvasAgg(fig) bigName = pyArgs[4] + "_big.png" # create
image name string..
    canvas.print_figure(bigName,dpi=200) # create date/timestamped file
    print "Large line-graph Created"

So, if I pass this thing price data that includes only 1 trade, or just a
few trades that are within a cent or two of each other, that's when the
bizarre axis scaling happens...

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40322.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnnow-d2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

I'm trying to boil this thing down to a simple example program for you, it's
part of a big project so it's kind of tough, all the data is stored in RAM,
never in files...

It seems like there should be some way to just tell matplotlib to NEVER use
scientific notation for the axes... I'd be willing to alter my matplotlibrc
or even my matplotlib build itself, because this is the only application I
use it for. I can't even seem to find the part of the code backend that is
responsible for this weird behavior, though.

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40324.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Ok, here is the distilled code that displays the scientific formatting
failure I'm trying to fix. I had to wait until the market gave me a snapshot
that triggered this bug. Here is a quick link to the image I get when I run
this program: http://imgur.com/77DUbZp

I greatly appreciate your help with this. Any suggestions are welcome. Also,
this code will become FOSS when it is completed. Do you think it is related
to the way I'm loading in the data with the "extent" command and then making
a subplot? I'm generating a custom heatmap of the market density for the
background, so I sort of need that...

#!/usr/bin/env python2.7
# coding: utf-8
import matplotlib
from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg
from datetime import datetime
import matplotlib.dates as dates
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import ScalarFormatter, FormatStrFormatter,
FuncFormatter
import matplotlib.image as mpimg

date = [ 734894.994583, 734894.995313, 734894.996042, 734894.997384,
734894.998032 ]
price = [ 17.37, 17.30001, 17.30001, 17.39, 17.39 ]
graphdate = []
for i in date:
   graphdate.append(i)

graph = np.zeros((100,100,3))
fig=Figure(figsize=(5.5,2.4))
pl=fig.add_subplot(111)
image=pl.imshow(graph,aspect="auto",extent=(min(date),max(date),min(price),max(price)))
fig.subplots_adjust(top=0.92,bottom=0.15,left=0.1,right=0.95)

# add plot of value across time
pl.plot_date(zip(graphdate),zip(price),'-',color='blue',lw='1',alpha=1)
matplotlib.rcParams.update({'font.size': 5})

locator = dates.AutoDateLocator()
pl.set_ylim([min(price),max(price)])
ax2=pl.twinx()
ax2.set_ylim([min(price),max(price)])

formatter = ScalarFormatter()
formatter.set_scientific(False)
formatter.set_powerlimits((-1000000000,10000))
pl.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
pl.yaxis.set_major_formatter(formatter)
pl.yaxis.set_minor_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
pl.yaxis.set_minor_formatter(formatter)
pl.set_autoscaley_on(False)
yfm = pl.yaxis.get_major_formatter()
yfm.set_powerlimits([ -100000000000, 10000])

ax2.yaxis.set_major_formatter(formatter)
ax2.yaxis.set_major_formatter(ScalarFormatter(useOffset=False))
ax2.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
ax2.yaxis.set_major_formatter(formatter)
ax2.yaxis.set_minor_formatter(ScalarFormatter(useOffset=False))
ax2.yaxis.set_minor_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
ax2.yaxis.set_major_formatter(formatter)
ax2.set_autoscaley_on(False)
yfm = ax2.yaxis.get_major_formatter()
yfm.set_powerlimits([ -100000000000, 10000])

period = 298; # duration in seconds for this graph
seconds = dates.SecondLocator(interval=30) # plot by 30-second increment
secondsFmt = dates.DateFormatter('%-I:%M:%S%P',tz=None)
pl.xaxis.set_major_locator(seconds)
pl.xaxis.set_major_formatter(secondsFmt)

labels = pl.get_xticklabels()
for label in labels:
   label.set_rotation(45)

pl.xaxis.grid(which='major',color='white', linestyle='--', linewidth=0.5,
alpha=0.15)
pl.xaxis.grid(which='minor',color='white', linestyle='--', linewidth=0.5,
alpha=0.15)
pl.yaxis.grid(which='major',color='white', linestyle='--', linewidth=0.5,
alpha=0.15)
pl.yaxis.grid(which='minor',color='white', linestyle='--', linewidth=0.5,
alpha=0.15)

canvas=FigureCanvasAgg(fig)
bigName = "sci_formatting_fail.png" # image name
canvas.print_figure(bigName,dpi=200) # create date/timestamped file
print "grapher completed"

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40326.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Ok, here is the distilled code that displays the scientific formatting
failure I'm trying to fix. I had to wait until the market gave me a snapshot
that triggered this bug. Here is a quick link to the image I get when I run
this program: http://imgur.com/77DUbZp

What you were seeing is not an exponent, it is an offset. The attached modification tells mpl not to use an offset, and it also simplifies things by omitting the twinx, which is not needed, and by omitting all the extra fussing with formatters, likewise not needed.

The algorithm for automatically choosing an offset (when an offset is desired) is known to be over-eager, but it turns out to be surprisingly difficult to come up with a better algorithm that works in a wide range of circumstances.

Now, are you sure you really always want to set the y limits to exactly your y data limits? You will never have only a single point? Or only identical y values?

Eric

test_y.py (1.88 KB)

···

On 2013/01/26 7:59 PM, Todamont wrote:

I greatly appreciate your help with this. Any suggestions are welcome. Also,
this code will become FOSS when it is completed. Do you think it is related
to the way I'm loading in the data with the "extent" command and then making
a subplot? I'm generating a custom heatmap of the market density for the
background, so I sort of need that...

#!/usr/bin/env python2.7
# coding: utf-8
import matplotlib
from matplotlib.figure import Figure
from matplotlib.backends.backend_agg import FigureCanvasAgg
from datetime import datetime
import matplotlib.dates as dates
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import ScalarFormatter, FormatStrFormatter,
FuncFormatter
import matplotlib.image as mpimg

date = [ 734894.994583, 734894.995313, 734894.996042, 734894.997384,
734894.998032 ]
price = [ 17.37, 17.30001, 17.30001, 17.39, 17.39 ]
graphdate = []
for i in date:
    graphdate.append(i)

graph = np.zeros((100,100,3))
fig=Figure(figsize=(5.5,2.4))
pl=fig.add_subplot(111)
image=pl.imshow(graph,aspect="auto",extent=(min(date),max(date),min(price),max(price)))
fig.subplots_adjust(top=0.92,bottom=0.15,left=0.1,right=0.95)

# add plot of value across time
pl.plot_date(zip(graphdate),zip(price),'-',color='blue',lw='1',alpha=1)
matplotlib.rcParams.update({'font.size': 5})

locator = dates.AutoDateLocator()
pl.set_ylim([min(price),max(price)])
ax2=pl.twinx()
ax2.set_ylim([min(price),max(price)])

formatter = ScalarFormatter()
formatter.set_scientific(False)
formatter.set_powerlimits((-1000000000,10000))
pl.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
pl.yaxis.set_major_formatter(formatter)
pl.yaxis.set_minor_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
pl.yaxis.set_minor_formatter(formatter)
pl.set_autoscaley_on(False)
yfm = pl.yaxis.get_major_formatter()
yfm.set_powerlimits([ -100000000000, 10000])

ax2.yaxis.set_major_formatter(formatter)
ax2.yaxis.set_major_formatter(ScalarFormatter(useOffset=False))
ax2.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
ax2.yaxis.set_major_formatter(formatter)
ax2.yaxis.set_minor_formatter(ScalarFormatter(useOffset=False))
ax2.yaxis.set_minor_formatter(FuncFormatter(lambda x, pos: '%.0f'%x))
ax2.yaxis.set_major_formatter(formatter)
ax2.set_autoscaley_on(False)
yfm = ax2.yaxis.get_major_formatter()
yfm.set_powerlimits([ -100000000000, 10000])

period = 298; # duration in seconds for this graph
seconds = dates.SecondLocator(interval=30) # plot by 30-second increment
secondsFmt = dates.DateFormatter('%-I:%M:%S%P',tz=None)
pl.xaxis.set_major_locator(seconds)
pl.xaxis.set_major_formatter(secondsFmt)

labels = pl.get_xticklabels()
for label in labels:
    label.set_rotation(45)

pl.xaxis.grid(which='major',color='white', linestyle='--', linewidth=0.5,
alpha=0.15)
pl.xaxis.grid(which='minor',color='white', linestyle='--', linewidth=0.5,
alpha=0.15)
pl.yaxis.grid(which='major',color='white', linestyle='--', linewidth=0.5,
alpha=0.15)
pl.yaxis.grid(which='minor',color='white', linestyle='--', linewidth=0.5,
alpha=0.15)

canvas=FigureCanvasAgg(fig)
bigName = "sci_formatting_fail.png" # image name
canvas.print_figure(bigName,dpi=200) # create date/timestamped file
print "grapher completed"

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40326.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnnow-d2d
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Thank you! Success!

Yeah, I learned a little from this little exercise. I will set the "extent"
on the image to include a buffer in the y-axis, a certain percentage of
max(price)-min(price), for some durations.

If there is only a single trade during any duration, well... I need handle
that as a special case, now that you mention it. It doesn't like that,
basically. I need to go back and get the second-latest trade, and plot it
*outside* the graph, or some variant or hack of that, so that it draws a
line to the most recent trade, and then create another artificial point at
the same price at the current time, to indicate a flat market...

Thanks again for your help. I've been at wit's end trying to fix that...
Cheers~

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40328.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

for the record I had to put the buffer on the set_limit commands, not the
"extent" of the image, like so:

   priceDiff = max(price)-min(price)
   diffQuant = priceDiff / 4
   minQuant = min(price)-diffQuant
   maxQuant = max(price)+diffQuant
   pl.set_ylim([minQuant,maxQuant])

and also include:

   pl.set_xlim([min(date),max(date)])

to get it working just right. Just for posterity. Thanks again~

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40329.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Hmmmmm. This stretches the background image (mhd data) to fit, leaving the
price line-plot not lining up with the background density plot. Now I have
to figure out how to make the density plot scale to fit...

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40330.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Ok, finally, to get the scaling to work, I rewrote the c density plotter to
accept a scaleFactor argument to affect a new max/min y-axis on that end and
pass it to the python script which reads the background as an mhd dataset.
Then I had to update my python to include the new scaled y-axis limits in
both the extent in the imshow() function *and* the set_ylim() commands. Now
I have a nice scalable buffer on the price band. Cheers~

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Matplotlib-INSISTS-on-using-scientific-notation-how-do-I-make-it-STOP-tp40320p40338.html
Sent from the matplotlib - users mailing list archive at Nabble.com.