date2num/num2date and ordinal date

Hi,

date2num and num2date perform conversion between datetime and 'days
since 0001-01-01' and vice versa.
For such task, they strictly use ordinal dates for their numeric days,
1 meaning '0001-01-01' by definition.
Thus, date2num(datetime.datetime(1,1,1,0,0,0)) return 1. which is
supposed to mean '1 days since 0001', which is wrong (because it
points to datetime.datetime(1,1,2,0,0,0)).

Since year zero cannot be used (here for time units) because it
doesn't exist, don't you think that the ordinal date
(datetime.datetime(1,1,2,0,0,0).tordinal()) should not be strictly
used as a reference numeric time, but its value-1?

···

--
Stephane Raynaud

Stephane Raynaud wrote:

Hi,

date2num and num2date perform conversion between datetime and 'days
since 0001-01-01' and vice versa.
For such task, they strictly use ordinal dates for their numeric days,
1 meaning '0001-01-01' by definition.
Thus, date2num(datetime.datetime(1,1,1,0,0,0)) return 1. which is
supposed to mean '1 days since 0001', which is wrong (because it
points to datetime.datetime(1,1,2,0,0,0)).

Since year zero cannot be used (here for time units) because it
doesn't exist, don't you think that the ordinal date
(datetime.datetime(1,1,2,0,0,0).tordinal()) should not be strictly
used as a reference numeric time, but its value-1?

Stephane, yes, what you say makes sense. Long ago I settled on a convention of using "decimal days" referenced to the start of a "yearbase" for time calculations and for plotting variables against time. So if the yearbase is 2008, then noon on January 1 of 2008 is 0.5. The more common convention in oceanography, though, was to label days of the year with a 1-based count and then add the fraction of the day, so what is 0.5 to me is 1.5 to many others. In this case, where the time scale origin (the start of the yearbase) might be in the middle of one's time series, the decimal day definition is clearly superior (at least to me). But in the case of the matplotlib dates module the distinction is less important, because the origin is quite arbitrary and will almost always be far smaller than the minimum of the range plotted. The datenum is mainly useful for calculations, not for direct display. Note that the datetime module (and therefore mpl.dates) simply doesn't work for BC dates.

Personally, I would be perfectly happy to implement your suggestion so that the reality would correspond to the dates module docstring; but maybe this would break some user code, so others might prefer to modify the docstring to reflect the present behavior instead.

John, I suspect you wrote the dates module and use it heavily--what do you think about the two methods of bringing the docstring and the behavior into alignment? Any problem with fixing the behavior?

Eric

The only thing I am worried about is people who have saved the
date2num output in pickle files or other array storage formats.
Changing the offset by 1 will break this data. This argues for
changing the docstring to accurately reflect what the code does,
rather than changing the code to reflect the docstring.

But I could be persuaded otherwise.

JDH

···

On Tue, Jun 3, 2008 at 3:29 PM, Eric Firing <efiring@...229...> wrote:

John, I suspect you wrote the dates module and use it heavily--what do you
think about the two methods of bringing the docstring and the behavior into
alignment? Any problem with fixing the behavior?

Just a note (that I realize will probably never get 'fixed'):

It would be nice if MPL could support more arbitrary date ranges (such as
negative numeric dates). This comes up more often than you might think.
Try writing a GUI w/ an embedded date plot - you can't switch a plot to date
format until you reset the range so it doesn't contain a zero and you can't
autoscale plots w/o data. In addition, we do plot things from a long time
ago every once in awhile. It would be nice if you could switch an arbitrary
plot to date format and not worry about the formatter throwing an exception
because the plot range isn't quite to its liking. There are algorithms for
handling numeric <-> calendar conversions that work for all time (at least
from 0 Julian Date == -4700'ish BC).

http://en.wikipedia.org/wiki/Julian_date

A switch to Julian date format for the numeric value would 'fix' these
problems though that would break existing data sets so I realize it's
unlikely to happen.

At some point in the future, I'll put a little test case together to show
how the problems w/ not supporting a zero date show up in embedded plots.

Ted

···

-----Original Message-----
From: matplotlib-devel-bounces@lists.sourceforge.net
[mailto:matplotlib-devel-bounces@lists.sourceforge.net] On Behalf Of
Eric Firing
Sent: Tuesday, June 03, 2008 1:29 PM
To: Stephane Raynaud; John Hunter
Cc: Matplotlib
Subject: Re: [matplotlib-devel] date2num/num2date and ordinal date

Stephane Raynaud wrote:
> Hi,
>
> date2num and num2date perform conversion between datetime and 'days
> since 0001-01-01' and vice versa.
> For such task, they strictly use ordinal dates for their numeric
days,
> 1 meaning '0001-01-01' by definition.
> Thus, date2num(datetime.datetime(1,1,1,0,0,0)) return 1. which is
> supposed to mean '1 days since 0001', which is wrong (because it
> points to datetime.datetime(1,1,2,0,0,0)).
>
> Since year zero cannot be used (here for time units) because it
> doesn't exist, don't you think that the ordinal date
> (datetime.datetime(1,1,2,0,0,0).tordinal()) should not be strictly
> used as a reference numeric time, but its value-1?

Stephane, yes, what you say makes sense. Long ago I settled on a
convention of using "decimal days" referenced to the start of a
"yearbase" for time calculations and for plotting variables against
time. So if the yearbase is 2008, then noon on January 1 of 2008 is
0.5. The more common convention in oceanography, though, was to label
days of the year with a 1-based count and then add the fraction of the
day, so what is 0.5 to me is 1.5 to many others. In this case, where
the time scale origin (the start of the yearbase) might be in the
middle
of one's time series, the decimal day definition is clearly superior
(at
least to me). But in the case of the matplotlib dates module the
distinction is less important, because the origin is quite arbitrary
and
will almost always be far smaller than the minimum of the range
plotted.
  The datenum is mainly useful for calculations, not for direct
display.
Note that the datetime module (and therefore mpl.dates) simply doesn't
work for BC dates.

Personally, I would be perfectly happy to implement your suggestion so
that the reality would correspond to the dates module docstring; but
maybe this would break some user code, so others might prefer to modify
the docstring to reflect the present behavior instead.

John, I suspect you wrote the dates module and use it heavily--what do
you think about the two methods of bringing the docstring and the
behavior into alignment? Any problem with fixing the behavior?

Eric

-----------------------------------------------------------------------
--
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Checked by AVG.
Version: 8.0.100 / Virus Database: 269.24.6/1480 - Release Date:
6/3/2008 7:00 AM

Eric Firing wrote:

Stephane Raynaud wrote:
  

Hi,

date2num and num2date perform conversion between datetime and 'days
since 0001-01-01' and vice versa.
For such task, they strictly use ordinal dates for their numeric days,
1 meaning '0001-01-01' by definition.
Thus, date2num(datetime.datetime(1,1,1,0,0,0)) return 1. which is
supposed to mean '1 days since 0001', which is wrong (because it
points to datetime.datetime(1,1,2,0,0,0)).

Since year zero cannot be used (here for time units) because it
doesn't exist, don't you think that the ordinal date
(datetime.datetime(1,1,2,0,0,0).tordinal()) should not be strictly
used as a reference numeric time, but its value-1?
    
Stephane, yes, what you say makes sense. Long ago I settled on a convention of using "decimal days" referenced to the start of a "yearbase" for time calculations and for plotting variables against time. So if the yearbase is 2008, then noon on January 1 of 2008 is 0.5. The more common convention in oceanography, though, was to label days of the year with a 1-based count and then add the fraction of the day, so what is 0.5 to me is 1.5 to many others. In this case, where the time scale origin (the start of the yearbase) might be in the middle of one's time series, the decimal day definition is clearly superior (at least to me). But in the case of the matplotlib dates module the distinction is less important, because the origin is quite arbitrary and will almost always be far smaller than the minimum of the range plotted. The datenum is mainly useful for calculations, not for direct display. Note that the datetime module (and therefore mpl.dates) simply doesn't work for BC dates.

Personally, I would be perfectly happy to implement your suggestion so that the reality would correspond to the dates module docstring; but maybe this would break some user code, so others might prefer to modify the docstring to reflect the present behavior instead.

John, I suspect you wrote the dates module and use it heavily--what do you think about the two methods of bringing the docstring and the behavior into alignment? Any problem with fixing the behavior?

Eric

Eric:

There are alternate versions num2date and a date2num in basemap that handle arbitrary calendars (not just 'proleptic gregorian') and arbitrary reference times (not just 'days since 0001-01-01 00:00:00'). The docstrings are at http://matplotlib.sourceforge.net/mpl_toolkits.basemap.basemap.html, down near the bottom of the page.

I don't know if this addresses the problem you're talking about though...

-Jeff

···

--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@...236...
325 Broadway Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

Hi,

There are alternate versions num2date and a date2num in basemap that handle
arbitrary calendars (not just 'proleptic gregorian') and arbitrary reference
times (not just 'days since 0001-01-01 00:00:00'). The docstrings are at
http://matplotlib.sourceforge.net/mpl_toolkits.basemap.basemap.html, down
near the bottom of the page.

I don't know if this addresses the problem you're talking about though...

I agree with you Jeff: your version of date2num (and num2date)
perfectly does the job.
Maybe the 'units' argument should defaults to 'days since 0001-01-01'
for backward compatibility.

And maybe, as for the DateConverter, the 'unit' argument in
axis_info() and convert() should follow the CF conventions
(http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.2/cf-conventions.html#time-coordinate)
as Jeff's functions, and default_units() should return 'days since
0001-01-01'.

···

On Tue, Jun 3, 2008 at 10:59 PM, Jeff Whitaker <jswhit@...196...> wrote:

-Jeff

--
Jeffrey S. Whitaker Phone : (303)497-6313
Meteorologist FAX : (303)497-6449
NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@...236...
325 Broadway Office : Skaggs Research Cntr 1D-113
Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg

--
Stephane Raynaud