Date overhaul?

Development matplotlib-devel
1

I was hoping for something a bit more extensive of the intenals. I have tried to understand the units code a couple of times now and always hit a brick wall.

···

On Thu, Jan 8, 2015 at 7:04 AM, Skip Montanaro
<skip@…503…> wrote:

I’m real naive about this stuff, but I have always wondered why

matplotlib didn’t just use datetime objects, or at least use

timezone-aware datetime objects as an “interchange” format to get the

timezone stuff right.

Time zone handling is a pain in the %}€*

And the definitions keep changing.

So you need a complex DB and library that needs frequent updating.

This is why neither the standard library nor numpy support time zone handling out of the box.

But the datetime object does support a hook to add timezone info.

The numpy datetime64 may implementation may provide a similar hook in the future.

There is the pytz package, which MPL could choose to depend on.

But even that is a bit ugly–e.g. from the pytz docs:

“”“Unfortunately using the tzinfo argument of the standard datetime constructors ‘’does not work’’ with pytz for many timezones.”“”

So my suggestion is that MPL punts, and stick with leaving time zone handling up to the user, I.e only use datetimes that are timezone “naive”. What this means is that MPL would always a assume all datetimes interacting with
each other are in the same time zone (including same DST status).

Anyway, I’m being a bit lazy here, so I may be wrong, but I think the issue at hand is that MPL currently uses a float array to store and manipulate datetimes, and the thought is that it may be better to use
numpy datetime64 arrays – that would give us more consistent precision, and less code to convert to/from various datetime formats.

I’m a bit on the fence about whether it’s time to do it, as datetime64 does have issues with the locale timezone, but as any implementation would want to work with not-just-the-latest numpy anyway, it may make sense to start now.
-Chris

–

Christopher Barker, Ph.D.

Oceanographer

Emergency Response Division

NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@…272…236…

2

Let’s say we want a time zone aware date time converter. The basic goal is to convert some input type (datetime) to the MPL internal type (float days past Jan 0, 0001). We also need to tell MPL how to format the axis (default formatter, locator, limits,
label).

  • The convert() method takes the input type (datetime) and the xunits (or yunits) keyword argument and converts it to the MPL type. The axis input can be used to change the results depending on the plot type (polar plots always require radians for example).
    For the TZ converter, would take the input value (datetime), convert it to the time zone requested by the units input, then convert that to a float using dates.date2num(). Note that the input can be a sequence or a single value so the converter has to handle
    both cases.

  • The axisinfo() method is used to provide the default axis locator and formatter objects when the user creates a plot with this type. The axis input is useful here to handle the result differently for a polar plot. For the TZ converter, this would be
    exactly the same as the web docs - return the default locator and formatter for dates. Most of the time this method can just return standard MPL formatters and locators (for either dates or numbers).

  • The default_units() method provides a default value for the xunits or yunits keyword argument if one isn’t specified. The default in this case might be “UTC”.

Hope that helps some, if you have more specific questions feel free to send them to me.

Ted

···

On Thu, Jan 8, 2015 at 7:04 AM, Skip Montanaro
<skip@…503…> wrote:

I’m real naive about this stuff, but I have always wondered why

matplotlib didn’t just use datetime objects, or at least use

timezone-aware datetime objects as an “interchange” format to get the

timezone stuff right.

Time zone handling is a pain in the %}€*

And the definitions keep changing.

So you need a complex DB and library that needs frequent updating.

This is why neither the standard library nor numpy support time zone handling out of the box.

But the datetime object does support a hook to add timezone info.

The numpy datetime64 may implementation may provide a similar hook in the future.

There is the pytz package, which MPL could choose to depend on.

But even that is a bit ugly–e.g. from the pytz docs:

“”“Unfortunately using the tzinfo argument of the standard datetime constructors ‘’does not work’’ with pytz for many timezones.”“”

So my suggestion is that MPL punts, and stick with leaving time zone handling up to the user, I.e only use datetimes that are timezone “naive”. What this means is that MPL would always a assume all datetimes interacting with
each other are in the same time zone (including same DST status).

Anyway, I’m being a bit lazy here, so I may be wrong, but I think the issue at hand is that MPL currently uses a float array to store and manipulate datetimes, and the thought is that it may be better to use
numpy datetime64 arrays – that would give us more consistent precision, and less code to convert to/from various datetime formats.

I’m a bit on the fence about whether it’s time to do it, as datetime64 does have issues with the locale timezone, but as any implementation would want to work with not-just-the-latest numpy anyway, it may make sense to start now.
-Chris

–

Christopher Barker, Ph.D.

Oceanographer

Emergency Response Division

NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@…272…236…

3

Thanks for the responses. If no one minds, I’m take a look at how to best implement datetime64 this weekend and the degree to which it might be useful, then maybe I can start an MEP for it.

I agree with Chris’s sentiment that it’s likely not a bad idea to start on it now, since there will almost certainly be a significant lag in raising the Numpy dependency version anyway, so if it can be implemented in some reasonable way now, we might as well, otherwise it may be some years before we get to it.

Let’s say we want a time zone aware date time converter. The basic goal is to convert some input type (datetime) to the MPL internal type (float days past Jan 0, 0001). We also need to tell MPL how to format the axis (default formatter, locator, limits,
label).

  • The convert() method takes the input type (datetime) and the xunits (or yunits) keyword argument and converts it to the MPL type. The axis input can be used to change the results depending on the plot type (polar plots always require radians for example).
    For the TZ converter, would take the input value (datetime), convert it to the time zone requested by the units input, then convert that to a float using dates.date2num(). Note that the input can be a sequence or a single value so the converter has to handle
    both cases.

  • The axisinfo() method is used to provide the default axis locator and formatter objects when the user creates a plot with this type. The axis input is useful here to handle the result differently for a polar plot. For the TZ converter, this would be
    exactly the same as the web docs - return the default locator and formatter for dates. Most of the time this method can just return standard MPL formatters and locators (for either dates or numbers).

  • The default_units() method provides a default value for the xunits or yunits keyword argument if one isn’t specified. The default in this case might be “UTC”.

Hope that helps some, if you have more specific questions feel free to send them to me.

Ted

···

On Thu, Jan 8, 2015 at 7:04 AM, Skip Montanaro
<skip@…503…> wrote:

I’m real naive about this stuff, but I have always wondered why

matplotlib didn’t just use datetime objects, or at least use

timezone-aware datetime objects as an “interchange” format to get the

timezone stuff right.

Time zone handling is a pain in the %}€*

And the definitions keep changing.

So you need a complex DB and library that needs frequent updating.

This is why neither the standard library nor numpy support time zone handling out of the box.

But the datetime object does support a hook to add timezone info.

The numpy datetime64 may implementation may provide a similar hook in the future.

There is the pytz package, which MPL could choose to depend on.

But even that is a bit ugly–e.g. from the pytz docs:

“”“Unfortunately using the tzinfo argument of the standard datetime constructors ‘’does not work’’ with pytz for many timezones.”“”

So my suggestion is that MPL punts, and stick with leaving time zone handling up to the user, I.e only use datetimes that are timezone “naive”. What this means is that MPL would always a assume all datetimes interacting with
each other are in the same time zone (including same DST status).

Anyway, I’m being a bit lazy here, so I may be wrong, but I think the issue at hand is that MPL currently uses a float array to store and manipulate datetimes, and the thought is that it may be better to use
numpy datetime64 arrays – that would give us more consistent precision, and less code to convert to/from various datetime formats.

I’m a bit on the fence about whether it’s time to do it, as datetime64 does have issues with the locale timezone, but as any implementation would want to work with not-just-the-latest numpy anyway, it may make sense to start now.
-Chris

–

Christopher Barker, Ph.D.

Oceanographer

Emergency Response Division

NOAA/NOS/OR&R (206) 526-6959 voice

7600 Sand Point Way NE (206) 526-6329 fax

Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker@…272…236…

4

Thanks for the responses. If no one minds, I'm take a look at how to
best implement datetime64 this weekend and the degree to which it might
be useful, then maybe I can start an MEP for it.

Paul,

I think everyone will be delighted to have you do this--preferably all the way to a PR.

I agree with Chris's sentiment that it's likely not a bad idea to start
on it now, since there will almost certainly be a significant lag in
raising the Numpy dependency version anyway, so if it can be implemented
in some reasonable way now, we might as well, otherwise it may be some
years before we get to it.

Yes, by all means.

Eric

···

On 2015/01/09 9:54 AM, Paul Ganssle wrote: