Dates interpretation on csv2rec

Community matplotlib-users
1

Dear all,

I have a CSV file with the first column with timestamps:

0:00:00 01/01/2007, 0.000, 10, 0.000, 10,
0.000, 10:
00:00:00 02/01/2007, 0.000, 10, 0.000, 10,
0.000, 10
00:00:00 03/01/2007, 0.000, 10, 0.000, 10,
0.000, 10
...
00:00:00 29/12/2009, 0.000, 10, 0.000, 10,
0.000, 10
00:00:00 30/12/2009, 0.000, 10, 0.000, 10,
0.000, 10
00:00:00 31/12/2009, 0.000, 10, 0.000, 10,
0.000, 10

As you can see, the format is hour:minute:second (nor relevant, always
00) day/month/year.

When loaded with mlab.csv2rec, it automatically detects them as dates
and creates datetime objects, but it is not consistent with the
interpretation of the dates. When the day is < 13, it interprets it as
month/day/year. It's loaded correctly otherwise (as day/month/year). I
could pre-process the files and change the dates format, but imho it's
not very elegant. Any suggestion about how to address this issue?

Regards,
Sergi

2

Dear all,

I have a CSV file with the first column with timestamps:

0:00:00 01/01/2007, 0.000, 10, 0.000, 10,
0.000, 10:
00:00:00 02/01/2007, 0.000, 10, 0.000, 10,
0.000, 10
00:00:00 03/01/2007, 0.000, 10, 0.000, 10,
0.000, 10
...
00:00:00 29/12/2009, 0.000, 10, 0.000, 10,
0.000, 10
00:00:00 30/12/2009, 0.000, 10, 0.000, 10,
0.000, 10
00:00:00 31/12/2009, 0.000, 10, 0.000, 10,
0.000, 10

As you can see, the format is hour:minute:second (nor relevant, always
00) day/month/year.

When loaded with mlab.csv2rec, it automatically detects them as dates
and creates datetime objects, but it is not consistent with the
interpretation of the dates. When the day is < 13, it interprets it as
month/day/year. It's loaded correctly otherwise (as day/month/year). I
could pre-process the files and change the dates format, but imho it's
not very elegant. Any suggestion about how to address this issue?

Regards,
Sergi

You could use numpy.genfromtxt together with its `converters` parameter:

    converters : variable, optional
        The set of functions that convert the data of a column to a value.
        The converters can also be used to provide a default value
        for missing data: ``converters = {3: lambda s: float(s or 0)}``.

Cheers,
A.

3

I finally used:

def mkdate(text):
    return(dt.datetime.strptime(text.decode('utf8'),\
            '%H:%M:%S %d/%m/%Y'))

and

data = np.genfromtxt(\
            os.path.join(sys.argv[1],f),\
            delimiter=',',\
            skip_header=4,\
            usecols = (0, 1, 2),\
            dtype=(object, float, float),\
            converters={0: mkdate},\
            autostrip=True,\
            )

Thank you!

ยทยทยท

On Fri, Jun 29, 2012 at 12:02 PM, Andreas Hilboll <lists@...3067...> wrote:

You could use numpy.genfromtxt together with its `converters` parameter:

    converters : variable, optional
        The set of functions that convert the data of a column to a value.
        The converters can also be used to provide a default value
        for missing data: ``converters = {3: lambda s: float(s or 0)}``.