mlab.load and non-numeric characters

Hi, I want to use mlab.load to load in some data:

   1) 2004/02/27 21:51:00           1     2553.51     2553.51    -99.0000N      3.217
   2) 2004/02/27 22:01:00           2     2553.47     2553.47    -99.0000N      3.217
   3) 2004/02/27 22:10:59           3     2553.45     2553.45    -99.0000N      3.218
   4) 2004/02/27 22:20:59           4     2553.46     2553.46    -99.0000N      3.223

unfortunately missing values are given as -99.000N, and these cause the following error:

In [98]: mlab.load(site_file,skiprows=29,usecols=[4])

···

ValueError Traceback (most recent call last)

/Users/evan/python/tools/fig_NEA_seas_paper_RAPID.py in ()
----> 1
2
3
4
5

/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib/mlab.pyc in load(fname, comments, delimiter, converters, skiprows, usecols, unpack, dtype)
1458 if usecols is not None:
1459 vals = splitfunc(line)
-> 1460 row = [converterseqj for j in usecols]
1461 else:
1462 row = [converterseqj

ValueError: invalid literal for float(): -99.00N

Is there any way around this, apart from editing all the data files to remove every ‘N’?

Many thanks, Evan

Make use of the converters argument:

mlab.load(…, converters={5:fix_func})

Where fix_func is a function that will convert the -99.0000N values to what you’re looking for.

Ryan

···

On Sat, May 9, 2009 at 8:35 PM, Evan Mason <evanmason@…120…287…> wrote:

Hi, I want to use mlab.load to load in some data:

   1) 2004/02/27 21:51:00           1     2553.51     2553.51    -99.0000N      3.217
   2) 2004/02/27 22:01:00           2     2553.47     2553.47    -99.0000N      3.217



   3) 2004/02/27 22:10:59           3     2553.45     2553.45    -99.0000N      3.218
   4) 2004/02/27 22:20:59           4     2553.46     2553.46    -99.0000N      3.223

unfortunately missing values are given as -99.000N, and these cause the following error:

In [98]: mlab.load(site_file,skiprows=29,usecols=[4])

ValueError Traceback (most recent call last)

/Users/evan/python/tools/fig_NEA_seas_paper_RAPID.py in ()
----> 1
2
3
4
5

/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib/mlab.pyc in load(fname, comments, delimiter, converters, skiprows, usecols, unpack, dtype)

1458 if usecols is not None:
1459 vals = splitfunc(line)
-> 1460 row = [converterseqj for j in usecols]
1461 else:
1462 row = [converterseqj

ValueError: invalid literal for float(): -99.00N

Is there any way around this, apart from editing all the data files to remove every ‘N’?


Ryan May
Graduate Research Assistant

School of Meteorology
University of Oklahoma

Thanks for that:

def ifmissing(x):
try: return float(x)
except: return np.nan

works just fine.

-Evan

···

On Sun, May 10, 2009 at 6:59 AM, Ryan May <rmay31@…83…287…> wrote:

On Sat, May 9, 2009 at 8:35 PM, Evan Mason <evanmason@…287…> wrote:

Hi, I want to use mlab.load to load in some data:

   1) 2004/02/27 21:51:00           1     2553.51     2553.51    -99.0000N      3.217
   2) 2004/02/27 22:01:00           2     2553.47     2553.47    -99.0000N      3.217




   3) 2004/02/27 22:10:59           3     2553.45     2553.45    -99.0000N      3.218
   4) 2004/02/27 22:20:59           4     2553.46     2553.46    -99.0000N      3.223

unfortunately missing values are given as -99.000N, and these cause the following error:

In [98]: mlab.load(site_file,skiprows=29,usecols=[4])

ValueError Traceback (most recent call last)

/Users/evan/python/tools/fig_NEA_seas_paper_RAPID.py in ()
----> 1
2
3
4
5

/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib/mlab.pyc in load(fname, comments, delimiter, converters, skiprows, usecols, unpack, dtype)

1458 if usecols is not None:
1459 vals = splitfunc(line)
-> 1460 row = [converterseqj for j in usecols]
1461 else:
1462 row = [converterseqj

ValueError: invalid literal for float(): -99.00N

Is there any way around this, apart from editing all the data files to remove every ‘N’?

Make use of the converters argument:

mlab.load(…, converters={5:fix_func})

Where fix_func is a function that will convert the -99.0000N values to what you’re looking for.

Ryan

Ryan May
Graduate Research Assistant

School of Meteorology
University of Oklahoma