False empty strings in csv2rec

Hi everybody,

I have a csv file which I would like to read with pylab.csv2rec . The
file has a header row, values speparated by commas, and a lot of
missings as empty entries. So, a line with missings looks like:
col1,col2,col6 . I tried specifyng the parameter with name
"missing", so it reads:

data = pylab.csv2rec("myfile.csv", missing='')

But it returned: ValueError: empty string for float()

I thought that csv2rec didn't like the empty string '', so I replaced
in my original files all the "," for ",nan,". I also replaced the ","
at the end of files for ",nan". Next I tried:

data = pylab.csv2rec("myfile.csv", missing="nan")

But the same error again:

/usr/lib/python2.6/site-packages/matplotlib/mlab.pyc in csv2rec(fname,
comments, skiprows, checkrows, delimiter, converterd, names, missing,
missingd, use_mrecords)
  2323 if not len(row): continue
  2324 if row[0].startswith(comments): continue
-> 2325 rows.append([func(name, val) for func, name, val in
zip(converters, names, row)])
  2326 rowmasks.append([ismissing(name, val) for name, val in
zip(names, row)])
  2327 fh.close()

/usr/lib/python2.6/site-packages/matplotlib/mlab.pyc in newfunc(name, val)
  2195 return default
  2196 else:
-> 2197 return func(val)
  2198 return newfunc
  2199

ValueError: empty string for float()

Next, I tried a manual check for empty strings, being paranoid:

fin = open("myfile.csv")
f = fin.readlines()
fin.close()
for i in range(len(f)):
  ....: tmp = f[i].split(',')
  ....: for c in tmp:
  ....: if c == '':
  ....: print i

And nothing is printed, so no empty strings. Then I tried
scipy.recfromcsv, and it worked perfectly! OpenOffice also has no
problem with it.

Therefore, it seems that the file is ok, but the csv2rec version from
matplotlib find some inconsistency. Any help? Could it be a bug? (as
the scipy version works flawlessly).

···

--
Sergi Pons Freixes
Ph.D. Student
Marine Technology Unit
Centre Mediterrani d'Investigacions Marines i Ambientals (CMIMA-CSIC)
Pg. Marítim Barceloneta, 37-49
E-08003 Barcelona (Spain)
Ph. +34 93 230 95 00 (ext. 1510)
Fax. +34 93 230 95 55
spons@...3237...
http://www.utm.csic.es

Can you post the file, here if it is small, or somewhere like drop.io
if it is large?

Thanks,
JDH

···

On Thu, Aug 12, 2010 at 8:10 PM, Sergi Pons Freixes <spons@...3237...> wrote:

Hi everybody,

I have a csv file which I would like to read with pylab.csv2rec . The
file has a header row, values speparated by commas, and a lot of
missings as empty entries. So, a line with missings looks like:
col1,col2,col6 . I tried specifyng the parameter with name
"missing", so it reads: