# Masking data problems and fill test

Hi list

I searched the list and google, but couldn't find a way to solve my pbm.

I have data stored in a list (from an sql query) , with these "columns":
x = time serie in hours
y = some level value

There are some missing values : eg between 08:33 and 08:40.
Here is my code :
sqla="SELECT * FROM import_parcelle a WHERE dat_loc='" + date_traite + "' AND code_uc='" + code_uc + "' ORDER BY a.heu_loc ;"
resa=db.query(sqla)
data=resa.dictresult() #x = time serie
x= [ datetime.datetime(*time.strptime(a["dat_loc"]+" "+a["heu_loc"],'%Y-%m-%d %H:%M:%S')[0:6] ) for a in data]
#y5 = tank level
y5= [float(a["niv_cuv"]) for a in data] # --> extraction de la colonne y1
figure()
# the plot command with no mask
plot_date(x,y5,color='b',linestyle='None',marker='',xdate=True,ydate=False)
#plot_date(x,ym5,color='r',linestyle='-',xdate=True,ydate=False)
#the filling under the curve
xs, ys = poly_between(x, 0, y5)
fill(xs,ys)

Here is the result:
http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.png
As you see, I commented the lines with the mask, because it did not change anything

To see what I am looking for, here is the result under R (a statistical tool)
http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.jpg
As you see, between 08:48 and 08:50 (for example), there is a gap, showing we have no data for this interval.

As I want to automatically draw this graph for different set of data, I can't look each set in detail.
I tried to folow the example masked_demo.py, but I could not adapt it to my case....

Any hint ?

Michael

You have run into a bug in the combination of poly_between and fill--maybe only the former, which not taking masked arrays into account. I have not looked at it enough to know whether it will be easy or hard to fix, but it certainly should be fixed. I can't look at it more right now, unfortunately.

Eric

Micha�l Douchin wrote:

···

Hi list

I searched the list and google, but couldn't find a way to solve my pbm.

I have data stored in a list (from an sql query) , with these "columns":
x = time serie in hours
y = some level value

There are some missing values : eg between 08:33 and 08:40.
Here is my code :
sqla="SELECT * FROM import_parcelle a WHERE dat_loc='" + date_traite + "' AND code_uc='" + code_uc + "' ORDER BY a.heu_loc ;"
resa=db.query(sqla)
data=resa.dictresult() #x = time serie
x= [ datetime.datetime(*time.strptime(a["dat_loc"]+" "+a["heu_loc"],'%Y-%m-%d %H:%M:%S')[0:6] ) for a in data]
#y5 = tank level
y5= [float(a["niv_cuv"]) for a in data] # --> extraction de la colonne y1
figure()
# the plot command with no mask
plot_date(x,y5,color='b',linestyle='None',marker='',xdate=True,ydate=False)
#plot_date(x,ym5,color='r',linestyle='-',xdate=True,ydate=False)
#the filling under the curve
xs, ys = poly_between(x, 0, y5)
fill(xs,ys)

Here is the result:
http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.png
As you see, I commented the lines with the mask, because it did not change anything

To see what I am looking for, here is the result under R (a statistical tool)
http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.jpg
As you see, between 08:48 and 08:50 (for example), there is a gap, showing we have no data for this interval.

As I want to automatically draw this graph for different set of data, I can't look each set in detail.
I tried to folow the example masked_demo.py, but I could not adapt it to my case....

Any hint ?

Michael

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Since I am a beginner in matplotlib, I don't want to blame the soft first. I mean it could be a bug, but I have the feeling my code for masking the missing data is useless.

My real question is : How to mask data that do not exist ??? In the masked_demo.py example, it is show how to mask data from a set like "My data are
x=[1,2,3,4,5,6,7,8,9,10]
y = sin(x) or anyelse continuous fonction and I want to mask 1,2,3 so I write : ym = ma.masked_where(y <3, y)

But in my case, it is more like :
x = [1,2,3,4,7,8,9,15,16,17]
y = discrete values (no continuous fonction !) like [5,2,20,17,3,12,9,18,18,9]
What I want to do with these data :
1/plot graph with no line connecting the point (x,y) = (4,17) and the point (7,3), etc.
2/fill the space under the plot but do not fill the space where there is no data (eg between the same 2 points).

The 1st thing seems to be easy = just follow the example masked_demo
But in my case, how can I mask data that don't exist ? I can't write ym = ma.masked_where(y <3, y)

Hum, I hope I was clearer....
Michael

Eric Firing a �crit :

···

You have run into a bug in the combination of poly_between and fill--maybe only the former, which not taking masked arrays into account. I have not looked at it enough to know whether it will be easy or hard to fix, but it certainly should be fixed. I can't look at it more right now, unfortunately. > > Eric > > Micha�l Douchin wrote:

Hi list

I searched the list and google, but couldn't find a way to solve my pbm.

I have data stored in a list (from an sql query) , with these "columns":
x = time serie in hours
y = some level value

There are some missing values : eg between 08:33 and 08:40.
Here is my code :
sqla="SELECT * FROM import_parcelle a WHERE dat_loc='" + date_traite + "' AND code_uc='" + code_uc + "' ORDER BY a.heu_loc ;"
resa=db.query(sqla)
data=resa.dictresult() #x = time serie
x= [ datetime.datetime(*time.strptime(a["dat_loc"]+" "+a["heu_loc"],'%Y-%m-%d %H:%M:%S')[0:6] ) for a in data]
#y5 = tank level
y5= [float(a["niv_cuv"]) for a in data] # --> extraction de la colonne y1
figure()
# the plot command with no mask
plot_date(x,y5,color='b',linestyle='None',marker='',xdate=True,ydate=False)

#plot_date(x,ym5,color='r',linestyle='-',xdate=True,ydate=False)
#the filling under the curve
xs, ys = poly_between(x, 0, y5)
fill(xs,ys)

Here is the result:
http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.png
As you see, I commented the lines with the mask, because it did not change anything

To see what I am looking for, here is the result under R (a statistical tool)
http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.jpg
As you see, between 08:48 and 08:50 (for example), there is a gap, showing we have no data for this interval.

As I want to automatically draw this graph for different set of data, I can't look each set in detail.
I tried to folow the example masked_demo.py, but I could not adapt it to my case....

Any hint ?

Michael

-------------------------------------------------------------------------

This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Michael,
Give scikits.timeseries a try. It's a package designed to handle time series,
with or without missing data or dates, for which matplotlib functions are
also available.
http://scipy.org/scipy/scikits/wiki/TimeSeries
The problem you're refering to (missing dates) is trivial with this package.
Let me know if you have any problem with it. Please note that plotting with
hourly frequency is still experimental (and we need some feedback).
HIH
P.

Thanks Pierre,
I will try this package and come back here to give a feedback

Michael

Pierre GM a �crit :

···

Michael,
Give scikits.timeseries a try. It's a package designed to handle time series, with or without missing data or dates, for which matplotlib functions are also available. http://scipy.org/scipy/scikits/wiki/TimeSeries
The problem you're refering to (missing dates) is trivial with this package.
Let me know if you have any problem with it. Please note that plotting with hourly frequency is still experimental (and we need some feedback). HIH
P.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users