Plotting NOAA grib2 data in basemap

Hi im new to Python and basemap i am trying to follow the script below that
was posted in another thread here
<http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-data-td19588.html>
but im getting this error

alec@...4479...:~$ python Desktop/test.py
/usr/lib/pymodules/python2.7/mpl_toolkits/__init__.py:2: UserWarning: Module
dap was already imported from None, but /usr/lib/python2.7/dist-packages is
being added to sys.path
  __import__('pkg_resources').declare_namespace(__name__)
a
fatal: environment variable not set
b
['VGRD_P0_L1_GLL0', 'WVDIR_P0_L1_GLL0', 'lon_0', 'PERSW_P0_L1_GLL0',
'WIND_P0_L1_GLL0', 'PERPW_P0_L1_GLL0', 'WDIR_P0_L1_GLL0', 'forecast_time0',
'DIRPW_P0_L1_GLL0', 'WVPER_P0_L1_GLL0', 'DIRSW_P0_L1_GLL0',
'HTSGW_P0_L1_GLL0', 'UGRD_P0_L1_GLL0', 'lat_0']
Traceback (most recent call last):
  File "Desktop/test.py", line 14, in <module>
    datavar = f.variables['WWSWHGT_P0_L1_GLL0']
KeyError: 'WWSWHGT_P0_L1_GLL0'

The script im trying to run below

import Nio
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np
f = Nio.open_file('akw.t00z.grib.grib2')
print f.variables.keys()
lons = f.variables['lon_0'][:]
# flip latitudes so data goes S-->N
lats = f.variables['lat_0'][::-1]
times = f.variables['forecast_time0'][:]
datavar = f.variables['WWSWHGT_P0_L1_GLL0']
ntime = 10
data = datavar[ntime,::-1]
print f.variables['WWSWHGT_P0_L1_GLL0']
print data.min(), data.max()
m = Basemap(projection='cyl',llcrnrlat=lats[0],llcrnrlon=lons[0],\
            urcrnrlat=lats[-1],urcrnrlon=lons[-1],resolution='l')
x, y = m(*np.meshgrid(lons, lats))
levels = np.arange(0,9.1,0.5)
m.contourf(x,y,data,levels)
m.drawcoastlines()
m.fillcontinents()
m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0])
m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1])
m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0])
m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1])
plt.title(datavar.long_name+' %s hr fcst'%(times[ntime]),fontsize=12)
plt.colorbar(orientation='horizontal',shrink=0.9,format="%g")
plt.show()

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

As the error message says, the problem is on Line 14:

print f.variables[‘WWSWHGT_P0_L1_GLL0’]

a KeyError means that you tried to access an element that is not in a dictionary. In this case “f.variables” is the dictionary and “‘WWSWHGT_P0_L1_GLL0’” is the element.

Did your data and script come of the same place? You can’t just throw any basemap script at any grid file.

···

On Thu, Jan 9, 2014 at 4:52 AM, Rolling Six <surfershort@…32…> wrote:

Hi im new to Python and basemap i am trying to follow the script below that

was posted in another thread here

<http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-data-td19588.html>

but im getting this error

alec@…4479…:~$ python Desktop/test.py

/usr/lib/pymodules/python2.7/mpl_toolkits/init.py:2: UserWarning: Module

dap was already imported from None, but /usr/lib/python2.7/dist-packages is

being added to sys.path

import(‘pkg_resources’).declare_namespace(name)

a

fatal: environment variable not set

b

[‘VGRD_P0_L1_GLL0’, ‘WVDIR_P0_L1_GLL0’, ‘lon_0’, ‘PERSW_P0_L1_GLL0’,

‘WIND_P0_L1_GLL0’, ‘PERPW_P0_L1_GLL0’, ‘WDIR_P0_L1_GLL0’, ‘forecast_time0’,

‘DIRPW_P0_L1_GLL0’, ‘WVPER_P0_L1_GLL0’, ‘DIRSW_P0_L1_GLL0’,

‘HTSGW_P0_L1_GLL0’, ‘UGRD_P0_L1_GLL0’, ‘lat_0’]

Traceback (most recent call last):

File “Desktop/test.py”, line 14, in

datavar = f.variables['WWSWHGT_P0_L1_GLL0']

KeyError: ‘WWSWHGT_P0_L1_GLL0’

The script im trying to run below

import Nio

from mpl_toolkits.basemap import Basemap

import matplotlib.pyplot as plt

import numpy as np

f = Nio.open_file(‘akw.t00z.grib.grib2’)

print f.variables.keys()

lons = f.variables[‘lon_0’][:]

flip latitudes so data goes S–>N

lats = f.variables[‘lat_0’][::-1]

times = f.variables[‘forecast_time0’][:]

datavar = f.variables[‘WWSWHGT_P0_L1_GLL0’]

ntime = 10

data = datavar[ntime,::-1]

print f.variables[‘WWSWHGT_P0_L1_GLL0’]

print data.min(), data.max()

m = Basemap(projection=‘cyl’,llcrnrlat=lats[0],llcrnrlon=lons[0],\

        urcrnrlat=lats[-1],urcrnrlon=lons[-1],resolution='l')

x, y = m(*np.meshgrid(lons, lats))

levels = np.arange(0,9.1,0.5)

m.contourf(x,y,data,levels)

m.drawcoastlines()

m.fillcontinents()

m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0])

m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1])

m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0])

m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1])

plt.title(datavar.long_name+’ %s hr fcst’%(times[ntime]),fontsize=12)

plt.colorbar(orientation=‘horizontal’,shrink=0.9,format="%g")

plt.show()

View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698.html

Sent from the matplotlib - users mailing list archive at Nabble.com.


CenturyLink Cloud: The Leader in Enterprise Cloud Services.

Learn Why More Businesses Are Choosing CenturyLink Cloud For

Critical Workloads, Development Environments & Everything In Between.

Get a Quote or Start a Free Trial Today.

http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


Matplotlib-users mailing list

Matplotlib-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Hi Paul,

Thanks for your reply, I managed to fix it after I realised the mistake I
was making.

I've currently got a new problem. If you look at the image below, there's a
lot of white showing up around the coasts which ideally I'd like to remove.
The map is drawn using basemap, is there a function/feature that will fill
this in. Or is it a probem with the grib files that I'm using?

<http://matplotlib.1069221.n5.nabble.com/file/n42701/figure_1.png>

If possible, I'd like it to look like the image below. If it is a problem
with our data, is there a way that we can draw a layer of colour on the
bottom to get rid of the white?

<http://matplotlib.1069221.n5.nabble.com/file/n42701/figure_1.png>

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42701.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

I think you posted the same image in both cases. Without seeing the problematic image, I can only guess that it’s caused by the resolution of your data.

···

On Thu, Jan 9, 2014 at 7:58 AM, A Short <surfershort@…32…> wrote:

Hi Paul,

Thanks for your reply, I managed to fix it after I realised the mistake I

was making.

I’ve currently got a new problem. If you look at the image below, there’s a

lot of white showing up around the coasts which ideally I’d like to remove.

The map is drawn using basemap, is there a function/feature that will fill

this in. Or is it a probem with the grib files that I’m using?

<http://matplotlib.1069221.n5.nabble.com/file/n42701/figure_1.png>

If possible, I’d like it to look like the image below. If it is a problem

with our data, is there a way that we can draw a layer of colour on the

bottom to get rid of the white?

<http://matplotlib.1069221.n5.nabble.com/file/n42701/figure_1.png>

View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42701.html
Sent from the matplotlib - users mailing list archive at Nabble.com.


CenturyLink Cloud: The Leader in Enterprise Cloud Services.

Learn Why More Businesses Are Choosing CenturyLink Cloud For

Critical Workloads, Development Environments & Everything In Between.

Get a Quote or Start a Free Trial Today.

http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


Matplotlib-users mailing list

Matplotlib-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-users

ok the file im using is this multi_2.glo_30m.t06z.grib2 from here
<ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/wave.20140109/> im
not sure its the right file to get wave heights of the North East Atlantic
so im trying different ones.

as you can see above in the top image there is some data missing (white
boxes) near the coast im wondering if its possible to either zoom into the
map slightly so the data runs underneath the land..?

Thanks
Alec

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42705.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

What I’m saying is that your top image and bottom image are identical and I don’t see any white boxes in either. What is the resolution of the grid?

-paul

···

On Thu, Jan 9, 2014 at 11:59 AM, A Short <surfershort@…32…> wrote:

ok the file im using is this multi_2.glo_30m.t06z.grib2 from here

ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/wave.20140109/ im

not sure its the right file to get wave heights of the North East Atlantic

so im trying different ones.

as you can see above in the top image there is some data missing (white

boxes) near the coast im wondering if its possible to either zoom into the

map slightly so the data runs underneath the land…?

Thanks

Alec

View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42705.html
Sent from the matplotlib - users mailing list archive at Nabble.com.


CenturyLink Cloud: The Leader in Enterprise Cloud Services.

Learn Why More Businesses Are Choosing CenturyLink Cloud For

Critical Workloads, Development Environments & Everything In Between.

Get a Quote or Start a Free Trial Today.

http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


Matplotlib-users mailing list

Matplotlib-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Thats strange they look different on this browser. Hopefully the one below
youll see what i mean

Thanks

<http://matplotlib.1069221.n5.nabble.com/file/n42708/figure_1.png>

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42708.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

How does it look if you remove the calls to m.drawcoastlines() and m.fillcontinents()?

···

On Thu, Jan 9, 2014 at 1:05 PM, A Short <surfershort@…32…> wrote:

Thats strange they look different on this browser. Hopefully the one below

youll see what i mean

Thanks

<http://matplotlib.1069221.n5.nabble.com/file/n42708/figure_1.png>

View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42708.html
Sent from the matplotlib - users mailing list archive at Nabble.com.


CenturyLink Cloud: The Leader in Enterprise Cloud Services.

Learn Why More Businesses Are Choosing CenturyLink Cloud For

Critical Workloads, Development Environments & Everything In Between.

Get a Quote or Start a Free Trial Today.

http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


Matplotlib-users mailing list

Matplotlib-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Ive also changed the grib file from multi_2.glo_30m.t06z.grib2 to
nww3.t12z.grib.grib2 i still cant figure out which is the right file for the
North East Atlantic neither...i wonder if my data source is wrong..?

<http://matplotlib.1069221.n5.nabble.com/file/n42710/figure_2.png>

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42710.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Looks like it’s just a coarse resolution to me. Try showing the data as an image with no iterpolation.

···

On Thu, Jan 9, 2014 at 3:26 PM, A Short <surfershort@…32…> wrote:

Ive also changed the grib file from multi_2.glo_30m.t06z.grib2 to

nww3.t12z.grib.grib2 i still cant figure out which is the right file for the

North East Atlantic neither…i wonder if my data source is wrong…?

<http://matplotlib.1069221.n5.nabble.com/file/n42710/figure_2.png>

View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42710.html
Sent from the matplotlib - users mailing list archive at Nabble.com.


CenturyLink Cloud: The Leader in Enterprise Cloud Services.

Learn Why More Businesses Are Choosing CenturyLink Cloud For

Critical Workloads, Development Environments & Everything In Between.

Get a Quote or Start a Free Trial Today.

http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


Matplotlib-users mailing list

Matplotlib-users@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-users

Thanks Paul i added m.imshow(data, origin='lower', interpolation='none')

Its made a little improvement but something still seems to be not right

<http://matplotlib.1069221.n5.nabble.com/file/n42712/figure_3.png>

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42712.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Hi - Ive now improved my code and confirmed the use of the right grib file
but i cant for the life of me figure out the missing data near the
coastline..? Could anyone help?

`import Nio
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np

f = Nio.open_file('nww3.t12z.grib(2).grib2')
lons = f.variables['lon_0'][:]
lats = f.variables['lat_0'][::-1] # flip latitudes so data goes S-->N
times = f.variables['forecast_time0'][:]
ntime = 5
data = f.variables['HTSGW_P0_L1_GLL0'][ntime,::-1]

fig = plt.figure(figsize=(16,16))
m = Basemap(llcrnrlon=-35.,llcrnrlat=42.,urcrnrlon=5.,urcrnrlat=65.,
            projection='lcc',lat_1=10.,lat_2=15.,lon_0=10.,
            resolution ='h',area_thresh=1000.)

x, y = m(*np.meshgrid(lons, lats))
m.fillcontinents(color='#477519')
m.drawcoastlines(linewidth=0.5, color='k', antialiased=1, ax=None,
zorder=None )

m.contourf(x, y, data, np.arange(0,9.9,0.1))
plt.show() `

Resulting plot is here
<http://matplotlib.1069221.n5.nabble.com/file/n42790/figure_7.png>

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42790.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

Hi - Ive now improved my code and confirmed the use of the right grib file
but i cant for the life of me figure out the missing data near the
coastline..? Could anyone help?

The present contouring algorithm works with rectangular blocks, and if any corner has missing data, nothing is filled for that block.

Eric

···

On 2014/01/28 10:01 AM, A Short wrote:

`import Nio
from mpl_toolkits.basemap import Basemap
import matplotlib.pyplot as plt
import numpy as np

f = Nio.open_file('nww3.t12z.grib(2).grib2')
lons = f.variables['lon_0'][:]
lats = f.variables['lat_0'][::-1] # flip latitudes so data goes S-->N
times = f.variables['forecast_time0'][:]
ntime = 5
data = f.variables['HTSGW_P0_L1_GLL0'][ntime,::-1]

fig = plt.figure(figsize=(16,16))
m = Basemap(llcrnrlon=-35.,llcrnrlat=42.,urcrnrlon=5.,urcrnrlat=65.,
             projection='lcc',lat_1=10.,lat_2=15.,lon_0=10.,
             resolution ='h',area_thresh=1000.)

x, y = m(*np.meshgrid(lons, lats))
m.fillcontinents(color='#477519')
m.drawcoastlines(linewidth=0.5, color='k', antialiased=1, ax=None,
zorder=None )

m.contourf(x, y, data, np.arange(0,9.9,0.1))
plt.show() `

Resulting plot is here
<http://matplotlib.1069221.n5.nabble.com/file/n42790/figure_7.png>

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42790.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

This will improve shortly, cutting off the corners of some of those empty
blocks. I am currently testing the new algorithm for this prior to
submitting it for others' approval.

Ian

···

On 29 January 2014 03:21, Eric Firing <efiring@...202...> wrote:

On 2014/01/28 10:01 AM, A Short wrote:
> Hi - Ive now improved my code and confirmed the use of the right grib
file
> but i cant for the life of me figure out the missing data near the
> coastline..? Could anyone help?

The present contouring algorithm works with rectangular blocks, and if
any corner has missing data, nothing is filled for that block.

Is there any work around so it looks like the below image?

Could anyone confirm that this would be the correct grib file for The North
Atlantic..?
ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/wave.20140129/nww3.t06z.grib.grib2

Thanks for all the help

<http://matplotlib.1069221.n5.nabble.com/file/n42798/figure_1.png>

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42798.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

     > Hi - Ive now improved my code and confirmed the use of the right
    grib file
     > but i cant for the life of me figure out the missing data near the
     > coastline..? Could anyone help?

    The present contouring algorithm works with rectangular blocks, and if
    any corner has missing data, nothing is filled for that block.

This will improve shortly, cutting off the corners of some of those
empty blocks. I am currently testing the new algorithm for this prior to
submitting it for others' approval.

Ian,

I'm glad to hear that! One possibility would be to use a temporary rcParam (temporary in that it might be phased out after a couple releases) to allow switching between the two algorithms. This would make it much easier to test, and it would also allow a transition during which people could reproduce results obtained with earlier mpl. It would also be a safety measure, in case someone hits a corner case which the new algorithm doesn't handle but the old one does--not that I'm expecting such cases to arise.

Eric

···

On 2014/01/28 11:40 PM, Ian Thomas wrote:

On 29 January 2014 03:21, Eric Firing <efiring@…202… > <mailto:efiring@…202…>> wrote:
    On 2014/01/28 10:01 AM, A Short wrote:

Ian

Is there any work around so it looks like the below image?

It looks like with any reasonable contouring algorithm, this would require interpolating into land regions, contouring, and then plotting the land on top. The key is the interpolation, not the plotting. The example you show might have been interpolated to a finer grid everywhere, not just in the missing value regions.

I can't comment on the file itself.

Eric

···

On 2014/01/29 5:41 AM, A Short wrote:

Could anyone confirm that this would be the correct grib file for The North
Atlantic..?
ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/wave.20140129/nww3.t06z.grib.grib2

Thanks for all the help

<http://matplotlib.1069221.n5.nabble.com/file/n42798/figure_1.png>

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42798.html
Sent from the matplotlib - users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users

IMHO that's the most straightforward approach.
He can use masked array for empty blocks (if contour data doesn't
already contain the holes as masked array) and apply inpainting, then
draw the land.
For more details about inpainting: http://stackoverflow.com/a/17125125/992005

···

On Wed, Jan 29, 2014 at 6:22 PM, Eric Firing <efiring@...202...> wrote:

On 2014/01/29 5:41 AM, A Short wrote:

Is there any work around so it looks like the below image?

It looks like with any reasonable contouring algorithm, this would
require interpolating into land regions, contouring, and then plotting
the land on top. The key is the interpolation, not the plotting. The
example you show might have been interpolated to a finer grid
everywhere, not just in the missing value regions.

Thanks Eric and Kio - ive been trying for a couple of days and i just cant
seem to get my head around the interpolation or masked arrays. Would it be
possible for anyone to give me a few pointers on where to start editing the
script.

Ive tried digesting these tutorials the first one seems to be relevant to
what im trying to achieve by using a Laplace filter but im still having
trouble installing all the modules to be able to play with it more

http://www.trondkristiansen.com/?page_id=846
<http://www.trondkristiansen.com/?page_id=846>

http://www.geophysique.be/2010/05/05/matplotlib-basemap-tutorial-part-03-masked-arrays-zoom/
<http://www.geophysique.be/2010/05/05/matplotlib-basemap-tutorial-part-03-masked-arrays-zoom/>

And reading all of the toolkit doc....several times!

Thanks again

···

--
View this message in context: http://matplotlib.1069221.n5.nabble.com/Plotting-NOAA-grib2-data-in-basemap-tp42698p42805.html
Sent from the matplotlib - users mailing list archive at Nabble.com.