MODIS data and true-color plotting

Hello groups,

I have two questions about working with MODIS data.

1-) Is there any light Pythonic HDF-EOS wrapper to handle HDF-EOS data other than PyNIO [http://www.pyngl.ucar.edu/Nio.shtml] Although, I have managed to install that package from its source, it took me many hours to figure out all the installation quirks. Something simpler to build and mainly for HDFEOS data??

2-) Another similar question: Has anybody attempted to create true-color MODIS images (like the ones shown at [http://rapidfire.sci.gsfc.nasa.gov/realtime/]) in Python? So far, I have seen one clear tutorial [ftp://ftp.ssec.wisc.edu/pub/IMAPP/MODIS/TrueColor/] to create natural color images, but uses ms2gt [http://nsidc.org/data/modis/ms2gt/], NDVI and IDL. Except the reflectance correction via NDVI, ms2gt and IDL parts seem to be implemented in Python.

Till now, I have some progress combining GOES imagery with aircraft data. My next task is to combine MODIS data with aircraft and radar data. I would be happy to get some guidance and code support if there is any previous work been done using Python.

Thanks.

Hello groups,

I have two questions about working with MODIS data.

    1-) Is there any light Pythonic HDF-EOS wrapper to handle

HDF-EOS data other than PyNIO [http://www.pyngl.ucar.edu/Nio.shtml ]
Although, I have managed to install that package from its
source, it took me many hours to figure out all the installation
quirks. Something simpler to build and mainly for HDFEOS data??

Gökhan:  netcdf4-python can read hdf5-eos files, and even hdf4-eos

files if the netcdf C lib is built with hdf4 support.

-Jeff
···

http://p.sf.net/sfu/rsa-sfdev2dev1Matplotlib-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/matplotlib-users

I can’t build netcdf4 C libraries with HDF4 support.

[gsever@…3858… hdf-4.2.6]$ ./configure --prefix=/usr/local --includedir=/usr/local/ --with-zlib=/usr/local --with-jpeg=/usr/local --disable-fortran --with-szip=/usr/local/ CFLAGS=-fPIC

I can’t build HDF4 without disabling the fortran support. Then trying to build netcdf4 with HDF4:

[gsever@…2980… netcdf-4.1.3]$ ./configure --prefix=/usr/local/netcdf4 --enable-netcdf-4 --with-hdf5=/usr/local/ --enable-shared --enable-hdf4 --with-hdf4=/usr/local/

Fails:

Making all in ncgen3

make[2]: Entering directory `/home/gsever/Downloads/netcdf-4.1.3/ncgen3’

/bin/sh …/libtool --tag=CC --mode=link gcc -g -O2 -o ncgen3 main.o load.o escapes.o getfill.o init.o genlib.o ncgentab.o …/liblib/libnetcdf.la -lhdf5_hl -lhdf5 -lm -lz -lcurl

libtool: link: gcc -g -O2 -o .libs/ncgen3 main.o load.o escapes.o getfill.o init.o genlib.o ncgentab.o …/liblib/.libs/libnetcdf.so -lhdf5_hl -lhdf5 -lm -lz -lcurl -Wl,-rpath -Wl,/usr/local/netcdf4/lib

…/liblib/.libs/libnetcdf.so: undefined reference to `SDstart’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDend’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDattrinfo’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDfileinfo’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDreadattr’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDselect’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDreaddata’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDdiminfo’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDgetfillvalue’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDgetinfo’

…/liblib/.libs/libnetcdf.so: undefined reference to `SDgetdimid’

collect2: ld returned 1 exit status

make[2]: *** [ncgen3] Error 1

make[2]: Leaving directory `/home/gsever/Downloads/netcdf-4.1.3/ncgen3’

make[1]: *** [all-recursive] Error 1

make[1]: Leaving directory `/home/gsever/Downloads/netcdf-4.1.3’

make: *** [all] Error 2

When I remove hdf4 part from config, it builds successfully. Any ideas?

···

On Sun, Nov 13, 2011 at 12:57 PM, Jeff Whitaker <jswhit@…3857…> wrote:

Gökhan:  netcdf4-python can read hdf5-eos files, and even hdf4-eos

files if the netcdf C lib is built with hdf4 support.

-Jeff

I think that paths needed to be passed to CPP/LDFLAGS like this:

CPPFLAGS=-I/usr/include/hdf LDFLAGS=-L/usr/lib ./configure --enable-hdf4 && make

then also package is dependent on latest hdf5 to be build (1.8.7), so
installing it globally would break possible dependencies in any
packaging system, and lastly it's just pain to build it if it's worth

···

On Sun, Nov 13, 2011 at 10:12 PM, Gökhan Sever <gokhansever@...287...> wrote:

When I remove hdf4 part from config, it builds successfully. Any ideas?

This one still fails with the same error. I will make a clean Fedora 16 installation in the near future, then I will test these on the new system. Firstly, hoping that F16 will provide these packages already built :slight_smile: Those -fPIC errors were killing me while building PyNIO…

···

On Sun, Nov 13, 2011 at 4:37 PM, klo uo <klonuo@…287…> wrote:

I think that paths needed to be passed to CPP/LDFLAGS like this:

CPPFLAGS=-I/usr/include/hdf LDFLAGS=-L/usr/lib ./configure --enable-hdf4 && make

then also package is dependent on latest hdf5 to be build (1.8.7), so

installing it globally would break possible dependencies in any

packaging system, and lastly it’s just pain to build it if it’s worth

On Sun, Nov 13, 2011 at 10:12 PM, Gökhan Sever <gokhansever@…287…> wrote:

When I remove hdf4 part from config, it builds successfully. Any ideas?


Gökhan

Hello all,

Here is my answer to my 2nd question:

http://imageshack.us/photo/my-images/713/modistrue1km.png/

Python script is at http://code.google.com/p/ccnworks/source/browse/trunk/modis/true.py

This code is based on TrueColor tutorial of Liam Gumley [ftp://ftp.ssec.wisc.edu/pub/IMAPP/MODIS/TrueColor/] and Peter Kuma’s ccplot tool [http://ccplot.org/]

Currently it only plots true color images from Aqua or Terra using Level 1B data. For the example image I use the data provided via TrueColor link.

Notes:

1-) Using PyNIO to open hdf-eos files, from ccplot using cctk for 2D data interpolation, basemap for map plotting and numpy/scipy for other essentials.

2-) crefl.1km.hdf is reflectance corrected data using NVDI’s crefl program. Set-up TrueColor tutorial and you should be able to get these correction applied data by calling its main script or set-up another.

3-) MOD03.hdf is the geolocation data.

4-) No need to run ms2gt or any other swath to grid conversion tools, since the interpolation routine handles this step.

5-) Code is in 100 lines. Unlike TrueColor tutorial it only works for 1KM resolution. HKM and QKM resolution plotting requires additional steps.

It takes about 2.5 seconds to get the plot on my screen.

Let me know if you have any comments or other suggestions.

Thanks.

···

On Sun, Nov 13, 2011 at 12:40 PM, Gökhan Sever <gokhansever@…287…> wrote:

Hello groups,

I have two questions about working with MODIS data.

1-) Is there any light Pythonic HDF-EOS wrapper to handle HDF-EOS data other than PyNIO [http://www.pyngl.ucar.edu/Nio.shtml] Although, I have managed to install that package from its source, it took me many hours to figure out all the installation quirks. Something simpler to build and mainly for HDFEOS data??

2-) Another similar question: Has anybody attempted to create true-color MODIS images (like the ones shown at [http://rapidfire.sci.gsfc.nasa.gov/realtime/]) in Python? So far, I have seen one clear tutorial [ftp://ftp.ssec.wisc.edu/pub/IMAPP/MODIS/TrueColor/] to create natural color images, but uses ms2gt [http://nsidc.org/data/modis/ms2gt/], NDVI and IDL. Except the reflectance correction via NDVI, ms2gt and IDL parts seem to be implemented in Python.

Till now, I have some progress combining GOES imagery with aircraft data. My next task is to combine MODIS data with aircraft and radar data. I would be happy to get some guidance and code support if there is any previous work been done using Python.

Thanks.


Gökhan