[Matplotlib-users] paletted PNG backend

Development matplotlib-devel
1

Jesper,

That is a cool application!

I am not sure doing this directly in Agg would be the best place (as I am
not sure how to conceptually combine anti-aliasing + compositing + color
pallet). I think a better place to extend would be the png writer to
directly export a palleted png (
https://github.com/matplotlib/matplotlib/blob/master/src/_png.cpp#L123)
from the Agg RBGA buffer.

You might also be interested in
Faster image generation in WebAgg/NbAgg backends by mdboom · Pull Request #5389 · matplotlib/matplotlib · GitHub where Mike did some work
to sort out which png settings are fastest.

Tom

PS The mailing list has moved to matplotlib-devel at python.org please
re-subscribe and send future emails there.

···

On Mon, Feb 15, 2016 at 3:19 AM Jesper Larsen <jesper.webmail at gmail.com> wrote:

Hi Matplotlib users,

We are using Matplotlib for a web service which makes PNG images on the
fly for presentation on a map (web site using the web service is here:
https://ifm-beta.fcoo.dk)

Performance and image size are two major concerns for us. We therefore
save the resulting RGBA PNG to a buffer and afterwards use Pillow (PIL) to
convert it to a P PNG (paletted PNG) to reduce the image size dramatically.

This procedure does however use a significant amount of our total
processing time per image. I would therefore be interested in extending
e.g. the AGG backend to produce paletted PNGs directly. I am of course
aware that this might not be useful for many others since one would have to
provide some extra information when rendering with this backend (possibly
output palette and quantizing method). But on the other hand it might be
useful for others doing web services using matplotlib.

My questions are:

1) Is it possible to extend the AGG backend for this and how?

2) Is it better to make a separate Pillow based backend for this (Pillow
is probably not as fast as AGG)?

Best regards,
Jesper

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at lists.sourceforge.net
matplotlib-users List Signup and Options

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20160215/e8896f33/attachment.html&gt;

2

I wonder whether we can take advantage of this: https://pngquant.org/.

Eric

···

On 2016/02/15 5:43 AM, Thomas Caswell wrote:

Jesper,

That is a cool application!

I am not sure doing this directly in Agg would be the best place (as I
am not sure how to conceptually combine anti-aliasing + compositing +
color pallet). I think a better place to extend would be the png writer
to directly export a palleted png
(https://github.com/matplotlib/matplotlib/blob/master/src/_png.cpp#L123)
from the Agg RBGA buffer.

You might also be interested in
Faster image generation in WebAgg/NbAgg backends by mdboom · Pull Request #5389 · matplotlib/matplotlib · GitHub where Mike did some
work to sort out which png settings are fastest.

Tom

PS The mailing list has moved to matplotlib-devel at python.org
<mailto:matplotlib-devel at python.org> please re-subscribe and send
future emails there.

On Mon, Feb 15, 2016 at 3:19 AM Jesper Larsen <jesper.webmail at gmail.com > <mailto:jesper.webmail at gmail.com>> wrote:

    Hi Matplotlib users,

    We are using Matplotlib for a web service which makes PNG images on
    the fly for presentation on a map (web site using the web service is
    here: https://ifm-beta.fcoo.dk)

    Performance and image size are two major concerns for us. We
    therefore save the resulting RGBA PNG to a buffer and afterwards use
    Pillow (PIL) to convert it to a P PNG (paletted PNG) to reduce the
    image size dramatically.

    This procedure does however use a significant amount of our total
    processing time per image. I would therefore be interested in
    extending e.g. the AGG backend to produce paletted PNGs directly. I
    am of course aware that this might not be useful for many others
    since one would have to provide some extra information when
    rendering with this backend (possibly output palette and quantizing
    method). But on the other hand it might be useful for others doing
    web services using matplotlib.

    My questions are:

    1) Is it possible to extend the AGG backend for this and how?

    2) Is it better to make a separate Pillow based backend for this
    (Pillow is probably not as fast as AGG)?

    Best regards,
    Jesper

    ------------------------------------------------------------------------------
    Site24x7 APM Insight: Get Deep Visibility into Application Performance
    APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
    Monitor end-to-end web transactions and take corrective actions now
    Troubleshoot faster and improve end-user experience. Signup Now!
    http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________
    Matplotlib-users mailing list
    Matplotlib-users at lists.sourceforge.net
    <mailto:Matplotlib-users at lists.sourceforge.net>
    matplotlib-users List Signup and Options

_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
Matplotlib-devel Info Page

3

We are using Matplotlib for a web service which makes PNG images on the
fly for presentation on a map (web site using the web service is here:
https://ifm-beta.fcoo.dk)

Performance and image size are two major concerns for us. We therefore
save the resulting RGBA PNG to a buffer and afterwards use Pillow (PIL) to
convert it to a P PNG (paletted PNG) to reduce the image size dramatically.

We have similar issues, though not with MPL -- in fact one reason we are
not using MPL is because we want highly optimized rendering -- all the
features MPL provides do add overhead...

And doing experiments with various rendering engines (PIL, AGG, SKIA, GD)
with found that they all have pretty darn similar performance, but that
performance scales pretty much with how many bytes you are pushing -- i.e.
larger images take longer to render, and 8 bits per pixel is faster than 32
bits per pixel (by pretty much a factor of four, if I recall).

So while you are now taking a lot of time converting, you might also get
faster rendering in the first place if you use 8bpp rendering.

But, as Thomas suggested, AGG is pretty much all about anti-aliasing and
that requires more than 255 colors....

So: the "right" way to solve this problem is to use a non-anti-aliasing,
8bpp rendering lib. You could make a new back-end that uses that instead of
AGG. ONE of the core MPLS devs (I can't remember who) suggested that MPL
could use a maybe-not-the-best-quality-but-fast back end last year at
SciPy, so maybe you'll get some help if you want to do it.

Also, a non-anti-aliasing back-end would be nice for things like contours
where you can get anti-aliasing artifacts where adjacent polygons are
supposed to line up exactly.

I suggest the libgd rendering lib:

It's not seeing a massive amount of development, but that's because it's
been around forever and is pretty darn robust and stable. It's also got an
ancient, semi-ugly C API, but we can deal with that :slight_smile:

If you want to make a MP back end with it, I suggest you start with my
Cython-based python wrapper:

It's not terribly complete, but provides the core functionality for
rendering 8 bit images, and numpy interactivity -- i.e. passing coordinates
of large polygons as a numpy array, and passing the raw image data in and
out as a numpy array.

I've also got conda packages for libgd and py_gd in our channel here:

https://conda.anaconda.org/noaa-orr-erd

The recipes for those are here:

Honestly, AGG has been pretty integral to MPL from the beginning (Or near
beginning), so I'm not sure how hard it would be to drop in a new renderer,
but I'd be glad to help.

2) Is it better to make a separate Pillow based backend for this (Pillow is

probably not as fast as AGG)?

NOTE: we looked at using PIL for our rendering, and it was kind-of-sort of
fast enough, but the rendering code, and python calls to it is all
hand-written C, so it looked a lot harder to extend and optimize
performance for.

With py_gd, we can direct calls to the C lib form numpy arrays directly,
and write time-sensitive loops in cython where there are speed bottlenecks.

One issue that will take a bit of effort is colormaps -- IIUC, MPL pretty
much assumes 32 bit (or 24 bit anyway) color. With 8bpp, colormaps have to
be limited to 255 colors, and, in practice, you probably want to save a few
"pure" colors for rendering text and the axes, etc: transparent, white,
black -- a few others?

I think you could get a fine colormap with, say 245 colors, saving 10 for
other uses, but then you could only have one colormap in use at a time. So
there may be some working around MPL's colormap code for this....

-Chris

···

On Mon, Feb 15, 2016 at 3:19 AM Jesper Larsen <jesper.webmail at gmail.com> > wrote:

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20160217/32016891/attachment.html&gt;

4

Hi Thomas

Thanks for your reply. The pull request by mdboom actually led me to find
an undocumented option in Pillow to set the compression level (we now use
compress_level=7). This reduced the cost of producing the paletted PNG very
much with very little effort.

We are however still interested in improving the performance further. And
the information that you have provided is definitely useful for this

Best regards,
Jesper

2016-02-15 16:43 GMT+01:00 Thomas Caswell <tcaswell at gmail.com>:

Jesper,

That is a cool application!

I am not sure doing this directly in Agg would be the best place (as I am
not sure how to conceptually combine anti-aliasing + compositing + color
pallet). I think a better place to extend would be the png writer to
directly export a palleted png (
https://github.com/matplotlib/matplotlib/blob/master/src/_png.cpp#L123)
from the Agg RBGA buffer.

You might also be interested in
Faster image generation in WebAgg/NbAgg backends by mdboom · Pull Request #5389 · matplotlib/matplotlib · GitHub where Mike did some
work to sort out which png settings are fastest.

Tom

PS The mailing list has moved to matplotlib-devel at python.org please
re-subscribe and send future emails there.

Hi Matplotlib users,

We are using Matplotlib for a web service which makes PNG images on the
fly for presentation on a map (web site using the web service is here:
https://ifm-beta.fcoo.dk)

Performance and image size are two major concerns for us. We therefore
save the resulting RGBA PNG to a buffer and afterwards use Pillow (PIL) to
convert it to a P PNG (paletted PNG) to reduce the image size dramatically.

This procedure does however use a significant amount of our total
processing time per image. I would therefore be interested in extending
e.g. the AGG backend to produce paletted PNGs directly. I am of course
aware that this might not be useful for many others since one would have to
provide some extra information when rendering with this backend (possibly
output palette and quantizing method). But on the other hand it might be
useful for others doing web services using matplotlib.

My questions are:

1) Is it possible to extend the AGG backend for this and how?

2) Is it better to make a separate Pillow based backend for this (Pillow
is probably not as fast as AGG)?

Best regards,
Jesper

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Matplotlib-users mailing list
Matplotlib-users at lists.sourceforge.net
matplotlib-users List Signup and Options

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20160217/a2e0bde6/attachment-0001.html&gt;

···

On Mon, Feb 15, 2016 at 3:19 AM Jesper Larsen <jesper.webmail at gmail.com> > wrote:

5

Hi Chris

gd seems like a great candidate for 8-bit rendering (which is what we
want). And thanks for the insights into PIL. I will look more into the
links you provided.

Best regards,
Jesper

2016-02-17 19:23 GMT+01:00 Chris Barker <chris.barker at noaa.gov>:

We are using Matplotlib for a web service which makes PNG images on the
fly for presentation on a map (web site using the web service is here:
https://ifm-beta.fcoo.dk)

Performance and image size are two major concerns for us. We therefore
save the resulting RGBA PNG to a buffer and afterwards use Pillow (PIL) to
convert it to a P PNG (paletted PNG) to reduce the image size dramatically.

We have similar issues, though not with MPL -- in fact one reason we are
not using MPL is because we want highly optimized rendering -- all the
features MPL provides do add overhead...

And doing experiments with various rendering engines (PIL, AGG, SKIA, GD)
with found that they all have pretty darn similar performance, but that
performance scales pretty much with how many bytes you are pushing -- i.e.
larger images take longer to render, and 8 bits per pixel is faster than 32
bits per pixel (by pretty much a factor of four, if I recall).

So while you are now taking a lot of time converting, you might also get
faster rendering in the first place if you use 8bpp rendering.

But, as Thomas suggested, AGG is pretty much all about anti-aliasing and
that requires more than 255 colors....

So: the "right" way to solve this problem is to use a non-anti-aliasing,
8bpp rendering lib. You could make a new back-end that uses that instead of
AGG. ONE of the core MPLS devs (I can't remember who) suggested that MPL
could use a maybe-not-the-best-quality-but-fast back end last year at
SciPy, so maybe you'll get some help if you want to do it.

Also, a non-anti-aliasing back-end would be nice for things like contours
where you can get anti-aliasing artifacts where adjacent polygons are
supposed to line up exactly.

I suggest the libgd rendering lib:

GitHub - libgd/libgd: GD Graphics Library

It's not seeing a massive amount of development, but that's because it's
been around forever and is pretty darn robust and stable. It's also got an
ancient, semi-ugly C API, but we can deal with that :slight_smile:

If you want to make a MP back end with it, I suggest you start with my
Cython-based python wrapper:

GitHub - NOAA-ORR-ERD/py_gd: python wrappers for libgd graphics drawing lib

It's not terribly complete, but provides the core functionality for
rendering 8 bit images, and numpy interactivity -- i.e. passing coordinates
of large polygons as a numpy array, and passing the raw image data in and
out as a numpy array.

I've also got conda packages for libgd and py_gd in our channel here:

Package repository for noaa-orr-erd :: Anaconda.org

The recipes for those are here:

GitHub - NOAA-ORR-ERD/orr-conda-recipes: Recipes for building conda packages for many (all?) of the third party packages required by ORR tools -- specifically GNOME

Honestly, AGG has been pretty integral to MPL from the beginning (Or near
beginning), so I'm not sure how hard it would be to drop in a new renderer,
but I'd be glad to help.

2) Is it better to make a separate Pillow based backend for this (Pillow

is probably not as fast as AGG)?

NOTE: we looked at using PIL for our rendering, and it was kind-of-sort of
fast enough, but the rendering code, and python calls to it is all
hand-written C, so it looked a lot harder to extend and optimize
performance for.

With py_gd, we can direct calls to the C lib form numpy arrays directly,
and write time-sensitive loops in cython where there are speed bottlenecks.

One issue that will take a bit of effort is colormaps -- IIUC, MPL pretty
much assumes 32 bit (or 24 bit anyway) color. With 8bpp, colormaps have to
be limited to 255 colors, and, in practice, you probably want to save a few
"pure" colors for rendering text and the axes, etc: transparent, white,
black -- a few others?

I think you could get a fine colormap with, say 245 colors, saving 10 for
other uses, but then you could only have one colormap in use at a time. So
there may be some working around MPL's colormap code for this....

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker at noaa.gov

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20160217/edc24275/attachment.html&gt;

···

On Mon, Feb 15, 2016 at 3:19 AM Jesper Larsen <jesper.webmail at gmail.com> >> wrote: