blurry images at native resolution with images.BboxImage: patch to fix

I was playing around with images.BboxImage, and found that if I displayed, say a 100x100 image at its native resolution (exactly 100x100 pixels on the plotting window), it was blurred. This is because of the interpolation jumping in and interpolating when it is not needed.

This might not be the best way to fix, but it does work…in matplotlib 1.0.0, it is image.py around line 1102 (this is BboxImage.make_image). Just after numrows and numcols is set in that routine, do

if (r-l) == numcols and (t-b) == numrows:

im.set_interpolation(0)

As you can see, all this does is just flip the interpolation off if the size of the image is the same size that it is about to be rendered as…and the regular interpolation is used otherwise.

I do apologize for the lack of a proper patch, but for little things like this I think a word description works as well or better; I need to sit down and both learn git and set of a development environment for matplotlib, but after reading through the docs on the web site about it, decided that would take me more time than I had at the moment.

In context:

def make_image(self, renderer, magnification=1.0):

if self._A is None:

raise RuntimeError(‘You must first set the image array or the image attribute’)

if self._imcache is None:

if self._A.dtype == np.uint8 and len(self._A.shape) == 3:

im = _image.frombyte(self._A, 0)

im.is_grayscale = False

else:

if self._rgbacache is None:

x = self.to_rgba(self._A, self._alpha)

self._rgbacache = x

else:

x = self._rgbacache

im = _image.fromarray(x, 0)

if len(self._A.shape) == 2:

im.is_grayscale = self.cmap.is_gray()

else:

im.is_grayscale = False

self._imcache = im

if self.origin==‘upper’:

im.flipud_in()

else:

im = self._imcache

image input dimensions

im.reset_matrix()

im.set_interpolation(self._interpd[self._interpolation])

im.set_resample(self._resample)

l, b, r, t = self.get_window_extent(renderer).extents #bbox.extents

widthDisplay = (round® + 0.5) - (round(l) - 0.5)

heightDisplay = (round(t) + 0.5) - (round(b) - 0.5)

widthDisplay *= magnification

heightDisplay *= magnification

numrows, numcols = self._A.shape[:2]

if (r-l) == numcols and (t-b) == numrows: # <----------------- add this

im.set_interpolation(0) #<-------------- and this

resize viewport to display

rx = widthDisplay / numcols

ry = heightDisplay / numrows

#im.apply_scaling(rxsx, rysy)

im.apply_scaling(rx, ry)

#im.resize(int(widthDisplay+0.5), int(heightDisplay+0.5),

norm=self._filternorm, radius=self._filterrad)

im.resize(int(widthDisplay), int(heightDisplay),

norm=self._filternorm, radius=self._filterrad)

return im

···


Daniel Hyams
dhyams@…149…

Daniel,

imshow() does take an “interpolation” kwarg, where you can specify “nearest” for pre v1.1.0 versions and “none” (not None) in the recent release. “nearest” would be close to what you want, while “none” is exactly what you want.

Personally, I don’t think it would be a good idea to automatically turn interpolation off on such a specific set of conditions, but I could be wrong. I would rather have the user explicitly state their interpolation intentions.

Ben Root

···

On Thursday, October 13, 2011, Daniel Hyams <dhyams@…149…> wrote:

I was playing around with images.BboxImage, and found that if I displayed, say a 100x100 image at its native resolution (exactly 100x100 pixels on the plotting window), it was blurred. This is because of the interpolation jumping in and interpolating when it is not needed.

This might not be the best way to fix, but it does work…in matplotlib 1.0.0, it is image.py around line 1102 (this is BboxImage.make_image). Just after numrows and numcols is set in that routine, do
if (r-l) == numcols and (t-b) == numrows:

        im.set_interpolation(0)

As you can see, all this does is just flip the interpolation off if the size of the image is the same size that it is about to be rendered as…and the regular interpolation is used otherwise.

I do apologize for the lack of a proper patch, but for little things like this I think a word description works as well or better; I need to sit down and both learn git and set of a development environment for matplotlib, but after reading through the docs on the web site about it, decided that would take me more time than I had at the moment.

In context:
def make_image(self, renderer, magnification=1.0):
if self._A is None:
raise RuntimeError(‘You must first set the image array or the image attribute’)

    if self._imcache is None:
        if self._A.dtype == np.uint8 and len(self._A.shape) == 3:
            im = _image.frombyte(self._A, 0)
            im.is_grayscale = False
        else:
            if self._rgbacache is None:
                x = self.to_rgba(self._A, self._alpha)
                self._rgbacache = x
            else:
                x = self._rgbacache
            im = _image.fromarray(x, 0)
            if len(self._A.shape) == 2:
                im.is_grayscale = self.cmap.is_gray()
            else:
                im.is_grayscale = False
        self._imcache = im
        if self.origin=='upper':
            im.flipud_in()
    else:
        im = self._imcache
    # image input dimensions
    im.reset_matrix()
    im.set_interpolation(self._interpd[self._interpolation])
    
    
    im.set_resample(self._resample)
    l, b, r, t = self.get_window_extent(renderer).extents #bbox.extents
    widthDisplay = (round(r) + 0.5) - (round(l) - 0.5)
    heightDisplay = (round(t) + 0.5) - (round(b) - 0.5)
    widthDisplay *= magnification
    heightDisplay *= magnification
    numrows, numcols = self._A.shape[:2]
    
    if (r-l) == numcols and (t-b) == numrows:    # <----------------- add this
        im.set_interpolation(0)                           #<-------------- and this
    # resize viewport to display
    rx = widthDisplay / numcols
    ry = heightDisplay  / numrows
    #im.apply_scaling(rx*sx, ry*sy)
    im.apply_scaling(rx, ry)
    #im.resize(int(widthDisplay+0.5), int(heightDisplay+0.5),
    #          norm=self._filternorm, radius=self._filterrad)
    im.resize(int(widthDisplay), int(heightDisplay),
              norm=self._filternorm, radius=self._filterrad)
    return im


Daniel Hyams
dhyams@…149…

imshow() does take an “interpolation” kwarg, where you can specify “nearest” for pre v1.1.0 versions and “none” (not None) in the recent release. “nearest” would be close to what you want, while “none” is exactly what you want.

Personally, I don’t think it would be a good idea to automatically turn interpolation off on such a specific set of conditions, but I could be wrong. I would rather have the user explicitly state their interpolation intentions.

Ben Root

Hello Ben,

Isn’t the purpose of interpolation to handle situations where the image is being displayed at a different size than its native resolution? It seems appropriate to me to turn interpolation off in such a case.

Just some context of what I was up to when I ran into this one…I’m trying to allow the user to paste images (either from file or clipboard) onto a matplotlib canvas and be able to drag and resize them, very similar to the way powerpoint works. So I wanted to work with the appropriate image object in matplotlib, and BboxImage was the best I found after digging in the code for a bit; imshow() is too tied to the axis…these are supposed to be just free floating images.

What I ran into was that, on initial paste (where I am sizing the BboxImage the same width/height as the incoming picture), the image was very blurry (because of the interpolation). I still want the interpolation on so that the image looks nice when it is resized by the user with the grab handles, but in the specific case where no interpolation is needed, this patch turns it off.

The pasting/dragging/resizing has actually turned out pretty well…once I get it cleaned up and working how I want it to, I’ll submit it to you guys for inclusion into matplotlib if you want it. I was hoping to formulate it as a mixin that will work for any artist (not just BboxImage) within reasonable limits.

···


Daniel Hyams
dhyams@…149…

Not solely, it can also be used to do local average of noisy images to
get a smoother view, eg bilinear or bicubic averaging of neighboring
pixels.

···

On Thu, Oct 13, 2011 at 10:06 AM, Daniel Hyams <dhyams@...149...> wrote:

Isn't the purpose of interpolation to handle situations where the image is
being displayed at a different size than its native resolution? It seems

Ah yes, I forget :confused: I was focused on images as being “pure” things that should be displayed, and forgot about the image processing angle.

So would the solution be a keyword argument that tells imshow/BboxImage and friends not to interpolate when at native resolution, which is set to the current behavior as default?

If that’s not an acceptable solution, I can just leave the patch in my own personal code and not worry about any further…I thought that I was fixing a bug there :slight_smile:

I guess the main difference is whether the image is treated as sacred and should be displayed perfectly when possible, versus the ability to modify the picture on purpose via the interpolations, for whatever reason the user wants. Understandably, matplotlib has taken the latter approach, because the context has always been (as far as I can tell from the examples) displaying the pixels for a scientific purpose. However, when you want to display an image for a annotational type purpose, the former approach should be taken, in my opinion.

···

On Thu, Oct 13, 2011 at 11:13 AM, John Hunter <jdh2358@…149…> wrote:

On Thu, Oct 13, 2011 at 10:06 AM, Daniel Hyams <dhyams@…149…> wrote:

Isn’t the purpose of interpolation to handle situations where the image is

being displayed at a different size than its native resolution? It seems

Not solely, it can also be used to do local average of noisy images to

get a smoother view, eg bilinear or bicubic averaging of neighboring

pixels.


Daniel Hyams
dhyams@…149…

On this topic, I have a patch that tells BboxImage whether or not to
always apply interpolation (as is the desire when doing data analysis)
or to apply interpolation only when the image is at a non-native
resolution (as is the desire when using an image to annotate
something).

The pull request is here:

Since we cannot attach files on github, I've attached the test script,
image file that I was using, and the before/after here.

And can the max post size be increased on matplotlib-users, so that
when even smallish images are attached, the messages don't go into
moderation?

screen.png

image_blurry.py (709 Bytes)

blurry.png

nonblurry.png

···

On Thu, Oct 13, 2011 at 11:40 AM, Daniel Hyams <dhyams@...149...> wrote:

Ah yes, I forget :confused: I was focused on images as being "pure" things that
should be displayed, and forgot about the image processing angle.
So would the solution be a keyword argument that tells imshow/BboxImage and
friends not to interpolate when at native resolution, which is set to the
current behavior as default?
If that's not an acceptable solution, I can just leave the patch in my own
personal code and not worry about any further...I thought that I was fixing
a bug there :slight_smile:

I guess the main difference is whether the image is treated as sacred and
should be displayed perfectly when possible, versus the ability to modify
the picture on purpose via the interpolations, for whatever reason the user
wants. Understandably, matplotlib has taken the latter approach, because
the context has always been (as far as I can tell from the examples)
displaying the pixels for a scientific purpose. However, when you want to
display an image for a annotational type purpose, the former approach should
be taken, in my opinion.

On Thu, Oct 13, 2011 at 11:13 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, Oct 13, 2011 at 10:06 AM, Daniel Hyams <dhyams@...149...> wrote:

> Isn't the purpose of interpolation to handle situations where the image
> is
> being displayed at a different size than its native resolution? It
> seems

Not solely, it can also be used to do local average of noisy images to
get a smoother view, eg bilinear or bicubic averaging of neighboring
pixels.

--
Daniel Hyams
dhyams@...149...

--
Daniel Hyams
dhyams@...149...