MPL_interactions: Verts not returned from image_segmenter object

So my problem is as follows:
I have a stack of images to segment (lets say its a (140,100,10) matrix).
I want to segment the 10 different slices seperately and save the masks and also the paths that I have drawn, so that I can later plot them on an anatomical image for visualization purposes.
They way I figured the masks saving out is to first make a list of image segmenters:

seg_list = [image_segmenter(anatomical_data[:,:,s],nclasses=n_rois) for s in range(anatomical_data.shape[2])]

I then give this list to a function that interactively lets me select each slice, segment it and then it also updates the seg_list object. I can then iterate through all seg_list entries and with seg_list[n].mask retrieve the respective masks.
However, I cannot get the coordinates of the lasso selector path. For one segmenter this works using segmenter.verts, but not for multiple ones.
Is verts not saved as a property like masks ? If not would it be possible to include that?

Can you post a complete minimal example?

Looking at the code I’m not really sure why self.verts even exists :confused: It gets defined here: mpl-interactions/generic.py at 0807fa96459ef7fd303bb7298d6d3b6981c624ad · ianhi/mpl-interactions · GitHub and then never used again. When you say that it doesn’t work do you mean it’s undefined, or that it’s empty?

If it’s the latter then it’s definitely worth noting that currently self.verts will also only give you the vertices of the most recently defined path. So if you do multiple paths then it won’t give you a complete picture.

I’m definitely not opposed to keeping tracking of the Paths that get drawn (or their verts, but i think the path objects are more useful) and would happily review a PR implementing that - otherwise lets on a feature request and I’ll get to it when I can

Just for reference: I would like to get the shapes of the ROIs I have drawn/segmented so I can later plot them on the images where I have segmented them from and also have them saved to my datastack.

Below is an example. I am working in a jupyter notebook just for reference, so I’ll post the cells here.

from mpl_interactions import image_segmenter
import numpy as np
%matplotlib widget
test_array = np.zeros((128,128))
foo = image_segmenter(test_array)

Next cell:

foo
# this now opens a window in which I can segment my image

Next cell:

print(foo.verts)

This outputs the coordinates of the roi that I have drawn:

[(44.3, 40.5070997362013), (42.97012987012987, 40.5070997362013), (39.64545454545454, 40.5070997362013), (33.661038961038955, 45.16164519074674), (33.32857142857142, 51.47852830762986),.......]

So fine so good. What I want to do now is to do this for a stack of images interactively. Therefore I have written a little function that uses ipywidgets and gets a list of image_segmenters as an argument
I do so in a new notebook:

from mpl_interactions import image_segmenter
import numpy as np
%matplotlib widget
import ipywidgets as widgets

test_array = np.zeros((128,128,4))
#create a 3D array simulating stacks of images
foo = [image_segmenter(test_array[:,:,n]) for n in range(4) ]

def draw_on_image(segmenter_list):
    def plot_imgs(n_slice, eraser_mode, roi_number):
        temp_seg = segmenter_list[n_slice]
        temp_seg.erasing = eraser_mode
        temp_seg.current_class = roi_number
        display(temp_seg)

    n_rois = segmenter_list[0].nclasses
    n_slices = len(segmenter_list)
    # Making the UI
    class_selector = widgets.Dropdown(
        options=list(range(1, n_rois + 1)), description="ROI number"
    )
    erasing_button = widgets.Checkbox(value=False, description="Erasing")
    slice_slider = widgets.IntSlider(
        value=n_slices // 2, min=0, max=n_slices - 1, description="Slice: "
    )


    # put both sliders inside a HBox for nice alignment  etc.
    ui = widgets.HBox(
        [erasing_button, slice_slider, class_selector],
        layout=widgets.Layout(display="flex"),
    )

    sliders = widgets.interactive_output(
        plot_imgs,
        {
            "n_slice": slice_slider,
            "eraser_mode": erasing_button,
            "roi_number": class_selector,
        },
    )

    display(ui, sliders)

I now call this function to draw as many rois as I want:

draw_on_image(foo)

After doing so I can access the masks using:

foo[0].mask
# for slice 0 for example

This allows me to save the masks for different ROIs.

But I cannot:

foo[0].verts

As it gives an error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[30], line 1
----> 1 foo[0].verts

AttributeError: 'image_segmenter' object has no attribute 'verts'

Any insight into this is greatly appreciated. Thans Ian!

First of all nice widget UI around the segmenter - I’d be happy to include something like that in docs if you ever feel inspired.

For the attribute error. When I run your code I only get the error for the segmenters on which I hadn’t drawn anything.


For anyone coming across this in the future this has also become a feature request here:

Thanks, I might do that when my code is nice. Would also be nice to have the option to give the ROIs names (if one has more than one class) and then select them interactively (lets say choose one named “kidney” or one named “muscle tissue”). I’ll try to implement something like that and follow up over on the mpl_interactions Github.
Thats weird, for me this works now too. However that still does not solve the issue of what happens when you draw multiple ROIs or start erasing.
Ultimately, I think your solution posted on Github using contours makes the most sense for this problem. Thanks for pointing this out.