Planning for 1.3.0

Congrats to everyone on a successful 1.2.1 -- there was a relatively small influx of bug reports following it -- perhaps a sign of improving quality?

In any event, as Phil Elson likes to repeatedly point out (<wink>), we have a great deal of awesome new features in master that it would be great to get out. As for time frame, I think if we aim for a 1.3.0rc1 of May 27, that's within a good time frame to get something out in time for Scipy. That will put us about 8 months past 1.2.0, which is not far off from my eventual goal of 6 month releases once things get streamlined. We can, of course, adjust the date as necessary as we go along.

So... let the bug triaging begin! As there are lot of new features in the queue, I think it's important to distinguish the "nice to have" from the "must have". I've created a new milestone "1.3.x blocker" that we should use for critical bugs that should hold up the release. All other new features that are looking close to ready for 1.3.x should be milestoned 1.3.x, and as we get closer, we can punt those on to 1.4.x. Simple bugs that apply to 1.2.x as well as master should be milestoned as "1.2.x" and merged onto the "v1.2.x" branch (as we've been doing), as I still think we should reserve the right to make another 1.2.2 bugfix release if necessary.

There are a couple major ongoing projects that I think we'll need to get to a steady interim state before we can make another release.

   MEP10: Documentation

      We already have a number of improved docstrings, and better organization throughout. I don't think we need to finish all of this work before the next release, but we should get it back into shape so that the doc build has fewer warnings (#1896) and the LaTeX build works again (#1836).

   MEP12: Reorganize and cleanup examples

      Again, I think a lot of great work has already been done, and we don't necessarily have to wait until it's done.

For both of the above, it would be nice to divide the work up so the leaders of those projects are less individually burdened. Nelle and Tony, if you know of any critical blockers or loose ends that should be tied up before a release should be made, please make issues for them and milestone them as "1.3.x blocker".

Cheers,
Mike

Great news! A lot of fantastic work has been done by a whole host of people to go into this release. It’s exciting stuff!

May 27th sounds like a sensible target to me. As you know, I’m an advocate of releasing often - the more frequently we make a release, the less we will have the “impending release rush” that can really hamper the whole release process.

Cheers,

···

On 11 April 2013 17:38, Michael Droettboom <mdroe@…31…> wrote:

Congrats to everyone on a successful 1.2.1 – there was a relatively

small influx of bug reports following it – perhaps a sign of improving

quality?

In any event, as Phil Elson likes to repeatedly point out (), we

have a great deal of awesome new features in master that it would be

great to get out. As for time frame, I think if we aim for a 1.3.0rc1

of May 27, that’s within a good time frame to get something out in time

for Scipy. That will put us about 8 months past 1.2.0, which is not far

off from my eventual goal of 6 month releases once things get

streamlined. We can, of course, adjust the date as necessary as we go

along.

So… let the bug triaging begin! As there are lot of new features in

the queue, I think it’s important to distinguish the “nice to have” from

the “must have”. I’ve created a new milestone “1.3.x blocker” that we

should use for critical bugs that should hold up the release. All other

new features that are looking close to ready for 1.3.x should be

milestoned 1.3.x, and as we get closer, we can punt those on to 1.4.x.

Simple bugs that apply to 1.2.x as well as master should be milestoned

as “1.2.x” and merged onto the “v1.2.x” branch (as we’ve been doing), as

I still think we should reserve the right to make another 1.2.2 bugfix

release if necessary.

There are a couple major ongoing projects that I think we’ll need to get

to a steady interim state before we can make another release.

MEP10: Documentation

  We already have a number of improved docstrings, and better

organization throughout. I don’t think we need to finish all of this

work before the next release, but we should get it back into shape so

that the doc build has fewer warnings (#1896) and the LaTeX build works

again (#1836).

MEP12: Reorganize and cleanup examples

  Again, I think a lot of great work has already been done, and we

don’t necessarily have to wait until it’s done.

For both of the above, it would be nice to divide the work up so the

leaders of those projects are less individually burdened. Nelle and

Tony, if you know of any critical blockers or loose ends that should be

tied up before a release should be made, please make issues for them and

milestone them as “1.3.x blocker”.

Cheers,

Mike


Precog is a next-generation analytics platform capable of advanced

analytics on semi-structured data. The platform includes APIs for building

apps and a phenomenal toolset for data science. Developers can use

our toolset for easy data analysis & visualization. Get a free account!

http://www2.precog.com/precogplatform/slashdotnewsletter


Matplotlib-devel mailing list

Matplotlib-devel@lists.sourceforge.net

https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Thanks and congratulations to everyone involved as well; I've built 1.2.1 on MacOS X with fink for
Python2.6-3.2 without any failures in the test suite!
I did run into a problem though that has actually existed since the first 1.2 release - with the MacOSX
backend line plots of somewhat larger arrays with significant "high-frequency" power had extremely
degraded, e.g. something like

x = np.linspace(0,10,1.e6)
y = np.cos(x)+0.2*np.sin(x*3.e3)+0.1*np.cos(x*4.e4)*np.random.rand(1.e6)
plt.plot(x, y)

would display within less than 2 seconds with 1.1.1, but with 1.2.x you literally have to wait minutes,
and it takes similarly long to zoom in as long as you have a substantial part of the line in the window.

I found in the current HEAD (9e477b3) this has finally been fixed - thanks for that as well, whatever
the problem was, but now in the 1.3 branch the _macosx backend has been altogether disabled!
I verified after removing that RuntimeError from _macosx.m that the backend still works and is indeed
up to its old speed; but if that change stays in, it won't be usable from non-framework Python installs
like the fink ones.
Personally, I am aware of the problems with the missing window manager control, and occasionally
am annoyed by hunting for a plot window that has sneaked somewhere underneath other windows,
but with that in mind I still prefer the MacOSX backend to any of the others, and I would suggest to
leave it at a warning rather than an error, so users can still decide for themselves if they want to put
up with the possible troubles.

Cheers,
              Derek

···

On 11.04.2013, at 6:38PM, Michael Droettboom <mdroe@...31...> wrote:

Congrats to everyone on a successful 1.2.1 -- there was a relatively
small influx of bug reports following it -- perhaps a sign of improving
quality?

Hi Derek,

The slow speed for long paths like the one in your example was due to a limitation to Quartz itself. This was solved by breaking the path up into subpaths of up to 100 points. But you mentioned that releases before 1.2 were not slow (and I verified this with matplotlib 1.1.1), suggesting that something else is going on. Can you check which change between 1.1.1 and 1.2 is causing the slowdown for your example?

Best,
-Michiel.

···

--- On Fri, 4/12/13, Derek Homeier <derek@...873...> wrote:

From: Derek Homeier <derek@...873...>
Subject: Re: [matplotlib-devel] Planning for 1.3.0
To: "matplotlib-devel@lists.sourceforge.net list" <matplotlib-devel@...923....sourceforge.net>
Date: Friday, April 12, 2013, 5:16 PM
On 11.04.2013, at 6:38PM, Michael > Droettboom <mdroe@...31...> > wrote:

> Congrats to everyone on a successful 1.2.1 -- there was
a relatively
> small influx of bug reports following it -- perhaps a
sign of improving
> quality?

Thanks and congratulations to everyone involved as well;
I've built 1.2.1 on MacOS X with fink for
Python2.6-3.2 without any failures in the test suite!
I did run into a problem though that has actually existed
since the first 1.2 release - with the MacOSX
backend line plots of somewhat larger arrays with
significant "high-frequency" power had extremely
degraded, e.g. something like

x = np.linspace(0,10,1.e6)
y =
np.cos(x)+0.2*np.sin(x*3.e3)+0.1*np.cos(x*4.e4)*np.random.rand(1.e6)
plt.plot(x, y)

would display within less than 2 seconds with 1.1.1, but
with 1.2.x you literally have to wait minutes,
and it takes similarly long to zoom in as long as you have a
substantial part of the line in the window.

I found in the current HEAD (9e477b3) this has finally been
fixed - thanks for that as well, whatever
the problem was, but now in the 1.3 branch the _macosx
backend has been altogether disabled!
I verified after removing that RuntimeError from _macosx.m
that the backend still works and is indeed
up to its old speed; but if that change stays in, it won't
be usable from non-framework Python installs
like the fink ones.
Personally, I am aware of the problems with the missing
window manager control, and occasionally
am annoyed by hunting for a plot window that has sneaked
somewhere underneath other windows,
but with that in mind I still prefer the MacOSX backend to
any of the others, and I would suggest to
leave it at a warning rather than an error, so users can
still decide for themselves if they want to put
up with the possible troubles.

Cheers,

Derek

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of
advanced
analytics on semi-structured data. The platform includes
APIs for building
apps and a phenomenal toolset for data science. Developers
can use
our toolset for easy data analysis & visualization. Get
a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Hi Michiel,

The slow speed for long paths like the one in your example was due to a limitation to Quartz itself. This was solved by breaking the path up into subpaths of up to 100 points. But you mentioned that releases before 1.2 were not slow (and I verified this with matplotlib 1.1.1), suggesting that something else is going on. Can you check which change between 1.1.1 and 1.2 is causing the slowdown for your example?

It's the passing of set_dpi (commit 6533674) - that's still unchanged in master,
but I don't see any speed penalty compared to 1.1.1 any more. I don't know if
the change you mentioned above completely fixed this or just made up for it
by speeding it up otherwise…
I have just merged all updates to backend_maxosx.py and _macosx.m back
into 1.2.1, and this seems to solve the issue and passes all tests as well.

Cheers,
          Derek

···

On 13.04.2013, at 1:30AM, Michiel de Hoon wrote:

Hi Derek,

That is good to hear.
The slowdown was caused by the performance of Quartz itself, but it depends strongly on the line width. In your example, the plot appears immediately if you use linewidth=0.9, but (with matplotlib 1.2.1) takes minutes to appear if you use linewidth=1.0. The change in set_dpi caused the line width actually used for drawing to increase slightly. The increase was very small, but big enough to trigger the ultraslow behavior of Quartz. As I mentioned, we solved this by breaking up the path into many subpaths, which solved the problem (without having to change set_dpi back).
Anyway, if I understand your mail correctly, the problem has been fixed in HEAD. Is the 1.3 branch also OK now? In your first post you mentioned that there was some RuntimeError.

Best,
-Michiel.

···

--- On Sat, 4/13/13, Derek Homeier <derek@...873...> wrote:

From: Derek Homeier <derek@...873...>
Subject: Re: [matplotlib-devel] Planning for 1.3.0
To: "matplotlib development list" <matplotlib-devel@lists.sourceforge.net>
Date: Saturday, April 13, 2013, 9:03 AM
Hi Michiel,

On 13.04.2013, at 1:30AM, Michiel de Hoon wrote:

> The slow speed for long paths like the one in your
example was due to a limitation to Quartz itself. This was
solved by breaking the path up into subpaths of up to 100
points. But you mentioned that releases before 1.2 were not
slow (and I verified this with matplotlib 1.1.1), suggesting
that something else is going on. Can you check which change
between 1.1.1 and 1.2 is causing the slowdown for your
example?

It's the passing of set_dpi (commit 6533674) - that's still
unchanged in master,
but I don't see any speed penalty compared to 1.1.1 any
more. I don't know if
the change you mentioned above completely fixed this or just
made up for it
by speeding it up otherwise…
I have just merged all updates to backend_maxosx.py and
_macosx.m back
into 1.2.1, and this seems to solve the issue and passes all
tests as well.

Cheers,

    Derek

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of
advanced
analytics on semi-structured data. The platform includes
APIs for building
apps and a phenomenal toolset for data science. Developers
can use
our toolset for easy data analysis & visualization. Get
a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Hi Michiel,

That is good to hear.
The slowdown was caused by the performance of Quartz itself, but it depends strongly on the line width. In your example, the plot appears immediately if you use linewidth=0.9, but (with matplotlib 1.2.1) takes minutes to appear if you use linewidth=1.0. The change in set_dpi caused the line width actually used for drawing to increase slightly. The increase was very small, but big enough to trigger the ultraslow behavior of Quartz. As I mentioned, we solved this by breaking up the path into many subpaths, which solved the problem (without having to change set_dpi back).
Anyway, if I understand your mail correctly, the problem has been fixed in HEAD. Is the 1.3 branch also OK now? In your first post you mentioned that there was some RuntimeError.

I saw a couple of warnings with Friday's checkout on 10.8, but the current one seems to
work fine (now on 10.7 however…). I've run the full test suite and only had three failures
in test_font_styles (basically all created fonts look like 'light'/'condensed').
The same with python3.2 after I upgraded pyparsing, only at the end of 'setup.py install'
there was an additional error, but this did not seem to affect the install (appended below).

The RuntimeError was enforced by the #ifdef WITH_NEXT_FRAMEWORK check that
does not allow to use the backend at all, so I had to change this to a RuntimeWarning
to be able to test the backend in the 1.3 branch.

Cheers,
          Derek

···

--
Processing matplotlib-1.3.x-py3.2-macosx-10.7-x86_64.egg
creating /Users/derek/lib/python3.2/site-packages/matplotlib-1.3.x-py3.2-macosx-10.7-x86_64.egg
Extracting matplotlib-1.3.x-py3.2-macosx-10.7-x86_64.egg to /Users/derek/lib/python3.2/site-packages
Adding matplotlib 1.3.x to easy-install.pth file

Installed /Users/derek/lib/python3.2/site-packages/matplotlib-1.3.x-py3.2-macosx-10.7-x86_64.egg
Processing dependencies for matplotlib==1.3.x
Traceback (most recent call last):
File "setup.py", line 228, in <module>
   'KnownFailure = matplotlib.testing.noseclasses:KnownFailure'
File "/sw/lib/python3.2/distutils/core.py", line 148, in setup
   dist.run_commands()
File "/sw/lib/python3.2/distutils/dist.py", line 917, in run_commands
   self.run_command(cmd)
File "/sw/lib/python3.2/distutils/dist.py", line 936, in run_command
   cmd_obj.run()
File "/sw/lib/python3.2/site-packages/setuptools/command/install.py", line 73, in run
   self.do_egg_install()
File "/sw/lib/python3.2/site-packages/setuptools/command/install.py", line 101, in do_egg_install
   cmd.run()
File "/sw/lib/python3.2/site-packages/setuptools/command/easy_install.py", line 358, in run
   self.easy_install(spec, not self.no_deps)
File "/sw/lib/python3.2/site-packages/setuptools/command/easy_install.py", line 582, in easy_install
   return self.install_item(None, spec, tmpdir, deps, True)
File "/sw/lib/python3.2/site-packages/setuptools/command/easy_install.py", line 634, in install_item
   self.process_distribution(spec, dist, deps)
File "/sw/lib/python3.2/site-packages/setuptools/command/easy_install.py", line 686, in process_distribution
   [requirement], self.local_index, self.easy_install
File "/sw/lib/python3.2/site-packages/pkg_resources.py", line 586, in resolve
   dist = best[req.key] = env.best_match(req, self, installer)
File "/sw/lib/python3.2/site-packages/pkg_resources.py", line 829, in best_match
   for dist in self[req.key]:
File "/sw/lib/python3.2/site-packages/pkg_resources.py", line 799, in __getitem__
   _sort_dists(dists)
File "/sw/lib/python3.2/site-packages/pkg_resources.py", line 2613, in _sort_dists
   tmp.sort()
TypeError: unorderable types: NoneType() < str()

Hi Derek,

···

--- On Sun, 4/14/13, Derek Homeier <derek@...873...> wrote:

The RuntimeError was enforced by the #ifdef
WITH_NEXT_FRAMEWORK check that
does not allow to use the backend at all, so I had to change
this to a RuntimeWarning
to be able to test the backend in the 1.3 branch.

This RuntimeError is there for a reason: If your Python is not installed as a framework, the backend will not work correctly (and if you ignore the RuntimeError, you won't know if any problems you encounter are real bugs, or simply due to your Python not being installed as a framework).

Best,
-Michiel

Hi Michiel,

This RuntimeError is there for a reason: If your Python is not installed as a framework, the backend will not work correctly (and if you ignore the RuntimeError, you won't know if any problems you encounter are real bugs, or simply due to your Python not being installed as a framework).

I have used the MacOSX backend ever since I started working with Python,
and there only was a warning for the last 3 years or so. Other than my plot
windows evading the control of Exposé/Application switcher I have never
noticed any problems with this. Thus my request in my initial post to leave it
as a mere warning, since I'd think people can switch on their own decision
if they do not like the interaction (or lack thereof) with the window manager.
Otherwise I would have to grudgingly switch to another backend (seems now
that I could live with QT4Agg-Quartz or the like though), since installation as
as framework is not an option with Fink.
Of course if there are any other possible negative effects besides the window
handling, I'd take your point.

Best,
            Derek

Hi Derek,

···

--- On Sun, 4/14/13, Derek Homeier <derek@...873...> wrote:

Of course if there are any other possible negative effects
besides the window handling, I'd take your point.

Several bugs have been reported in the past that turned out to be due to Python not being installed as a framework. For example, the file selection window when saving a figure doesn't respond. This has been a major hassle, since each of those bug reports take time to investigate before realizing that it is due to the Python installation.

Best,
-Michiel.

Hi Michiel,

Of course if there are any other possible negative effects
besides the window handling, I'd take your point.

Several bugs have been reported in the past that turned out to be due to Python not being installed as a framework. For example, the file selection window when saving a figure doesn't respond. This has been a major hassle, since each of those bug reports take time to investigate before realizing that it is due to the Python installation.

OK, that is a valid reason! I vaguely remember some problems with that in the past,
though haven't experienced any of that in a long time (just tested 'Save File', and I've
been regularly using 'Configure Subplots'). But this is probably a case where a
Warning might not keep all users from filing a bug. :frowning:
The QT4Agg backend has its ups as well, though there still seem to be some quirks,
too (e.g. the zoom rectangle only shows up with the left and lower border); I will probably
become a fan of the line configuration tool that allows you to switch back to linear from a
log axis scale (in the command line there seems to be no return from plt.semilogy() or
plt.loglog())!

Cheers,
            Derek

···

On 15.04.2013, at 6:03AM, Michiel de Hoon <mjldehoon@...42...> wrote:

--- On Sun, 4/14/13, Derek Homeier <derek@...873...> wrote:

Hi Derek,

Can you perhaps ask the Fink developers to provide a framework installation of Python? Most matplotlib users who ran into framework-related bugs were Fink users.

Best,
-Michiel.

···

--- On Mon, 4/15/13, Derek Homeier <derek@...873...> wrote:

From: Derek Homeier <derek@...873...>
Subject: Re: [matplotlib-devel] Planning for 1.3.0
To: "matplotlib development list" <matplotlib-devel@lists.sourceforge.net>
Date: Monday, April 15, 2013, 11:00 AM
Hi Michiel,

On 15.04.2013, at 6:03AM, Michiel de Hoon <mjldehoon@...42...> > wrote:

> --- On Sun, 4/14/13, Derek Homeier <derek@...873...> > wrote:
>> Of course if there are any other possible negative
effects
>> besides the window handling, I'd take your point.
>
> Several bugs have been reported in the past that turned
out to be due to Python not being installed as a framework.
For example, the file selection window when saving a figure
doesn't respond. This has been a major hassle, since each of
those bug reports take time to investigate before realizing
that it is due to the Python installation.

OK, that is a valid reason! I vaguely remember some problems
with that in the past,
though haven't experienced any of that in a long time (just
tested 'Save File', and I've
been regularly using 'Configure Subplots'). But this is
probably a case where a
Warning might not keep all users from filing a bug. :frowning:
The QT4Agg backend has its ups as well, though there still
seem to be some quirks,
too (e.g. the zoom rectangle only shows up with the left and
lower border); I will probably
become a fan of the line configuration tool that allows you
to switch back to linear from a
log axis scale (in the command line there seems to be no
return from plt.semilogy() or
plt.loglog())!

Cheers,

Derek

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of
advanced
analytics on semi-structured data. The platform includes
APIs for building
apps and a phenomenal toolset for data science. Developers
can use
our toolset for easy data analysis & visualization. Get
a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Hi Michiel,

Can you perhaps ask the Fink developers to provide a framework installation of Python? Most matplotlib users who ran into framework-related bugs were Fink users.

I've already looked for that in the list archives and it seems this topic comes up about once a
year when some other package broke with the non-framework build. Changing the build does
not seem a particular problem, but was always declined as it would mean all (or a large number)
of the other Python-dependent packages would have to be fixed at the same time.
But I can of course bring this up for discussion again pointing out that the macosx backend support
is going to be discontinued.

Cheers,
            Derek

···

On 16.04.2013, at 12:03AM, Michiel de Hoon <mjldehoon@...42...> wrote: