v1.0.x branch seems confused

Hi,

...

Even without the foulup, I think you would see that the merges from
maintenance branches into subsequent branches and into master make it
very hard to figure out what has actually been done on any given branch.

I strongly disagree. The only reason you get cleaner history graphs
with cherry picking is because it doesn't graph the cherry picks! If
you want to know what has been merged, you have to inspect the commit
message in one branch and match it up with the commit message in
another branch. How does that make it easier to figure out what has
been done on any given branch?

I think Eric's point is that it kind of feels (and looks) wrong to
merge maintenance into master, rather than backporting fixes from
master with cherry-picks. Maybe 'feels wrong' might be translatable
as 'harder to think about' and therefore 'more error prone'? I can
see the argument for doing it though. It is a common workflow?

See you,

Matthew

···

On Thu, Jun 2, 2011 at 5:07 PM, Darren Dale <dsdale24@...149...> wrote:

On Thu, Jun 2, 2011 at 7:46 PM, Eric Firing <efiring@...229...> wrote:

Matthew,

Hi,

...

Even without the foulup, I think you would see that the merges from
maintenance branches into subsequent branches and into master make it
very hard to figure out what has actually been done on any given branch.

I strongly disagree. The only reason you get cleaner history graphs
with cherry picking is because it doesn't graph the cherry picks! If
you want to know what has been merged, you have to inspect the commit
message in one branch and match it up with the commit message in
another branch. How does that make it easier to figure out what has
been done on any given branch?

I think Eric's point is that it kind of feels (and looks) wrong to
merge maintenance into master, rather than backporting fixes from
master with cherry-picks. Maybe 'feels wrong' might be translatable
as 'harder to think about' and therefore 'more error prone'?

Maybe "feels wrong" translates to "unfamiliar", and has nothing to do
with difficulty or potential for error. Git is a powerful tool. If we
used a cherry-picking workflow (which I would not advocate), there
would still be chances for error by inappropriately specifying hash
ranges, for example. I think there is much more potential for error
using cherry-picking. There are other advantages as well, see

(note the warnings about rebasing, which are not relevant to this
discussion)

I can
see the argument for doing it though. It is a common workflow?

Yes, I believe it is a very common workflow.

Before we made the git transition, I read about various workflows.
What we are doing now is somewhat similar to what used to be done with
svnmerge. I just googled "git workflow", and found
gitworkflows(7) .
See the section on "Merging Upwards":

···

On Thu, Jun 2, 2011 at 8:17 PM, Matthew Brett <matthew.brett@...149...> wrote:

On Thu, Jun 2, 2011 at 5:07 PM, Darren Dale <dsdale24@...149...> wrote:

On Thu, Jun 2, 2011 at 7:46 PM, Eric Firing <efiring@...229...> wrote:

---
The "downwards graduation" discussed above cannot be done by actually
merging downwards, however, since that would merge all changes on the
unstable branch into the stable one. Hence the following:

Rule: Merge upwards
Always commit your fixes to the oldest supported branch that require
them. Then (periodically) merge the integration branches upwards into
each other.

This gives a very controlled flow of fixes. If you notice that you
have applied a fix to e.g. master that is also required in maint, you
will need to cherry-pick it (using git-cherry-pick(1)) downwards. This
will happen a few times and is nothing to worry about unless you do it
very frequently.
---

With cherry picking, imagine we get ready to cut a v1.0.2 release, and
we want to know if all of the bug fixes that should be applied to the
maintenance branch have been applied. How do we verify? Much easier is
to apply them like the docs at kernel.org suggest, to the oldest
supported branch that require them and then merge upwards. Then the
history graph can tell you that the bug fixes in older versions have
been applied to newer branches.

Merging is going to happen anyway, whenever someone files a pull
request. We have a good workflow, we just had a small mistake and have
now overcome it.

Darren

Hi,

Matthew,

Hi,

...

Even without the foulup, I think you would see that the merges from
maintenance branches into subsequent branches and into master make it
very hard to figure out what has actually been done on any given branch.

I strongly disagree. The only reason you get cleaner history graphs
with cherry picking is because it doesn't graph the cherry picks! If
you want to know what has been merged, you have to inspect the commit
message in one branch and match it up with the commit message in
another branch. How does that make it easier to figure out what has
been done on any given branch?

I think Eric's point is that it kind of feels (and looks) wrong to
merge maintenance into master, rather than backporting fixes from
master with cherry-picks. Maybe 'feels wrong' might be translatable
as 'harder to think about' and therefore 'more error prone'?

Maybe "feels wrong" translates to "unfamiliar", and has nothing to do
with difficulty or potential for error.

Ah no - I mean that the way I think most of think of bugfix workflow
is that we fix in trunk and backport the fix to the maintenance
branch. Here though you are fixing in the maintenance branch and
_merging_ to trunk. The counter-argument is 'well think of it the
other way round and it will be fine'. It's a little difficult to know
if that's true I suppose.

It's a funny kind of thing too, because it puts an extra constraint on
the bugfix. For example, imagine a bugfix in maintenance, in some
code that has crazy-changed in trunk. Now you have to decide if you
try and merge it anyway, and basically do a rewrite as you fix the
merge conflict. Or you could merge-ours on that commit and write a
completely different fix on trunk. But that's getting a bit dark and
magic. And so it imposes some slight what-will-happen brain overhead
on the person writing the fix.

Git is a powerful tool. If we
used a cherry-picking workflow (which I would not advocate), there
would still be chances for error by inappropriately specifying hash
ranges, for example. I think there is much more potential for error
using cherry-picking. There are other advantages as well, see
Git Cherry-pick vs Merge Workflow - Stack Overflow
(note the warnings about rebasing, which are not relevant to this
discussion)

I can
see the argument for doing it though. It is a common workflow?

Yes, I believe it is a very common workflow.

Ah - OK, I guess I had not seen it before.

Before we made the git transition, I read about various workflows.
What we are doing now is somewhat similar to what used to be done with
svnmerge. I just googled "git workflow", and found
gitworkflows(7) .
See the section on "Merging Upwards":

Yes, I think I see why it would be attractive.

With cherry picking, imagine we get ready to cut a v1.0.2 release, and
we want to know if all of the bug fixes that should be applied to the
maintenance branch have been applied. How do we verify? Much easier is
to apply them like the docs at kernel.org suggest, to the oldest
supported branch that require them and then merge upwards. Then the
history graph can tell you that the bug fixes in older versions have
been applied to newer branches.

I believe that the git log --cherry etc machinery is designed to deal
with this case, but I haven't used it myself. I have to emphasize,
the projects I've been involved with have fewer developers and smaller
code-bases,

Cheers,

Matthew

···

On Thu, Jun 2, 2011 at 7:06 PM, Darren Dale <dsdale24@...149...> wrote:

On Thu, Jun 2, 2011 at 8:17 PM, Matthew Brett <matthew.brett@...149...> wrote:

On Thu, Jun 2, 2011 at 5:07 PM, Darren Dale <dsdale24@...149...> wrote:

On Thu, Jun 2, 2011 at 7:46 PM, Eric Firing <efiring@...229...> wrote:

Responding to a few of the points mentioned above – definitely +1 on more releases. I have been the bottleneck here and can do more. With that in mind, let’s plan on getting a trunk release out with a 2 week timeline or so.

I think there is a solid case for a release branch, so we can test/refine/fix while allowing development in the trunk. Once it is released, one suggestion is to can keep it around as maintenance but only for release critical bugs so that if we discover a show stopper three weeks after the release, we can cut a bugfix release w/ a shorted round of testing. Perhaps if we limit it to release critical bugs, this will lower the development burden on maintaining it and merging it.

On the animations issue, I have noticed one strange behavior when I tried to remove the gtk extension code and replace it with pure python and found the new animations API behaved strangely (but not the old) under the new code. I never committed my changes because I could not resolve whether it was my code or Ryan’s that was causing the problem, and he wasn’t sure either. But incompletely baked features (and I wouldn’t even call the animations code incompletely baked) are OK for releases. It’s better to get them out there and in use so our users can find and fix the bugs :slight_smile:

JDH

···

On Thu, Jun 2, 2011 at 9:06 PM, Darren Dale <dsdale24@…149…> wrote:

Before we made the git transition, I read about various workflows.

What we are doing now is somewhat similar to what used to be done with

svnmerge. I just googled “git workflow”, and found

http://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html .

See the section on “Merging Upwards”:

debian has been pretty clear that they want to build their packages from tarballs which we have uploaded and announced.

JDH

···

On Wed, Jun 1, 2011 at 2:58 PM, Eric Firing <efiring@…706…29…> wrote:

Do the packagers use the tip of the maintenance branch, or do they use

the most recent release? If the former, then that bumps up the priority

of keeping such a branch. If the latter, it bumps up the priority of

having frequent high-quality releases, regardless of what they are called.

Before we made the git transition, I read about various workflows.
What we are doing now is somewhat similar to what used to be done with
svnmerge. I just googled "git workflow", and found
gitworkflows(7) .
See the section on "Merging Upwards":

Responding to a few of the points mentioned above -- definitely +1 on more
releases. I have been the bottleneck here and can do more. With that in
mind, let's plan on getting a trunk release out with a 2 week timeline or
so.

That would be great. I am eager to merge the py3 stuff back into the
main repository and start thinking about a 1.2 release.

I think there is a solid case for a release branch, so we can
test/refine/fix while allowing development in the trunk. Once it is
released, one suggestion is to can keep it around as maintenance but only
for release critical bugs so that if we discover a show stopper three weeks
after the release, we can cut a bugfix release w/ a shorted round of
testing. Perhaps if we limit it to release critical bugs, this will lower
the development burden on maintaining it and merging it.

Sounds like a good idea.

···

On Fri, Jun 3, 2011 at 4:39 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, Jun 2, 2011 at 9:06 PM, Darren Dale <dsdale24@...149...> wrote:

Well if you need to rip it out/"temporarily" break it to improve the
gtk backend, by all means do so. I'll be bogged down for a few more
months, after which I'll be able to work more on it. (FINALLY.) The
only reason I even checked it in originally was so that others could
play, but I was (and still am) not ready to commit completely to the
API.

Ryan

···

On Fri, Jun 3, 2011 at 3:39 AM, John Hunter <jdh2358@...149...> wrote:

On the animations issue, I have noticed one strange behavior when I tried to
remove the gtk extension code and replace it with pure python and found the
new animations API behaved strangely (but not the old) under the new code.
I never committed my changes because I could not resolve whether it was my
code or Ryan's that was causing the problem, and he wasn't sure either. But
incompletely baked features (and I wouldn't even call the animations code
incompletely baked) are OK for releases. It's better to get them out there
and in use so our users can find and fix the bugs :slight_smile:

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma

Well if you need to rip it out/“temporarily” break it to improve the

gtk backend, by all means do so. I’ll be bogged down for a few more

months, after which I’ll be able to work more on it. (FINALLY.) The

only reason I even checked it in originally was so that others could

play, but I was (and still am) not ready to commit completely to the

API.

Not at all, I think it is at least as likely that there is a problem with my gtk code rather than a problem in the animations API. I only brought this up as an example of how I would rather release mostly functioning code with a few possible warts than incubate it in the trunk for months. What you have done is so much better than what we have that it needs to get out there even if we have to fix or change the API later.

JDH

The problem is that there aren’t a lot of people playing with this code, and people are still using the documentation’s examples for animations. Maybe we ought to consider a mechanism to release it with the next release with a huge “experimental” sticker on it, somehow?

Ben Root

···

On Fri, Jun 3, 2011 at 8:50 AM, Ryan May <rmay31@…149…> wrote:

On Fri, Jun 3, 2011 at 3:39 AM, John Hunter <jdh2358@…149…> wrote:

On the animations issue, I have noticed one strange behavior when I tried to

remove the gtk extension code and replace it with pure python and found the

new animations API behaved strangely (but not the old) under the new code.

I never committed my changes because I could not resolve whether it was my

code or Ryan’s that was causing the problem, and he wasn’t sure either. But

incompletely baked features (and I wouldn’t even call the animations code

incompletely baked) are OK for releases. It’s better to get them out there

and in use so our users can find and fix the bugs :slight_smile:

Well if you need to rip it out/“temporarily” break it to improve the

gtk backend, by all means do so. I’ll be bogged down for a few more

months, after which I’ll be able to work more on it. (FINALLY.) The

only reason I even checked it in originally was so that others could

play, but I was (and still am) not ready to commit completely to the

API.

Ryan

Why not just call it a release candidate and follow a similar process for it as numpy just did a short while ago?

Plus, with regards to timing. Do we want to release before or after the upcoming SciPy conference in July? Depending on who will be there (unfortunately, I won’t be), we might want to wait for after that conference to take advantage of any activity then.

Also, I know I have repeatedly stated that I won’t be a release manager, but I am going to put together a wish-list of mine for what I would like to see for the next release. Maybe with that as a base, we can come up with a final set of goals for v1.1.0 and determine how much time it would take to get there?

Ben Root

···

On Fri, Jun 3, 2011 at 3:39 AM, John Hunter <jdh2358@…149…> wrote:

On Thu, Jun 2, 2011 at 9:06 PM, Darren Dale <dsdale24@…55…149…> wrote:

Before we made the git transition, I read about various workflows.

What we are doing now is somewhat similar to what used to be done with

svnmerge. I just googled “git workflow”, and found

http://www.kernel.org/pub/software/scm/git/docs/gitworkflows.html .

See the section on “Merging Upwards”:

Responding to a few of the points mentioned above – definitely +1 on more releases. I have been the bottleneck here and can do more. With that in mind, let’s plan on getting a trunk release out with a 2 week timeline or so.

I think there is a solid case for a release branch, so we can test/refine/fix while allowing development in the trunk. Once it is released, one suggestion is to can keep it around as maintenance but only for release critical bugs so that if we discover a show stopper three weeks after the release, we can cut a bugfix release w/ a shorted round of testing. Perhaps if we limit it to release critical bugs, this will lower the development burden on maintaining it and merging it.

If we have the resources, it might be better to release before, and
encourage activity at scipy to focus on py3 support (assuming it has
been merged into master by then).

···

On Fri, Jun 3, 2011 at 10:04 AM, Benjamin Root <ben.root@...553...> wrote:

On Fri, Jun 3, 2011 at 3:39 AM, John Hunter <jdh2358@...149...> wrote:

On Thu, Jun 2, 2011 at 9:06 PM, Darren Dale <dsdale24@...149...> wrote:

Before we made the git transition, I read about various workflows.
What we are doing now is somewhat similar to what used to be done with
svnmerge. I just googled "git workflow", and found
gitworkflows(7) .
See the section on "Merging Upwards":

Responding to a few of the points mentioned above -- definitely +1 on
more releases. I have been the bottleneck here and can do more. With that
in mind, let's plan on getting a trunk release out with a 2 week timeline or
so.

I think there is a solid case for a release branch, so we can
test/refine/fix while allowing development in the trunk. Once it is
released, one suggestion is to can keep it around as maintenance but only
for release critical bugs so that if we discover a show stopper three weeks
after the release, we can cut a bugfix release w/ a shorted round of
testing. Perhaps if we limit it to release critical bugs, this will lower
the development burden on maintaining it and merging it.

Why not just call it a release candidate and follow a similar process for it
as numpy just did a short while ago?

Plus, with regards to timing. Do we want to release before or after the
upcoming SciPy conference in July? Depending on who will be there
(unfortunately, I won't be), we might want to wait for after that conference
to take advantage of any activity then.

Plus, with regards to timing. Do we want to release before or after the upcoming SciPy conference in July? Depending on who will be there (unfortunately, I won’t be), we might want to wait for after that conference to take advantage of any activity then.

I don’t feel the need to wait – things move slowly enought that we might get one release out in the next couple of weeks and one bugfix release 2-4 weeks after that and that will dovetail with scipy conference. Also, my June is a lot better than my July as I will be traveling in July for the first couple of weeks.

Also, I know I have repeatedly stated that I won’t be a release manager, but I am going to put together a wish-list of mine for what I would like to see for the next release. Maybe with that as a base, we can come up with a final set of goals for v1.1.0 and determine how much time it would take to get there?

This might be a better idea for a 1.2 release, since this sounds like it will take a while. There has been a lot of progress in the trunk since 1.0.1 and with the release-early-release-often philosophy I’d rather get something out there and then if we want to do something more formalized for the next iteration we will have plenty of time for it. I would shoot for people to get their changes in for a 1.1 release by early next week, branch and cut a release candidate, test for a week and put in bug fixes, cut a second release candidate a week later. If people would like more time for a formal process and bug closing extravaganza, I’m fine with that too.

JDH

···

On Fri, Jun 3, 2011 at 9:04 AM, Benjamin Root <ben.root@…553…> wrote: