New release plan
|
| 1 2 3 |
|
As you know, lately we are suffering a quality problem in releases. There are multiple things we need to do to fix that, such as more tests and early regression testing with plugins, but it's also clear that we need to adjust the release process. So this is the proposal to make that adjustment. - Let's shoot for once-a-week release cycle, on Friday evening pacific time. - We allow commits freely in Sat, Sun, Mon, and Tue. Tuesday evening is RC cut off. - We reserve Wed, Thu, and Fri as the quiet period. We make commits to the core conservatively, and I'll use this period to deploy RC bits to my production Hudson to be a guinea pig. - This should give us more time to write tests. I know there will be all sorts of different ways to do releases, and I don't want to argue which is better than which. So my main question is, putting my dictator hat on, is this something that people can live with? -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
smime.p7s (4K)
|
|
||||||||
|
|
Peter Reilly wrote:
> For plugins or just for hudson core? Just Hudson core. > > Peter > > On Tue, Mar 10, 2009 at 7:05 PM, Kohsuke Kawaguchi > <[hidden email]> wrote: >> >> As you know, lately we are suffering a quality problem in releases. There >> are multiple things we need to do to fix that, such as more tests and early >> regression testing with plugins, but it's also clear that we need to adjust >> the release process. >> >> So this is the proposal to make that adjustment. >> >> - Let's shoot for once-a-week release cycle, on Friday evening pacific >> ?time. >> >> - We allow commits freely in Sat, Sun, Mon, and Tue. Tuesday evening >> ?is RC cut off. >> >> - We reserve Wed, Thu, and Fri as the quiet period. >> ?We make commits to the core conservatively, and I'll use this period >> ?to deploy RC bits to my production Hudson to be a guinea pig. >> >> - This should give us more time to write tests. >> >> >> I know there will be all sorts of different ways to do releases, and I don't >> want to argue which is better than which. So my main question is, putting my >> dictator hat on, is this something that people can live with? >> >> -- >> Kohsuke Kawaguchi >> Sun Microsystems ? ? ? ? ? ? ? ? ? http://weblogs.java.net/blog/kohsuke/ >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > For additional commands, e-mail: [hidden email] > > -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
|
||||||||
|
|
Eric Lefevre-Ardant wrote:
> What about translations in the core? would we put the same restraints? > Personally, I am very inconsistent in the delays between 2 commits to the > French translation; sometimes a couple of days, sometimes one month. I don't > see myself paying much attention to days when I should commit. Not that I > don't want to. I can't think of any past incident where l10n/i18n commits resulted in a problematic release, so I think you should consider the repository open for commits all the time. (And if do a release branch, this point will be moot anyway) > Eric > > 2009/3/10 Kohsuke Kawaguchi <[hidden email]> > >> Peter Reilly wrote: >> >>> For plugins or just for hudson core? >>> >> >> Just Hudson core. >> >> >> >>> Peter >>> >>> On Tue, Mar 10, 2009 at 7:05 PM, Kohsuke Kawaguchi >>> <[hidden email]> wrote: >>> >>>> >>>> As you know, lately we are suffering a quality problem in releases. There >>>> are multiple things we need to do to fix that, such as more tests and >>>> early >>>> regression testing with plugins, but it's also clear that we need to >>>> adjust >>>> the release process. >>>> >>>> So this is the proposal to make that adjustment. >>>> >>>> - Let's shoot for once-a-week release cycle, on Friday evening pacific >>>> ?time. >>>> >>>> - We allow commits freely in Sat, Sun, Mon, and Tue. Tuesday evening >>>> ?is RC cut off. >>>> >>>> - We reserve Wed, Thu, and Fri as the quiet period. >>>> ?We make commits to the core conservatively, and I'll use this period >>>> ?to deploy RC bits to my production Hudson to be a guinea pig. >>>> >>>> - This should give us more time to write tests. >>>> >>>> >>>> I know there will be all sorts of different ways to do releases, and I >>>> don't >>>> want to argue which is better than which. So my main question is, putting >>>> my >>>> dictator hat on, is this something that people can live with? >>>> >>>> -- >>>> Kohsuke Kawaguchi >>>> Sun Microsystems ? ? ? ? ? ? ? ? ? http://weblogs.java.net/blog/kohsuke/ >>>> >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [hidden email] >>> For additional commands, e-mail: [hidden email] >>> >>> >>> >> >> -- >> Kohsuke Kawaguchi >> Sun Microsystems http://weblogs.java.net/blog/kohsuke/ >> > -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
In reply to this post
by Eric Lefevre-Ardant
|
||||||||
|
|
||||||||
|
|
|
||||||||
|
|
||||||||
|
|
In reply to this post
by BrandonB
BrandonB wrote:
> Whether the decision is to go with a "quiet period" or branches, I make the > following suggestions: > > 1) Add code coverage checks (Cobertura or Clover) to the > http://hudson.glassfish.org/job/hudson/ job, against the existing tests the > Hudson core already has. This should identify weaknesses in coverage, and > prompt the team/users to start contributing more tests. Yes, I need to do this. > 2) Before a Hudson core release is made, compile the core against the latest > release of all plugins. Wherever possible, run the plugin unit tests against > the new core. If, for whatever reason, the hudson.glassfish.org server > cannot run plugin unit tests (eg, it might be hard to run the AccuRev plugin > tests without AccuRev server or licenses), notify (automatically?) the > plugin owners to run their unit tests against the about-to-be-released core. +1. > 3) Do any of the plugins have Hudson jobs on a public Hudson instance? If > not, can they be added to hudson.glassfish.org? If so, can they be moved > there? Currently there's a single build that builds the entire Hudson including all the plugins. If we create one job per plugin, I need a new feature in Hudson to manage that. > 4) All plugins should have code coverage (Cobertura or Clover) checks > against their unit tests. Right, I believe we get this with the same effort to do your #1. > 5) When the core release is made, publish a document (somewhere, wiki, > release notes, part of the changelog, don't know) stating which versions of > plugins were tested against the new core. I believe some of the plugins > already document which versions of the core *they* were compiled/tested > against when they are released - we need the reverse. Isn't it better to just bake that information in Hudson itself or update center metadata, so that the user gets a reassuring "tested" icon or something on UC? > 6) In fact, including the information from #5 in the 'Updates' and/or > 'Available' tabs of the 'Manage Plugins' table would be extremely useful. > (I'm envisioning another column stating something to the effect of "This > version of Hudson was tested againstplugin release #X.Y.) This can help > people identify whether they *should* upgrade plugins, instead of just > crossing their fingers and upgrading blindly (or manually comparing release > dates of core and plugins). OK, so you've already thought about that, then. -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
In reply to this post
by Kohsuke Kawaguchi
|
||||||||
|
On Wed, Mar 11, 2009 at 04:16:25PM -0400, John McNair wrote:
> I only have a handful of commits on Hudson to my credit, but I thought I'd > share a little bit of experience that I don't see mentioned here. Many > people have offered good suggestions. Increasing test coverage and its > measurement is always a good idea. More frequent CI is good. > > But still there is the question of how to use the SCM to its best > advantage. Others have pointed out the problem with quiet periods. A > good SCM should never be off. That is, developers should always be able > to commit _somewhere_. Someone else pointed out the potential pain with > temporary release branches. It can cause merging pain. our use of Git and Hudson, and we do precisely this, reserving master ("trunk") for stable, release quality code, while using "topic branches" as a means of doing high-frequency commits. The problem with this, at least as far as Hudson development goes is that branches are not cheap at all in Subversion. For a developer to create a topic branch in Subversion they would need to execute the `svn copy` and then run `svn checkout` on the new tree, resulting in a whole new working copy, not ideal for the sporadic developers among us. My approach thus far has actually been git-svn with Hudson, such that I'm committing early and often for the random tidbits that I contribute to Hudson and then when those changes are stabilized, do I execute the `git-svn dcommit` and actually sync those changes upstream to the Hudson repository; I'm not saying this is the solution, just saying it's my solution because I'm deathly afraid of destabilizing the tree :-P At Slide we've run through a couple of the tribulations that have been listed on this thread thus far: * Creation of a release branch to cope with a volatile trunk results in large dangerous merges when it comes time to release * Creation of topic branches for bug fixes/features results in large amounts of merge pain (in Subversion at least) especially as the time difference between the branch creation and the most recent sync with trunk grows. (warning, crazy idea inbound!) I think this is a prime opportunity for eating our own dog food, but taking it one step further, based on the release branch suggestion: no manual merges to release! I'm making large assumtopions about the quality and quantity of test coverage Hudson has as a project, but imagine a build was associated with *every* single commit into trunk, i.e. hudson.r1234.war, etc. When the job associated with r1234 finishes executing, the following might happen: SUCCESS: * r1234 is merged automatically into release branch * Tests run on an hourly basis to ensure that release branch remains stable FAILURE: * A very "loud" email is sent to relevant parties (including committer stating the failure of what/when/where * Maybe r1234 is reverted out of trunk? * Trunk is "broken", merges into release branch are ceased for future jobs unless manually okayed. I think we already have support to do everything except the latter two bullet points in the failure case. But I think it would prevent the commit-schedule issues and also help pinpoint exact "problem commits" given the large range of developers committing into trunk. IMHO it solves the stability issues, but also holds Hudson commiters to a high standard and is the appropriate amount of CI applied to a CI-server ;) Cheers -- -R. Tyler Ballance Slide, Inc. |
||||||||
|
|
||||||||
|
In reply to this post
by R. Tyler Ballance
|
||||||||
|
|
||||||||
|
In reply to this post
by Ringo De Smet
|
||||||||
|
|
In reply to this post
by R. Tyler Ballance
|
||||||||
|
|
||||||||
|
|
In reply to this post
by Nigel Magnay
Without arguing the technical benefits of a better version control system, changing the SCM is too big a task to attack now. Between the various automations set around the releases, issue-scm-autolink, and java.net role handling, there's just too much work to do. I think a more gradual approach to Git should be possible, with a gateway so that people can contribute changes to Hudson via Git if they'd like. We could gradually improve Hudson to do automatic testing on all those commits, and perhaps even automate the merging process to various degree. So let's do this slowly, separate from the immediate short term fix. Nigel Magnay wrote: >> >> (warning, crazy idea inbound!) >> >> I think this is a prime opportunity for eating our own dog food, but >> taking it one step further, based on the release branch suggestion: no >> manual merges to release! >> > > FWIW, I don't think that's crazy at all. > > In fact, this is exactly what we've been doing, albeit with the git > plugin. The 'master' build branch pulls in candidate merge topics, > runs the CI build, and commits the merge IFF the build (and the tests) > succeed. > > You could have various branches each with different quality standards > - e.g a releng branch that only accepts and commits merges from > 'master' that pass a full regression test on multiple different > platforms. > > The elephant in the room of course is that SVN just isn't very good at > this. Everyone knows that the 'all commit to trunk' svn model is > horrible (and actually, kudos to people for hudson core *not* being > that unstable) - just as much as they know that branch/merge in svn is > even worse. In particular, I don't think (though I may be wrong) that > you can 'back out' of a merge if you decide it was a bad idea - this > makes the above functionality, which is trivial in git, hard. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > For additional commands, e-mail: [hidden email] > > -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
In reply to this post
by Kohsuke Kawaguchi
|
||||||||
| Free Embeddable Forum Powered by Nabble | Help |
