New release plan
|
| 1 2 3 |
|
In reply to this post
by Ulli Hafner
Hafner Ullrich wrote:
> I'm not a frequent committer to Hudson core but I would prefer using a > release branch... E.g., I'm working on Hudson not on a daily basis: so > if I have some time it would be sad if I could not check in my changes > just because its end of the week. OK. That's a good point. > Improve the Hudson core stability: > - Since we don't have a testing team the only way I see is to increase the test coverage. > - Maybe we can set up a team of users and plug-in developers that act as kind of beta tester: if a given number of group members confirms that > the release works for their use cases then flag a release as stable and > make it available via update center (otherwise you need to download > manually). We could invent some convention so that willing beta testers can pick up the RC bits and then veto a release. I was going to be the beta tester myself, but it makes sense to open it up for others --- more eyeballs are better. > Improve the interaction testing between Hudson and plugins: > - It would help if we could build AND deploy the plug-ins in a public > Hudson. This should work at least for the reporting plug-ins. (This > would additionally be quite usefule as a live-experience for new > users, better than static Wiki pages:-) I can deploy a selected number of plugins, but automatically deploying every plugin is probably difficult, if that's what you are asking for. > Ulli > >> -----Original Message----- >> From: Kohsuke Kawaguchi [mailto:[hidden email]] >> Sent: Dienstag, 10. M?rz 2009 20:46 >> To: [hidden email] >> Subject: Re: New release plan >> >> Dean Yu wrote: >> > Hi Kohsuke, >> > Is there a reason you don't want to create a branch on Tuesday >> > evening instead of cutting off commits? It seems ironic >> that we don't >> > try to practice CI for building Hudson itself. >> >> Yeah, we should do that. I guess I could automate the branch >> handling on my side, and the merging shouldn't be too >> painful. changelog.html needs to be changed a bit so that we >> can list changes in N+2. >> >> And I think we do practice CI for building Hudson --- there's >> Hudson on Hudson as you can see on >> http://hudson.glassfish.org/job/hudson/ >> >> But my feeling is that our current test coverage is just not >> high enough to catch all the problems this way. >> >> >> > In the end you're our dictator and I'll go along with whatever you >> > decide. :) >> >> -- >> Kohsuke Kawaguchi >> Sun Microsystems >> http://weblogs.java.net/blog/kohsuke/ >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > For additional commands, e-mail: [hidden email] > > -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
smime.p7s (4K) |
|
In reply to this post
by Andrew Bayer
Andrew Bayer wrote:
> This all brings up another question I had - what's the release plan for > plugins? Are plugins only released when the owner of that plugin chooses? Yes. > What if there's no active owner? Right now, we end up with situations where, > for example, the build promotion plugin had a major bug in it that was fixed > over a week ago, but there hasn't been a new release of that plugin yet. > People are actually still submitting new bug reports of that same bug. Sometimes a plugin ends up in a limbo state that doesn't have any 'owner', in which case a problem like that could happen. I don't know how to improve that, although it'd be nice if we could somehow automatically keep track of the state of plugins. As for the promoted build plugin, I just pushed a new release. > On Wed, Mar 11, 2009 at 11:11 AM, Hafner Ullrich > <[hidden email]>wrote: > >> I'm not a frequent committer to Hudson core but I would prefer using a >> release branch... E.g., I'm working on Hudson not on a daily basis: so if I >> have some time it would be sad if I could not check in my changes just >> because its end of the week. >> >> Improve the Hudson core stability: >> - Since we don't have a testing team the only way I see is to increase the >> test coverage. >> - Maybe we can set up a team of users and plug-in developers that act as >> kind of beta tester: if a given number of group members confirms that the >> release works for their use cases then flag a release as stable and make it >> available via update center (otherwise you need to download manually). >> >> Improve the interaction testing between Hudson and plugins: >> - It would help if we could build AND deploy the plug-ins in a public >> Hudson. This should work at least for the reporting plug-ins. (This would >> additionally be quite usefule as a live-experience for new users, better >> than static Wiki pages:-) >> >> Ulli >> >> > -----Original Message----- >> > From: Kohsuke Kawaguchi [mailto:[hidden email]] >> > Sent: Dienstag, 10. März 2009 20:46 >> > To: [hidden email] >> > Subject: Re: New release plan >> > >> > Dean Yu wrote: >> > > Hi Kohsuke, >> > > Is there a reason you don't want to create a branch on Tuesday >> > > evening instead of cutting off commits? It seems ironic >> > that we don't >> > > try to practice CI for building Hudson itself. >> > >> > Yeah, we should do that. I guess I could automate the branch >> > handling on my side, and the merging shouldn't be too >> > painful. changelog.html needs to be changed a bit so that we >> > can list changes in N+2. >> > >> > And I think we do practice CI for building Hudson --- there's >> > Hudson on Hudson as you can see on >> > http://hudson.glassfish.org/job/hudson/ >> > >> > But my feeling is that our current test coverage is just not >> > high enough to catch all the problems this way. >> > >> > >> > > In the end you're our dictator and I'll go along with whatever you >> > > decide. :) >> > >> > -- >> > Kohsuke Kawaguchi >> > Sun Microsystems >> > http://weblogs.java.net/blog/kohsuke/ >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [hidden email] >> For additional commands, e-mail: [hidden email] >> >> > -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
In reply to this post
by Kohsuke Kawaguchi
|
||||||||
|
In reply to this post
by John McNair
On Wed, Mar 11, 2009 at 04:16:25PM -0400, John McNair wrote:
> I only have a handful of commits on Hudson to my credit, but I thought I'd > share a little bit of experience that I don't see mentioned here. Many > people have offered good suggestions. Increasing test coverage and its > measurement is always a good idea. More frequent CI is good. > > But still there is the question of how to use the SCM to its best > advantage. Others have pointed out the problem with quiet periods. A > good SCM should never be off. That is, developers should always be able > to commit _somewhere_. Someone else pointed out the potential pain with > temporary release branches. It can cause merging pain. our use of Git and Hudson, and we do precisely this, reserving master ("trunk") for stable, release quality code, while using "topic branches" as a means of doing high-frequency commits. The problem with this, at least as far as Hudson development goes is that branches are not cheap at all in Subversion. For a developer to create a topic branch in Subversion they would need to execute the `svn copy` and then run `svn checkout` on the new tree, resulting in a whole new working copy, not ideal for the sporadic developers among us. My approach thus far has actually been git-svn with Hudson, such that I'm committing early and often for the random tidbits that I contribute to Hudson and then when those changes are stabilized, do I execute the `git-svn dcommit` and actually sync those changes upstream to the Hudson repository; I'm not saying this is the solution, just saying it's my solution because I'm deathly afraid of destabilizing the tree :-P At Slide we've run through a couple of the tribulations that have been listed on this thread thus far: * Creation of a release branch to cope with a volatile trunk results in large dangerous merges when it comes time to release * Creation of topic branches for bug fixes/features results in large amounts of merge pain (in Subversion at least) especially as the time difference between the branch creation and the most recent sync with trunk grows. (warning, crazy idea inbound!) I think this is a prime opportunity for eating our own dog food, but taking it one step further, based on the release branch suggestion: no manual merges to release! I'm making large assumtopions about the quality and quantity of test coverage Hudson has as a project, but imagine a build was associated with *every* single commit into trunk, i.e. hudson.r1234.war, etc. When the job associated with r1234 finishes executing, the following might happen: SUCCESS: * r1234 is merged automatically into release branch * Tests run on an hourly basis to ensure that release branch remains stable FAILURE: * A very "loud" email is sent to relevant parties (including committer stating the failure of what/when/where * Maybe r1234 is reverted out of trunk? * Trunk is "broken", merges into release branch are ceased for future jobs unless manually okayed. I think we already have support to do everything except the latter two bullet points in the failure case. But I think it would prevent the commit-schedule issues and also help pinpoint exact "problem commits" given the large range of developers committing into trunk. IMHO it solves the stability issues, but also holds Hudson commiters to a high standard and is the appropriate amount of CI applied to a CI-server ;) Cheers -- -R. Tyler Ballance Slide, Inc. |
||||||||
|
In reply to this post
by Kohsuke Kawaguchi
|
||||||||
|
In reply to this post
by R. Tyler Ballance
|
||||||||
|
|
||||||||
|
In reply to this post
by R. Tyler Ballance
|
||||||||
|
In reply to this post
by R. Tyler Ballance
|
||||||||
|
|
In reply to this post
by Ringo De Smet
|
||||||||
|
In reply to this post
by Ringo De Smet
|
||||||||
|
|
In reply to this post
by Cliffano Subagio
Cliffano Subagio wrote:
> How about automated web testing using Selenium / watir / canoo etc? > Various scenarios can be scripted, various combinations of plugins, various > job types, various scm... I'd be happy to expand the test harness if that makes it easier for other people to contribute tests. We do have a test harness right now, which uses HtmlUnit and does the scripted HTTP access. I'm happy enough with this arrangement personally. So the main question for me is, if I spend time to support the web testing framework X, can I expect other people to contribute more tests? If so, it's worth spending time. -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
|
In reply to this post
by Erik Ramfelt
Erik Ramfelt wrote:
> On Thu, Mar 12, 2009 at 6:33 PM, Kohsuke Kawaguchi > <[hidden email]> wrote: >> Hafner Ullrich wrote: >>> >>> I'm not a frequent committer to Hudson core but I would prefer using a >>> release branch... E.g., I'm working on Hudson not on a daily basis: so if I >>> have some time it would be sad if I could not check in my changes just >>> because its end of the week. >> >> OK. That's a good point. > > I agree with Ullrich here, even if it was a year since I worked with core. > >>> Improve the Hudson core stability: >>> - Since we don't have a testing team the only way I see is to increase the >>> test coverage. >>> - Maybe we can set up a team of users and plug-in developers that act as >>> kind of beta tester: if a given number of group members confirms that ?the >>> release works for their use cases then flag a release as stable and ?make it >>> available via update center (otherwise you need to download ?manually). >> >> We could invent some convention so that willing beta testers can pick up the >> RC bits and then veto a release. >> >> I was going to be the beta tester myself, but it makes sense to open it up >> for others --- more eyeballs are better. > > How about adding an option in Hudson configuration where the user can > choose to be notified when there is a new nightly build (?), latest > released or stable version. This way the user can choose when to > update similar to how debian makes it possible to either run a stable > version, a testing version or the latest bleeding edge. > > //Erik > >>> Improve the interaction testing between Hudson and plugins: >>> - It would help if we could build AND deploy the plug-ins in a public >>> ?Hudson. This should work at least for the reporting plug-ins. (This ?would >>> additionally be quite usefule as a live-experience for new ?users, better >>> than static Wiki pages:-) >> >> I can deploy a selected number of plugins, but automatically deploying every >> plugin is probably difficult, if that's what you are asking for. >> >> >>> Ulli >>> >>>> -----Original Message----- >>>> From: Kohsuke Kawaguchi [mailto:[hidden email]] Sent: >>>> Dienstag, 10. M?rz 2009 20:46 >>>> To: [hidden email] >>>> Subject: Re: New release plan >>>> >>>> Dean Yu wrote: >>>> > Hi Kohsuke, >>>> > ? Is there a reason you don't want to create a branch on Tuesday > >>>> > evening instead of cutting off commits? It seems ironic that we don't > try >>>> > to practice CI for building Hudson itself. >>>> >>>> Yeah, we should do that. I guess I could automate the branch handling on >>>> my side, and the merging shouldn't be too painful. changelog.html needs to >>>> be changed a bit so that we can list changes in N+2. >>>> >>>> And I think we do practice CI for building Hudson --- there's Hudson on >>>> Hudson as you can see on http://hudson.glassfish.org/job/hudson/ >>>> >>>> But my feeling is that our current test coverage is just not high enough >>>> to catch all the problems this way. >>>> >>>> >>>> > ? In the end you're our dictator and I'll go along with whatever you >>>> > decide. :) >>>> >>>> -- >>>> Kohsuke Kawaguchi >>>> Sun Microsystems ? ? ? ? ? ? ? ? ? http://weblogs.java.net/blog/kohsuke/ >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [hidden email] >>> For additional commands, e-mail: [hidden email] >>> >>> >> >> >> -- >> Kohsuke Kawaguchi >> Sun Microsystems ? ? ? ? ? ? ? ? ? http://weblogs.java.net/blog/kohsuke/ >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > For additional commands, e-mail: [hidden email] > > -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
|
In reply to this post
by Nigel Magnay
Without arguing the technical benefits of a better version control system, changing the SCM is too big a task to attack now. Between the various automations set around the releases, issue-scm-autolink, and java.net role handling, there's just too much work to do. I think a more gradual approach to Git should be possible, with a gateway so that people can contribute changes to Hudson via Git if they'd like. We could gradually improve Hudson to do automatic testing on all those commits, and perhaps even automate the merging process to various degree. So let's do this slowly, separate from the immediate short term fix. Nigel Magnay wrote: >> >> (warning, crazy idea inbound!) >> >> I think this is a prime opportunity for eating our own dog food, but >> taking it one step further, based on the release branch suggestion: no >> manual merges to release! >> > > FWIW, I don't think that's crazy at all. > > In fact, this is exactly what we've been doing, albeit with the git > plugin. The 'master' build branch pulls in candidate merge topics, > runs the CI build, and commits the merge IFF the build (and the tests) > succeed. > > You could have various branches each with different quality standards > - e.g a releng branch that only accepts and commits merges from > 'master' that pass a full regression test on multiple different > platforms. > > The elephant in the room of course is that SVN just isn't very good at > this. Everyone knows that the 'all commit to trunk' svn model is > horrible (and actually, kudos to people for hudson core *not* being > that unstable) - just as much as they know that branch/merge in svn is > even worse. In particular, I don't think (though I may be wrong) that > you can 'back out' of a merge if you decide it was a bad idea - this > makes the above functionality, which is trivial in git, hard. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [hidden email] > For additional commands, e-mail: [hidden email] > > -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
|
In reply to this post
by Daniel Lindner-2
Daniel Lindner wrote:
> Hafner Ullrich wrote: >> I'm not a frequent committer to Hudson core but I would prefer using a release branch... E.g., I'm working on Hudson not on a daily basis: so if I have some time it would be sad if I could not check in my changes just because its end of the week. >> >> Improve the Hudson core stability: >> - Since we don't have a testing team the only way I see is to increase the test coverage. >> - Maybe we can set up a team of users and plug-in developers that act as kind of beta tester: if a given number of group members confirms that the release works for their use cases then flag a release as stable and make it available via update center (otherwise you need to download manually). >> > I'm no committer to hudson core (yet), so please don't weight my opinion > as much as the active developer's ones. > But i really like the idea of "beta testers". I will happily play this > role with my few hudson installations. Perhaps it is possible to let > each hudson administrator choose in the update center which "flavor" of > hudson (stable, testing, bleeding edge - reminds me of debian a bit) he > wants to obtain. The ones that do beta tests run "testing". Really early > adopters run "bleeding edge". All others run "stable" and have the > confidence of double-checked stability, as the early and beta testers > run real-world hudsons just like them. The only difference between the > user groups would be that testers have earlier access to new features > but are capable of dealing with some bugs. > Perhaps this idea can be integrated with your anticipated branch management. thought :-) -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
|
In reply to this post
by Kohsuke Kawaguchi
Incorporating all the feedbacks, the slightly revised proposal is: - Let's shoot for once-a-week release cycle, on Friday evening pacific time. - trunk is open for commits all the time, just like it is now. We do encourage feature branches for sizable changes, just like how we've been doing that. - I'll cut an RC branch mid-week, and the release will be from this branch. After the release, I merge the branch back to the trunk and delete the branch. And this is just for the core, not the plugins. Release branches only live for a few days, so the branching and merging shouldn't be too bad. Relevant adjustments: - start measuring the code coverage as an incentive for more tests, both for core and plugins. - continuously build the tip of the plugins with the tip of Hudson to detect regressions. - continuously build the last released versions of the plugins with the tip of Hudson to detect regressions. - improve update center so that people can nominate themselves as beta testers for the RC bits. - changelog.html convention needs to be adjusted to work with this model. Looking for the community contributions: - Git/SVN bridge for Hudson's codebase. Kohsuke Kawaguchi wrote: > As you know, lately we are suffering a quality problem in releases. > There are multiple things we need to do to fix that, such as more tests > and early regression testing with plugins, but it's also clear that we > need to adjust the release process. > > So this is the proposal to make that adjustment. > > - Let's shoot for once-a-week release cycle, on Friday evening pacific > time. > > - We allow commits freely in Sat, Sun, Mon, and Tue. Tuesday evening > is RC cut off. > > - We reserve Wed, Thu, and Fri as the quiet period. > We make commits to the core conservatively, and I'll use this period > to deploy RC bits to my production Hudson to be a guinea pig. > > - This should give us more time to write tests. > > > I know there will be all sorts of different ways to do releases, and I > don't want to argue which is better than which. So my main question is, > putting my dictator hat on, is this something that people can live with? > -- Kohsuke Kawaguchi Sun Microsystems http://weblogs.java.net/blog/kohsuke/ |
||||||||
|
|
||||||||
|
|
||||||||
|
|
||||||||
|
In reply to this post
by Kohsuke Kawaguchi
|
||||||||
| Free Embeddable Forum Powered by Nabble | Help |
