## Template for LP Production Meeting logs. Just paste xchat log below and the format IRC line will take care of formatting correctly
#format IRC
<matsubara> #startmeeting 
<MootBot> Meeting started at 10:00. The chair is matsubara.
<MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
<matsubara> [TOPIC] Roll Call 
<MootBot> New Topic:  Roll Call
<matsubara> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! 
<herb`> me
<sinzui> me
<rockstar> me
<matsubara> Ursinha: intellectronica, flacoste: hi
<bigjools> me
<Ursinha> hi, me
<intellectronica> me
<matsubara> stub is excused today. he already sent the dba report to the list
<flacoste> me
<matsubara> henninge: hi
<henninge> me
<henninge> hi mars
<henninge> hi matsubara,  I meant
<matsubara> ok, everyone here.
<matsubara> [TOPIC] Agenda 
<MootBot> New Topic:  Agenda
<matsubara>  * Actions from last meeting
<matsubara>  * Oops report & Critical Bugs 
<matsubara>  * Operations report (mthaddon/herb/spm)
<matsubara>  * DBA report (stub)
<matsubara> [TOPIC] * Actions from last meeting
<MootBot> New Topic:  * Actions from last meeting
<matsubara>     * sinzui to email the list how we should address critical bugs on unmaintained apps (e.g. blueprint)
<matsubara>     * intellectronica to talk to gmb about bug 269538
<matsubara>     * sinzui to take bug 358332
<matsubara>     * sinzui to ask salgado to fix bug 357307
<ubottu> Launchpad bug 269538 in bugzilla-launchpad/bugzilla-3.2 "Compilation error in plugin when authenticating" [Critical,Fix released] https://launchpad.net/bugs/269538
<ubottu> Launchpad bug 358332 in launchpad-registry "[API] OOPS when distribution.getSeries() is called with an invalid name or version" [High,Triaged] https://launchpad.net/bugs/358332
<ubottu> Launchpad bug 357307 in launchpad-foundations "TypeError when creating new account in lpnet" [High,Fix released] https://launchpad.net/bugs/357307
<sinzui> matsubara: done and discussed further at the teamleads meeting
* e-jat has quit (Read error: 104 (Connection reset by peer))
<matsubara> sinzui: good. thanks. what was the outcome?
<intellectronica> matsubara: i talked to gmb and we agreed that the bug is now fix released
<sinzui> matsubara: for the time being, foundations and registry need to do better triage. We want to assign people/teams to these projects in the future to do small development and regular maitenance
<matsubara> all right. the only pending item is #358332 which wasn't fixed for 2.2.4 but it's triaged and set to high. I guess that's going to be fixed for 2.2.5 then?
<sinzui> matsubara: I'm pushing bug 358332 to 2.5 and I will take it as a part of my series work
<ubottu> Launchpad bug 358332 in launchpad-registry "[API] OOPS when distribution.getSeries() is called with an invalid name or version" [High,Triaged] https://launchpad.net/bugs/358332
<matsubara> sinzui: perfect
<matsubara> thanks intellectronica 
<matsubara> thanks sinzui 
<matsubara> let's move on
<matsubara> [TOPIC] * Oops report & Critical Bugs 
<MootBot> New Topic:  * Oops report & Critical Bugs
<matsubara> go ahead Ursinha 
* Ursinha waves
<Ursinha> we have three bugs today, two high and one annoying
<Ursinha> the two high are already being handled: bug 351641 (salgado) and bug 369858 (henninge)
<Ursinha> the annoying is bug 357316, that's bug's team - could someone do something/set the importance on this one, intellectronica?
<ubottu> Launchpad bug 351641 in shipit "OpenID callback view may not find the account that was just created" [High,Triaged] https://launchpad.net/bugs/351641
<ubottu> Launchpad bug 369858 in rosetta "OOPS if new suggestions count change while in +translate page" [High,In progress] https://launchpad.net/bugs/369858
<ubottu> Launchpad bug 357316 in malone "hwdb +submit failing with KeyError OOPS" [Undecided,Triaged] https://launchpad.net/bugs/357316
<Ursinha> thanks ubottu
<intellectronica> Ursinha: sure, will check
* matsubara wants an annoying status
<Ursinha> thanks intellectronica
* Ursinha wants too
<Ursinha> :)
<rockstar> matsubara: aren't all bugs annoying?
<Ursinha> rockstar, that's also true
<Ursinha> that's all from me from the oopses-after-the-rollout land
<Ursinha> matsubara, want to point anything?
<matsubara> we have one in progress critical bug, which flacoste is investigating
<matsubara> related to the shipit + openid thing
<matsubara> bug 368767
<ubottu> Launchpad bug 368767 in launchpad-foundations "Reactivation of shipit causing "Connection timed out" OOPSes" [Critical,In progress] https://launchpad.net/bugs/368767
<matsubara> but that's all
<matsubara> let's move on
<matsubara> thanks everyone
<flacoste> matsubara: actullay, it seems solved
<herb`> shipit is up and running, btw.
<flacoste> matsubara: i'll confirm and update the status
<Ursinha> matsubara, we didn't have more of those (except some 5xx for a period of one minute after the rollout,m about 10)
<matsubara> flacoste: that's great news.
<matsubara> flacoste: thanks!
<herb`> Ursinha: we might have had a burst of 5xxs just before the meeting started.  I haven't had a chance to investigate yet.  but the service seems to be operating fine at the moment.
* herb` is now known as herb
<matsubara> as usual, we'll be checking the OOPS reports all day long and bring up any issues we find to the teams
<Ursinha> herb, I'll check right after the meeting, thanks for the heads up
<matsubara> thanks again
<matsubara> [TOPIC] * Operations report (mthaddon/herb/spm)
<MootBot> New Topic:  * Operations report (mthaddon/herb/spm)
<matsubara> there you go herb 
<herb> 2009-04-23 - The spike in traffic to shipit after the jaunty rollout caused some service disruptions to launchpad. We're still working to isolate the root cause, but have some good ideas and think we have a solid work around.
<herb> 2009-04-24 - Cherry pick of r8270 and r8273 to lpnet* and the internal xmlrpc server.
<herb> 2009-04-29 - Rolled out 2.2.4 to production. Process went relatively smoothly. Because of a bit of short-sightedness read-only wasn't available for this rollout but will be for the next.
* e-jat (n=fenris@ubuntu/member/fenris-) has joined #launchpad-meeting
<herb> Keeping it on the radar: bug 156453 and bug 118625 continue to be pain points for the LOSAs
<herb> With Tom on holiday today and tomorrow, if there is going to be a reroll the code *must* be available by 2030 UTC tomorrow (or it gets pushed to Monday). I would like to be able to have at least an hour post-reroll to make sure there aren't any problems before I EOD.
<ubottu> Launchpad bug 156453 in loggerhead "production loggerhead branch leaks memory" [Critical,In progress] https://launchpad.net/bugs/156453
<ubottu> Launchpad bug 118625 in launchpad-code "codebrowse sometimes hangs" [High,Triaged] https://launchpad.net/bugs/118625
<herb> I already mentioned the 5xx alarm I saw before the meeting. and the fact that shipit is back on as of yesterday around 1900 UTC.
* matsubara has changed the topic to:  Launchpad Meeting Grounds | Channel logs: http://irclogs.ubuntu.com/ | Meeting logs and agenda: https://dev.launchpad.net/MeetingAgenda
<matsubara> herb: most likely that we'll have a re-roll today 
<matsubara> I'll confirm with kiko and keep you posted
<herb> matsubara: there is a config change that needs to land
<herb> matsubara: I don't have a checkout at the moment.  Can I piggyback a diff on someone else's commit?
<matsubara> herb: I think it's ok. bac is working on a fix for a timeout. it might be a good candidate
<herb> matsubara: I will sync up with him.
<matsubara> in any case we can sort that out after the meeting. thanks for bringing it up
<rockstar> herb: hopefully, the loggerhead stuff will start dissipating with the new code.
<herb> rockstar: woohoo!
<matsubara> cool
<matsubara> anything else for herb?
<matsubara> [action] matsubara to add to the CRB page herb's config change
<MootBot> ACTION received:  matsubara to add to the CRB page herb's config change
<matsubara> thanks herb 
<herb> thanks matsubara
<matsubara> [TOPIC] * DBA report (stub)
<MootBot> New Topic:  * DBA report (stub)
<matsubara> so stub is excused today. he sent the dba report to the list
<matsubara> if you have questions, follow up there.
<matsubara> that's all
<matsubara> anything else before I close the meeting?
<matsubara> all right. Thank you all for attending this week's Launchpad Production Meeting. See the channel topic for the location of the logs.
<matsubara> a good job on this release. went really smooth!
<matsubara> #endmeeting