## Template for LP Production Meeting logs. Just paste xchat log below and the format IRC line will take care of formatting correctly
#format IRC
<matsubara> #startmeeting
<MootBot> Meeting started at 10:00. The chair is matsubara.
<MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
<matsubara> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. 
<matsubara> [TOPIC] Roll Call 
<MootBot> New Topic:  Roll Call
<matsubara> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! 
<Ursinha-afk> me
<gary_poster> me
<intellectronica> me
<barry> me
* Ursinha-afk is now known as Ursinha
<rockstar> ni!
<bigjools> me
<intellectronica> Ursinha: and i thought you're doing your telepathic typing trick again...
<Ursinha> intellectronica, hehe
<matsubara> Chex, hi
<Chex> matsubara: hello
<matsubara> ok, so we're missing stub and henning/danilo
<danilos> me
<matsubara> hi danilos 
<matsubara> :-)
<danilos> matsubara: no you are not :)
<matsubara> so, from now on, you'll be attending the prod meeting, right?
<danilos> matsubara: well, not necessarily, let's see :)
<matsubara> ok, let's move on then
<matsubara> [TOPIC] Agenda 
<danilos> matsubara: but soon I will be replacing henninge as our QA contact
<MootBot> New Topic:  Agenda
<matsubara>  * Actions from last meeting
<matsubara>  * Oops report & Critical Bugs & Broken scripts
<matsubara>  * Operations report (mthaddon/Chex/spm/mbarnett)
<matsubara>  * DBA report (stub)
<matsubara>  * Proposed items
<matsubara> danilos, ok
<matsubara> [TOPIC] * Actions from last meeting
<MootBot> New Topic:  * Actions from last meeting
<matsubara>   * cprov to follow up on bug 413749
<matsubara>   * stub to investigate garbo-hourly failure after spm adjusted script checking to 12h
<matsubara>   * sinzui to poke barry about ExpatError OOPSes (bug 403606)
<matsubara>   * danilos, Ursinha and matsubara to discuss oops summaries split per team
<ubottu> Bug 413749 on http://launchpad.net/bugs/413749 is private
<ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
<matsubara> ok, I know that stub replied to the garbo-hourly failure email, so I think that's done
<danilos> matsubara: we kind of did that, flacoste is involved as well, and it's all on track I'd say :)
<matsubara> danilos, right. if you have more suggestions let me know
<matsubara> barry, hi!
<danilos> sinzui is on unavailable I believe
* barry has some thoughts about the expat errors
<barry> matsubara: hi!
<matsubara> barry, what's up with those expat errors?
<cprov> matsubara: I did reply on 413749 as well, needs discussion, though.
<matsubara> thanks cprov 
* noodles775 has quit (Read error: 110 (Connection timed out))
<Ursinha> matsubara, the one with the expaterrors I'd bring in the oops section
* noodles775_ (n=miken@f053074246.adsl.alicedsl.de) has joined #launchpad-meeting
<barry> matsubara: well, some more investigation might be warranted, but they are intermittent and i've always chalked them up to problems with xmlrpc.  i believe they can be safely ignored though because the system will "catch up" the next time mailman talks to xmlrpc server.
<Ursinha> barry, here's the problem, they happen every day
<barry> matsubara: ultimately, we should get rid of the xmlrpc stuff.  it was a solution to a problem we no longer have (namely gpl pollution between mailman and launchpad)
<matsubara> if it's safe to ignore them, can we add some code to catch the exception and not log the oops?
* stub (n=stub@canonical/launchpad/stub) has joined #launchpad-meeting
<matsubara> hi stub 
<Ursinha> you said in the bug report that we have only a handful a week, but I see them everyday, and their numbers are the highest count
<barry> Ursinha: i get many oops reports for mailman that are clean.  am i looking at the wrong reports?
<Ursinha> barry, I think so :)
<Ursinha> look at the lpnet reports
<barry> Ursinha: meaning the daily emails
<barry> hmm.  okay.  i see them only rarely in the mailman daily email.  i will double check next time i see the report, but i think ignoring the error is the right thing to do
<Ursinha> barry, ok. it floods the summary, and that's annoying
<barry> Ursinha: gotcha!  i hadn't realized that.  i'll do something about it then
<gary_poster> When I discussed hiding the problems, given the frequency, Francis suggested that we not hide them but investigate a bit.  Maybe a small timebox would be appropriate?
<Ursinha> thaaaanks muchly barry!
<gary_poster> eh, nm
<gary_poster> :-)
<Ursinha> gary_poster, :)
<barry> gary_poster: yes, we can do that.  it will take losa intervention and some cowboy debugging, but it's probably worth it
<gary_poster> ok cool
<matsubara> [action] barry to continue debug on bug 403606
<MootBot> ACTION received:  barry to continue debug on bug 403606
<ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
<matsubara> thanks barry, gary_poster, Ursinha 
<Ursinha> thanks
<matsubara> let's move on
<matsubara> [TOPIC] * Oops report & Critical Bugs & Broken scripts
<MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
<Ursinha> ok
<matsubara> Ursinha, take the stage please
<Ursinha> so the ExpatErrors were already discussed here
<Ursinha> I have only one bug
<Ursinha> intellectronica, bug 408738
<ubottu> Launchpad bug 408738 in malone "OOPS when rendering bug activity" [Undecided,Triaged] https://launchpad.net/bugs/408738
<Ursinha> that's another one that happens pretty often
<intellectronica> really? if so then we should at least recover gracefully from it
<Ursinha> intellectronica, I see those everyday as well
<intellectronica> i mean in addition to figuring out why we have records like those
<Ursinha> I agree
<Ursinha> intellectronica, could you take a look or poke someone to do so, please?
<intellectronica> yes yes
<Ursinha> thanks intellectronica :)
<Ursinha> matsubara, [action] intellectronica to take a look or find someone to take a look on bug 408738
<ubottu> Launchpad bug 408738 in malone "OOPS when rendering bug activity" [Undecided,Triaged] https://launchpad.net/bugs/408738
<matsubara> [action] intellectronica to take a look or find someone to take a look on bug 408738
<MootBot> ACTION received:  intellectronica to take a look or find someone to take a look on bug 408738
<Ursinha> thanks :)
<matsubara> thanks Ursinha and intellectronica 
<matsubara> we have 2 critical bugs
<Ursinha> critical bugs are fix committed, and the process-pending-packagediffs script failure was already explained by cprov
<matsubara> both fix committed
<Ursinha> so, we're done!
<Ursinha> you can move on, matsubara
<Ursinha> thanks guys!
<matsubara> thanks everyone
<matsubara> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
<MootBot> New Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)
<Chex> hello everyone :
<Ursinha> hi Chex :)
<Chex> the report for this week:
<Chex> Lots more cherry picks this week
<Chex> More and more problems with codebrowse (per the Incident Log)
<Chex> New losas getting up to speed (Chex and mbarnett)
<Chex> Testing tarmac for oops-tools project - making progress but not quite there yet
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=tools
<Chex> Will begin process of implementing recommendations from SplitIt Sprint before too long
<Chex> One of our top priority bugs is bug 416990 - others per https://bugs.launchpad.net/~canonical-losas
<Chex> That's it unless there are any questions
<ubottu> Launchpad bug 416990 in bzr "Memory usage of codehosting processes is excessive" [High,Confirmed] https://launchpad.net/bugs/416990
<matsubara> Chex, re: tarmac setup for oops-tools, I'll submit a merge proposal to tarmac today with the remaining fixes needed.
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=tools
<matsubara> Chex, I'll coordinate with mthaddon later on
<mthaddon> cool
<Chex> matsubara: ok, great
<matsubara> rockstar, can you poke people in your team about the codehosting memory issue?
<danilos> matsubara: is it worth fixing ubottu not to choke on oops-tools mention? ;)
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=tools
<Ursinha> lol
<matsubara> I can ask the maintainer
<danilos> matsubara: thanks :)
<matsubara> [action] matsubara to chase ubottu maintainer to fix regexp used to identify oopses
<MootBot> ACTION received:  matsubara to chase ubottu maintainer to fix regexp used to identify oopses
<matsubara> let's move on
<matsubara> thanks Chex
<matsubara> [TOPIC] * DBA report (stub)
<MootBot> New Topic:  * DBA report (stub)
<stub> The long running transaction killer is back set to 3 hours, thanks to the Rosetta team sorting out the language pack export issue quickly.
<stub> oot
<Ursinha> that was fast
<matsubara> hehe
<bigjools> like mrevell, it was short
<matsubara> thanks stub and rosetta people :-)
<matsubara> lol
<Ursinha> lol
<matsubara> [TOPIC] * Proposed items
<MootBot> New Topic:  * Proposed items
<matsubara> we have no proposed items
<mrevell> haha bigjools
<matsubara> anything else before I close?
<matsubara> hi mrevell 
<matsubara> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
<mrevell> matsubara: nothing from me, just responding to bigjools... :)
<matsubara> #endmeeting 
<MootBot> Meeting finished at 10:24.