## Template for LP Production Meeting logs. Just paste xchat log below and the format IRC line will take care of formatting correctly
#format IRC
<matsubara> #startmeeting
<MootBot> Meeting started at 10:00. The chair is matsubara.
<MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
<matsubara> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. 
<matsubara> [TOPIC] Roll Call 
<MootBot> New Topic:  Roll Call
<herb> me
<matsubara> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! 
<henninge> me
<Ursinha> me (just watching)
<intellectronica> me
<matsubara> sinzui, rockstar, bigjools: hi
<rockstar> ni!
<matsubara> apologies from sprinters: stub and flacoste
<sinzui> me
<bigjools> yarp
<matsubara> [TOPIC] Agenda 
<MootBot> New Topic:  Agenda
<matsubara>  * Actions from last meeting
<matsubara>  * Oops report & Critical Bugs & Broken scripts
<matsubara>  * Operations report (mthaddon/herb/spm)
<matsubara>  * DBA report (stub)
<matsubara> [TOPIC] * Actions from last meeting
<MootBot> New Topic:  * Actions from last meeting
<matsubara> * matsubara to chase mwhudson/jml about failure on updatebranches script
<matsubara> * stub to delegate bug 354593 to ISD
<matsubara> * sinzui to ask barry to fix bug 403606
<matsubara> * ursinha to chase mars about OOPS-1307J16 and file a bug about it
<matsubara> * stub to request CP for his branch that fixes oops logging
<matsubara> * henninge to investigate rosetta-poimport script failure on the Aug 5th and report back to the list
<ubottu> Launchpad bug 354593 in canonical-identity-provider "SSO exceptions views need proper branding" [High,Triaged] https://launchpad.net/bugs/354593
<ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1307J16
<matsubara> I haven't chased jml about updatebranches script. it haven't failed in a while, so I think it's fine to remove the action item
<Ursinha> matsubara, I suck, I need to do that
<rockstar> matsubara, I'm pretty sure mwhudson was looking at it before it left.  I'd be surprised if he left without it working.
<matsubara> [action] * ursinha to chase mars about OOPS-1307J16 and file a bug about it
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1307J16
<MootBot> ACTION received:  * ursinha to chase mars about OOPS-1307J16 and file a bug about it
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1307J16
<matsubara> I think both stub action items were done
<matsubara> henninge, any news about the failure in the rosetta-poimport script?
<sinzui> matsubara: barry has not had time to look into bug 403606, He had a more pressing bug to work on
<ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
<matsubara> sinzui, will barry have time to fix it for 2.2.8? this is one of the most frequent oops in the summaries
<henninge> matsubara: There were several problems on that day and I had meant to get back to you and ask which one you meant ... ;-)
<henninge> matsubara: but they are all fixed now .
<sinzui> matsubara: I cannot say. We do not know what is wrong
<matsubara> henninge, cool. thanks
<matsubara> sinzui, all right. please keep it in your radar
<matsubara> I think that's all for the action items from last week
<matsubara> thanks everyone
<matsubara> [TOPIC] * Oops report & Critical Bugs & Broken scripts
<MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
<matsubara> Today we have a few OOPS for bigjools and sinzui
<matsubara> bigjools, OOPS-1315A253
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1315A253
<bigjools>  /o\
<matsubara> sinzui, OOPS-1318S626 AssertionError using the API to create a ProductRelease
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318S626
<matsubara> sinzui, OOPS-1316EA164 TraversalError adding a release
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1316EA164
<bigjools> matsubara: why is that a soyuz oops?
<matsubara> sinzui, OOPS-1321EB223 and OOPS-1318EA4 are the same bug (or going to be fixed by the same fix) for bug 402725?
<ubottu> Launchpad bug 402725 in launchpad-registry "Delete series does not untarget series-targeted blueprints" [Low,Triaged] https://launchpad.net/bugs/402725
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1321EB223
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318EA4
<matsubara> bigjools, look at the URL failing in when %s is used as the search term
<bigjools> matsubara: look at the URL on the left, not the search term :)
<matsubara> looks like the ppa search OOPSes when you pass %s
<sinzui> matsubara: one oops is  hacked url. We can make the form not render if the release already exists
<matsubara> bigjools, well, users are trying to use the global search and they're getting an OOPS because the ppa search can't handle %s 
<sinzui> matsubara: the other oops is from the baltix owner trying to do his job. The code that lets him do his job will not be on edge until saturday
<matsubara> [action] matsubara to file a bug for OOPS-1315A253
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1315A253
<MootBot> ACTION received:  matsubara to file a bug for OOPS-1315A253
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1315A253
<bigjools> matsubara: I don't see where there's a ppa search going on.  Someone is pasting a URL into the search box aren't they?
<matsubara> bigjools, nope, I was able to reproduce by just searching for %s in the global search
<bigjools> matsubara: ok let's talk after the meeting
<gary_poster> me :-)
<matsubara> perhaps the global search also uses the ppa search internally?
<gary_poster> I'm here for foundations
<bigjools> well not that I am aware of
<bigjools> global search uses google
* cprov is now known as cprov-lunch
<matsubara> bigjools, ok. we can talk after the meeting and then I'll file a bug in the correct project
<bigjools> okidoki
<matsubara> gary_poster, hi and welcome! :-)
<gary_poster> thanks :-)
<matsubara> sinzui, which one is going to be rolled out to edge on saturday?
* mbarnett (n=mbarnett@cpe-66-25-160-175.austin.res.rr.com) has joined #launchpad-meeting
<matsubara> sinzui, the traversalerror or the assertionerror?
<matsubara> sinzui, how about the IntegrityErrors?
<sinzui> matsubara: traversal error
<sinzui> matsubara: if you have not created bugs, I will so that I can track this
<matsubara> [action] sinzui to file bugs for OOPS-1318S626, OOPS-1321EB223 and OOPS-1318EA4
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318S626
<MootBot> ACTION received:  sinzui to file bugs for OOPS-1318S626, OOPS-1321EB223 and OOPS-1318EA4
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1321EB223
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318EA4
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318S626
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1321EB223
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318EA4
<matsubara> sinzui, what do you mean by not render the form? isn't the user using the api?
<sinzui> matsubara: you are correct, I am mistaken
<sinzui> I am not sure how to deal with that then.
<matsubara> sinzui, right. let's have bugs reported to those and then discuss in the bug tracker.
<matsubara> sinzui, thanks
<matsubara> We have two fix committed critical bugs
<matsubara> one which have been CP already
<matsubara> the other one is bug 389541
<ubottu> Launchpad bug 389541 in launchpad-foundations "Oops re-assigning bug to launchpad-registry project" [Critical,Fix committed] https://launchpad.net/bugs/389541
<matsubara> sinzui, are you going to request a CP for that one?
<matsubara> we had two script failures this week: garbo-hourly and librarian-gc
<matsubara> garbo-hourly has been fixed
<matsubara> no news about the librarian-gc
<sinzui> matsubara: I know nothing about that bug.
<matsubara> gary_poster, can you check why the librarian-gc failed and reply to the failure email sent to the internal list?
<gary_poster> matsubara: yes
* leonardr has quit ("Leaving")
<matsubara> sinzui, actually that's fix released already
<gary_poster> matsubara: (re librarian-gc, I'll ask flacoste or you for details later; I assume he has them)
<matsubara> [action] gary_poster to chase librarian-gc failure and report back to the list
<MootBot> ACTION received:  gary_poster to chase librarian-gc failure and report back to the list
<matsubara> gary_poster, right. ping me after the meeting and I can trawl some logs to find out the failure.
<gary_poster> matsubara: cool thanks
<matsubara> I think that's all for this section
<matsubara> thanks everyone
<matsubara> [TOPIC] * Operations report (mthaddon/herb/spm)
<MootBot> New Topic:  * Operations report (mthaddon/herb/spm)
<herb> 2009-08-12 - We had no fewer than 8 cherry picks land.
<herb> Codebrowse (loggerhead) has regressed and is leaking memory again. I've re-opened bug #156453.
<herb> Despite our request to review the High & Triaged bugs on https://bugs.edge.launchpad.net/~canonical-losas, we haven't seen an update on any of them except the mailman issue (thanks barry).
<ubottu> Launchpad bug 156453 in loggerhead "production loggerhead branch leaks memory" [Critical,Confirmed] https://launchpad.net/bugs/156453
<herb> There is currently 1 cherry pick request that has been approved but failed testing (possibly spuriously) and 2 requests awaiting approval.
<herb> Finally, I wanted to mention that this will be my last meeting as a LOSA. Starting next week I will be moving over to a GSA role. But in return you'll get two new LOSAs who will both be more than capable of taking up what little slack is left by my transition. Chex will be moving over to join the LOSA team. And on Monday we get another new addition to the LOSA team as mbarnett starts.
<herb> Obviously I'll still be around, but since I won't be interfacing with all of you as directly as I do now, I wanted to take this opportunity to say thank you to everyone for your diligence, attention to detail and frankly all the hard work you all do to improve Launchpad and keep it running. So, thank you.
<mthaddon> we'll miss you, herb... /o\
<gary_poster> thank you herb, you'll be missed
<matsubara> rockstar, can you take a look at the codebrowse memory leak issue?
* bigjools shakes herb's hand
<rockstar> matsubara, yeah, we've been doing a perpetual "take a look" at the memory leak.
<matsubara> rockstar, looks like it got worse recently
<rockstar> matsubara, yeah, looks like it.
<matsubara> gary_poster, intellectronica, rockstar: please look at the list of high and triaged bugs: https://bugs.edge.launchpad.net/~canonical-losas and see what can be done for 2.2.8 and 2.2.9
<gary_poster> matsubara, will do.
<intellectronica> matsubara: tbh, i looked last week already, and the conclusions was that pretty much nothing can be done for now :(
<matsubara> please either target them to a release or at least give some feedback to the losas in the report if those bugs can't be fixed before 2.2.9
<herb> even an update saying, we won't get to this before october is helpful.
<herb> then we know we can ping you in october about it. :)
<gary_poster> heh
<matsubara> intellectronica, right. if you say so in the bug report we'll have better expectation management :-)
<intellectronica> herb: sure, makes sense. also we then know to look again after 3.0 is out and we start scheduling new bugs again
<matsubara> perfect. thanks everyone
<herb> thanks everybody
<matsubara> herb, thank you for all your hard work. wish you luck in your new role!
<herb> ta
<matsubara> [TOPIC] * DBA report (stub)
<MootBot> New Topic:  * DBA report (stub)
<matsubara> stub is sprint so i'll ask him to email the dba report to the list
<matsubara> [action] matsubara to ask stub to email the dba report to the list
<MootBot> ACTION received:  matsubara to ask stub to email the dba report to the list
<matsubara> and I think that's all for today
<Ursinha> nope
<matsubara> anything else before I close?
<Ursinha> I have something to ask
<matsubara> go ahead
<Ursinha> we're doing benchmark tests on basic navigation in Launchpad
<Ursinha> so, I've asked bigjools already, I wanted to know from you pages that are most likely to have timeouts
<Ursinha> rockstar, I'm already benching /launchpad and /launchpad/+activereviews
<Ursinha> rockstar, do you have any other suggestions?
<Ursinha> henninge, hi :) do you have suggestions for rosetta as well?
<matsubara> Ursinha, code.launchpad.net/launchpad/
<Ursinha> matsubara, as I told rockstar, I'm doing that already :)
<rockstar> Ursinha, no, I think you've got them then.
<bigjools> Ursinha: you got my suggestion, +queue ?
<Ursinha> matsubara, but thanks for the suggestion, do you have any others?
<Ursinha> bigjools, I took note, yes
<matsubara> there was a timeout on person code pages but that's been fixed by tim recently
<bigjools> cool
<Ursinha> thanks!
<Ursinha> matsubara, I know we had on +branches also, but tim fixed as well
<matsubara> I think that's the one I meant, IIRC
<henninge> Ursinha: For rosetta: POFile:+translate and DistroSeriesLanguage:+index
<Ursinha> intellectronica, what do you think?
<Ursinha> henninge, awesome, thanks!
<Ursinha> thanks rockstar
<matsubara> Ursinha, and look in the oops summaries if you're out of ideas
<intellectronica> Ursinha: don't know of the top of my head. we still have problems with bugs with many subscribers (though not the whole page times out)
<Ursinha> intellectronica, the portlet does?
<intellectronica> Ursinha: exactly
<Ursinha> maybe benching only the portlet
<Ursinha> intellectronica, can you take a quick look and tell me later about the timeouts you'd want me to bench, please?
<Ursinha> we're doing this today
<intellectronica> Ursinha: actually, i know there are some timeouts in +filebug lately. that's potentially quite serious
<Ursinha> intellectronica, I already created a test for /ubuntu/+filebug
<intellectronica> cool
<Ursinha> great
<Ursinha> sinzui, on registry side, do you have any suggestions?
<matsubara> Ursinha, bug 408457 is the one about the portlet timingout
<ubottu> Launchpad bug 408457 in malone "Launchpad unable to show subscribers for bug with 400 duplicates" [High,Triaged] https://launchpad.net/bugs/408457
<bigjools> Ursinha: oh one more, there's an XHR request for the PPA repo size that frequently times out
<Ursinha> matsubara, thanks
<sinzui> Ursinha: I have no suggestions
<Ursinha> sinzui, maybe milestone page...?
<matsubara> there's an open bug for timeouts on adminteammerge
<matsubara> but I guess it's not the kind of page you're looking for
<Ursinha> matsubara, I'm looking for pages of common usage
<sinzui> Ursinha: I have hacked that enough. The correct fix for that is to use ajax to retrieve rows in batches of 50. The rows are already engineered to do that in fact. The Javascript will take time away from 3.0, and my team is way behind because of other distractions
<Ursinha> thanks again
<Ursinha> okay sinzui, thanks
<Ursinha> that's fine then
<Ursinha> bigjools, I took note of the XHR one
<Ursinha> thanks
<Ursinha> that
<Ursinha> that's all from me
<matsubara> okay. thanks Ursinha 
<Ursinha> thanks matsubara and all
<matsubara> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. 
<matsubara> #endmeeting 
<MootBot> Meeting finished at 10:45.