1 <matsubara> #startmeeting
   2 <MootBot> Meeting started at 10:00. The chair is matsubara.
   3 <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
   4 <matsubara> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. 
   5 <matsubara> [TOPIC] Roll Call 
   6 <MootBot> New Topic:  Roll Call
   7 <herb> me
   8 <matsubara> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! 
   9 <henninge> me
  10 <Ursinha> me (just watching)
  11 <intellectronica> me
  12 <matsubara> sinzui, rockstar, bigjools: hi
  13 <rockstar> ni!
  14 <matsubara> apologies from sprinters: stub and flacoste
  15 <sinzui> me
  16 <bigjools> yarp
  17 <matsubara> [TOPIC] Agenda 
  18 <MootBot> New Topic:  Agenda
  19 <matsubara>  * Actions from last meeting
  20 <matsubara>  * Oops report & Critical Bugs & Broken scripts
  21 <matsubara>  * Operations report (mthaddon/herb/spm)
  22 <matsubara>  * DBA report (stub)
  23 <matsubara> [TOPIC] * Actions from last meeting
  24 <MootBot> New Topic:  * Actions from last meeting
  25 <matsubara> * matsubara to chase mwhudson/jml about failure on updatebranches script
  26 <matsubara> * stub to delegate bug 354593 to ISD
  27 <matsubara> * sinzui to ask barry to fix bug 403606
  28 <matsubara> * ursinha to chase mars about OOPS-1307J16 and file a bug about it
  29 <matsubara> * stub to request CP for his branch that fixes oops logging
  30 <matsubara> * henninge to investigate rosetta-poimport script failure on the Aug 5th and report back to the list
  31 <ubottu> Launchpad bug 354593 in canonical-identity-provider "SSO exceptions views need proper branding" [High,Triaged] https://launchpad.net/bugs/354593
  32 <ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
  33 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1307J16
  34 <matsubara> I haven't chased jml about updatebranches script. it haven't failed in a while, so I think it's fine to remove the action item
  35 <Ursinha> matsubara, I suck, I need to do that
  36 <rockstar> matsubara, I'm pretty sure mwhudson was looking at it before it left.  I'd be surprised if he left without it working.
  37 <matsubara> [action] * ursinha to chase mars about OOPS-1307J16 and file a bug about it
  38 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1307J16
  39 <MootBot> ACTION received:  * ursinha to chase mars about OOPS-1307J16 and file a bug about it
  40 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1307J16
  41 <matsubara> I think both stub action items were done
  42 <matsubara> henninge, any news about the failure in the rosetta-poimport script?
  43 <sinzui> matsubara: barry has not had time to look into bug 403606, He had a more pressing bug to work on
  44 <ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
  45 <matsubara> sinzui, will barry have time to fix it for 2.2.8? this is one of the most frequent oops in the summaries
  46 <henninge> matsubara: There were several problems on that day and I had meant to get back to you and ask which one you meant ... ;-)
  47 <henninge> matsubara: but they are all fixed now .
  48 <sinzui> matsubara: I cannot say. We do not know what is wrong
  49 <matsubara> henninge, cool. thanks
  50 <matsubara> sinzui, all right. please keep it in your radar
  51 <matsubara> I think that's all for the action items from last week
  52 <matsubara> thanks everyone
  53 <matsubara> [TOPIC] * Oops report & Critical Bugs & Broken scripts
  54 <MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
  55 <matsubara> Today we have a few OOPS for bigjools and sinzui
  56 <matsubara> bigjools, OOPS-1315A253
  57 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1315A253
  58 <bigjools>  /o\
  59 <matsubara> sinzui, OOPS-1318S626 AssertionError using the API to create a ProductRelease
  60 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318S626
  61 <matsubara> sinzui, OOPS-1316EA164 TraversalError adding a release
  62 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1316EA164
  63 <bigjools> matsubara: why is that a soyuz oops?
  64 <matsubara> sinzui, OOPS-1321EB223 and OOPS-1318EA4 are the same bug (or going to be fixed by the same fix) for bug 402725?
  65 <ubottu> Launchpad bug 402725 in launchpad-registry "Delete series does not untarget series-targeted blueprints" [Low,Triaged] https://launchpad.net/bugs/402725
  66 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1321EB223
  67 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318EA4
  68 <matsubara> bigjools, look at the URL failing in when %s is used as the search term
  69 <bigjools> matsubara: look at the URL on the left, not the search term :)
  70 <matsubara> looks like the ppa search OOPSes when you pass %s
  71 <sinzui> matsubara: one oops is  hacked url. We can make the form not render if the release already exists
  72 <matsubara> bigjools, well, users are trying to use the global search and they're getting an OOPS because the ppa search can't handle %s 
  73 <sinzui> matsubara: the other oops is from the baltix owner trying to do his job. The code that lets him do his job will not be on edge until saturday
  74 <matsubara> [action] matsubara to file a bug for OOPS-1315A253
  75 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1315A253
  76 <MootBot> ACTION received:  matsubara to file a bug for OOPS-1315A253
  77 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1315A253
  78 <bigjools> matsubara: I don't see where there's a ppa search going on.  Someone is pasting a URL into the search box aren't they?
  79 <matsubara> bigjools, nope, I was able to reproduce by just searching for %s in the global search
  80 <bigjools> matsubara: ok let's talk after the meeting
  81 <gary_poster> me :-)
  82 <matsubara> perhaps the global search also uses the ppa search internally?
  83 <gary_poster> I'm here for foundations
  84 <bigjools> well not that I am aware of
  85 <bigjools> global search uses google
  86 * cprov is now known as cprov-lunch
  87 <matsubara> bigjools, ok. we can talk after the meeting and then I'll file a bug in the correct project
  88 <bigjools> okidoki
  89 <matsubara> gary_poster, hi and welcome! :-)
  90 <gary_poster> thanks :-)
  91 <matsubara> sinzui, which one is going to be rolled out to edge on saturday?
  92 * mbarnett (n=mbarnett@cpe-66-25-160-175.austin.res.rr.com) has joined #launchpad-meeting
  93 <matsubara> sinzui, the traversalerror or the assertionerror?
  94 <matsubara> sinzui, how about the IntegrityErrors?
  95 <sinzui> matsubara: traversal error
  96 <sinzui> matsubara: if you have not created bugs, I will so that I can track this
  97 <matsubara> [action] sinzui to file bugs for OOPS-1318S626, OOPS-1321EB223 and OOPS-1318EA4
  98 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318S626
  99 <MootBot> ACTION received:  sinzui to file bugs for OOPS-1318S626, OOPS-1321EB223 and OOPS-1318EA4
 100 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1321EB223
 101 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318EA4
 102 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318S626
 103 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1321EB223
 104 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1318EA4
 105 <matsubara> sinzui, what do you mean by not render the form? isn't the user using the api?
 106 <sinzui> matsubara: you are correct, I am mistaken
 107 <sinzui> I am not sure how to deal with that then.
 108 <matsubara> sinzui, right. let's have bugs reported to those and then discuss in the bug tracker.
 109 <matsubara> sinzui, thanks
 110 <matsubara> We have two fix committed critical bugs
 111 <matsubara> one which have been CP already
 112 <matsubara> the other one is bug 389541
 113 <ubottu> Launchpad bug 389541 in launchpad-foundations "Oops re-assigning bug to launchpad-registry project" [Critical,Fix committed] https://launchpad.net/bugs/389541
 114 <matsubara> sinzui, are you going to request a CP for that one?
 115 <matsubara> we had two script failures this week: garbo-hourly and librarian-gc
 116 <matsubara> garbo-hourly has been fixed
 117 <matsubara> no news about the librarian-gc
 118 <sinzui> matsubara: I know nothing about that bug.
 119 <matsubara> gary_poster, can you check why the librarian-gc failed and reply to the failure email sent to the internal list?
 120 <gary_poster> matsubara: yes
 121 * leonardr has quit ("Leaving")
 122 <matsubara> sinzui, actually that's fix released already
 123 <gary_poster> matsubara: (re librarian-gc, I'll ask flacoste or you for details later; I assume he has them)
 124 <matsubara> [action] gary_poster to chase librarian-gc failure and report back to the list
 125 <MootBot> ACTION received:  gary_poster to chase librarian-gc failure and report back to the list
 126 <matsubara> gary_poster, right. ping me after the meeting and I can trawl some logs to find out the failure.
 127 <gary_poster> matsubara: cool thanks
 128 <matsubara> I think that's all for this section
 129 <matsubara> thanks everyone
 130 <matsubara> [TOPIC] * Operations report (mthaddon/herb/spm)
 131 <MootBot> New Topic:  * Operations report (mthaddon/herb/spm)
 132 <herb> 2009-08-12 - We had no fewer than 8 cherry picks land.
 133 <herb> Codebrowse (loggerhead) has regressed and is leaking memory again. I've re-opened bug #156453.
 134 <herb> Despite our request to review the High & Triaged bugs on https://bugs.edge.launchpad.net/~canonical-losas, we haven't seen an update on any of them except the mailman issue (thanks barry).
 135 <ubottu> Launchpad bug 156453 in loggerhead "production loggerhead branch leaks memory" [Critical,Confirmed] https://launchpad.net/bugs/156453
 136 <herb> There is currently 1 cherry pick request that has been approved but failed testing (possibly spuriously) and 2 requests awaiting approval.
 137 <herb> Finally, I wanted to mention that this will be my last meeting as a LOSA. Starting next week I will be moving over to a GSA role. But in return you'll get two new LOSAs who will both be more than capable of taking up what little slack is left by my transition. Chex will be moving over to join the LOSA team. And on Monday we get another new addition to the LOSA team as mbarnett starts.
 138 <herb> Obviously I'll still be around, but since I won't be interfacing with all of you as directly as I do now, I wanted to take this opportunity to say thank you to everyone for your diligence, attention to detail and frankly all the hard work you all do to improve Launchpad and keep it running. So, thank you.
 139 <mthaddon> we'll miss you, herb... /o\
 140 <gary_poster> thank you herb, you'll be missed
 141 <matsubara> rockstar, can you take a look at the codebrowse memory leak issue?
 142 * bigjools shakes herb's hand
 143 <rockstar> matsubara, yeah, we've been doing a perpetual "take a look" at the memory leak.
 144 <matsubara> rockstar, looks like it got worse recently
 145 <rockstar> matsubara, yeah, looks like it.
 146 <matsubara> gary_poster, intellectronica, rockstar: please look at the list of high and triaged bugs: https://bugs.edge.launchpad.net/~canonical-losas and see what can be done for 2.2.8 and 2.2.9
 147 <gary_poster> matsubara, will do.
 148 <intellectronica> matsubara: tbh, i looked last week already, and the conclusions was that pretty much nothing can be done for now :(
 149 <matsubara> please either target them to a release or at least give some feedback to the losas in the report if those bugs can't be fixed before 2.2.9
 150 <herb> even an update saying, we won't get to this before october is helpful.
 151 <herb> then we know we can ping you in october about it. :)
 152 <gary_poster> heh
 153 <matsubara> intellectronica, right. if you say so in the bug report we'll have better expectation management :-)
 154 <intellectronica> herb: sure, makes sense. also we then know to look again after 3.0 is out and we start scheduling new bugs again
 155 <matsubara> perfect. thanks everyone
 156 <herb> thanks everybody
 157 <matsubara> herb, thank you for all your hard work. wish you luck in your new role!
 158 <herb> ta
 159 <matsubara> [TOPIC] * DBA report (stub)
 160 <MootBot> New Topic:  * DBA report (stub)
 161 <matsubara> stub is sprint so i'll ask him to email the dba report to the list
 162 <matsubara> [action] matsubara to ask stub to email the dba report to the list
 163 <MootBot> ACTION received:  matsubara to ask stub to email the dba report to the list
 164 <matsubara> and I think that's all for today
 165 <Ursinha> nope
 166 <matsubara> anything else before I close?
 167 <Ursinha> I have something to ask
 168 <matsubara> go ahead
 169 <Ursinha> we're doing benchmark tests on basic navigation in Launchpad
 170 <Ursinha> so, I've asked bigjools already, I wanted to know from you pages that are most likely to have timeouts
 171 <Ursinha> rockstar, I'm already benching /launchpad and /launchpad/+activereviews
 172 <Ursinha> rockstar, do you have any other suggestions?
 173 <Ursinha> henninge, hi :) do you have suggestions for rosetta as well?
 174 <matsubara> Ursinha, code.launchpad.net/launchpad/
 175 <Ursinha> matsubara, as I told rockstar, I'm doing that already :)
 176 <rockstar> Ursinha, no, I think you've got them then.
 177 <bigjools> Ursinha: you got my suggestion, +queue ?
 178 <Ursinha> matsubara, but thanks for the suggestion, do you have any others?
 179 <Ursinha> bigjools, I took note, yes
 180 <matsubara> there was a timeout on person code pages but that's been fixed by tim recently
 181 <bigjools> cool
 182 <Ursinha> thanks!
 183 <Ursinha> matsubara, I know we had on +branches also, but tim fixed as well
 184 <matsubara> I think that's the one I meant, IIRC
 185 <henninge> Ursinha: For rosetta: POFile:+translate and DistroSeriesLanguage:+index
 186 <Ursinha> intellectronica, what do you think?
 187 <Ursinha> henninge, awesome, thanks!
 188 <Ursinha> thanks rockstar
 189 <matsubara> Ursinha, and look in the oops summaries if you're out of ideas
 190 <intellectronica> Ursinha: don't know of the top of my head. we still have problems with bugs with many subscribers (though not the whole page times out)
 191 <Ursinha> intellectronica, the portlet does?
 192 <intellectronica> Ursinha: exactly
 193 <Ursinha> maybe benching only the portlet
 194 <Ursinha> intellectronica, can you take a quick look and tell me later about the timeouts you'd want me to bench, please?
 195 <Ursinha> we're doing this today
 196 <intellectronica> Ursinha: actually, i know there are some timeouts in +filebug lately. that's potentially quite serious
 197 <Ursinha> intellectronica, I already created a test for /ubuntu/+filebug
 198 <intellectronica> cool
 199 <Ursinha> great
 200 <Ursinha> sinzui, on registry side, do you have any suggestions?
 201 <matsubara> Ursinha, bug 408457 is the one about the portlet timingout
 202 <ubottu> Launchpad bug 408457 in malone "Launchpad unable to show subscribers for bug with 400 duplicates" [High,Triaged] https://launchpad.net/bugs/408457
 203 <bigjools> Ursinha: oh one more, there's an XHR request for the PPA repo size that frequently times out
 204 <Ursinha> matsubara, thanks
 205 <sinzui> Ursinha: I have no suggestions
 206 <Ursinha> sinzui, maybe milestone page...?
 207 <matsubara> there's an open bug for timeouts on adminteammerge
 208 <matsubara> but I guess it's not the kind of page you're looking for
 209 <Ursinha> matsubara, I'm looking for pages of common usage
 210 <sinzui> Ursinha: I have hacked that enough. The correct fix for that is to use ajax to retrieve rows in batches of 50. The rows are already engineered to do that in fact. The Javascript will take time away from 3.0, and my team is way behind because of other distractions
 211 <Ursinha> thanks again
 212 <Ursinha> okay sinzui, thanks
 213 <Ursinha> that's fine then
 214 <Ursinha> bigjools, I took note of the XHR one
 215 <Ursinha> thanks
 216 <Ursinha> that
 217 <Ursinha> that's all from me
 218 <matsubara> okay. thanks Ursinha 
 219 <Ursinha> thanks matsubara and all
 220 <matsubara> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. 
 221 <matsubara> #endmeeting 
 222 <MootBot> Meeting finished at 10:45.