1 <Ursinha> #startmeeting
   2 <Ursinha> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. 
   3 <MootBot> Meeting started at 11:02. The chair is Ursinha.
   4 <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
   5 <rockstar> "Gimp's sleepin'" "Well wake him up"
   6 <Ursinha> [TOPIC] Roll Call
   7 <Ursinha> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! 
   8 <MootBot> New Topic:  Roll Call
   9 <sinzui> me
  10 * rockstar  
  11 <Ursinha> me
  12 <Ursinha> and all the others that already meed
  13 <Ursinha> stub is excused
  14 <Ursinha> gary_poster, allenap, hello
  15 <allenap> me
  16 <danilos> me (maybe I get counted twice)
  17 <gary_poster> Ursinha: on call but sort of here
  18 <Ursinha> :)
  19 <Ursinha> gary_poster, ok, maybe matsubara can join us in your behalf?
  20 <Ursinha> I'm moving on, will poke you when it's time :)
  21 <Ursinha> [TOPIC] Agenda
  22 <Ursinha>  * Actions from last meeting
  23 <Ursinha>  * Oops report & Critical Bugs & Broken scripts
  24 <Ursinha>  * Operations report (mthaddon/Chex/spm/mbarnett)
  25 <Ursinha>  * DBA report (stub)
  26 <Ursinha>  * QA stats of the week
  27 <Ursinha>  * Proposed items
  28 <MootBot> New Topic:  Agenda
  29 <Ursinha> [TOPIC] * Actions from last meeting
  30 <Ursinha> * matsubara to file a bug for OOPS-1553EC1332
  31 <Ursinha>     * Filed https://bugs.edge.launchpad.net/launchpad-code/+bug/558597
  32 <Ursinha> * matsubara to chase critical bug 556839 with noodles or someone from soyuz
  33 <Ursinha>     * chatted with noodles about this one. Jelmer is investigating.
  34 <Ursinha> * matsubara chase someone from soyuz about expire-ppa-files failing script
  35 <Ursinha>     * chatted with noodles about this one and he's going to reply to the failure email
  36 <Ursinha> * gary_poster to investigate OOPS-1557EA572 and re-open or file a new bug for it
  37 <Ursinha>     * Not needed. it's a code issue (bug 558585)
  38 <Ursinha> * matsubara to file a bug on launchpad-code about OOPS-1557EA572 including
  39 <Ursinha>   details from salgado ("the OOPS above is different from the one I'd seen when
  40 <Ursinha>   fixing the bug.  that's in the view that generates merge-proposal diffs.  
  41 <Ursinha>   that view should either use StreamOrRedirectLibraryFileAliasView (which knows
  42 <Ursinha>   what to do when the librarian is down) or do something similar to that")
  43 <Ursinha>     * Done: https://bugs.edge.launchpad.net/launchpad-code/+bug/558585
  44 <Ursinha> * matsubara to talk to foundations about launchpadlib untriaged items
  45 <Ursinha>     * Done: talked to Gary about this and triaged a few bugs.
  46 <MootBot> New Topic:  * Actions from last meeting
  47 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1553EC1332
  48 <ubottu> Launchpad bug 558597 in launchpad-code "OOPS using person branch feed" [High,Triaged]
  49 <ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/556839)
  50 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1557EA572
  51 <ubottu> Launchpad bug 558585 in launchpad-code "branch merge proposal diff page should handle librarian outage better" [High,Triaged] https://launchpad.net/bugs/558585
  52 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1557EA572
  53 <Ursinha> it seems all bugs were filed and oopses sorted out
  54 <ubottu> Launchpad bug 558585 in launchpad-code "branch merge proposal diff page should handle librarian outage better" [High,Triaged]
  55 <Ursinha> [TOPIC] * Oops report & Critical Bugs & Broken scripts
  56 <MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
  57 <Ursinha> all relevant bugs were discussed with people during this week; the most offenders are bug 553361 (~1200 oopses a day, targetted to 10.04) and bug 54005 (we're not sure this is the real one, given it's old)
  58 <Ursinha> bug 54005 (OOPS-1564CEMAIL1) is hard to debug given the almost empty oops report, but this emptyness is bug 230106, which deryck said won't be fixed now but discussed soon
  59 <Ursinha> we're having other email related oops, besides the OOPS-1564CEMAIL1 one: OOPS-1563CEMAIL101, to which I filed bug 563055, and gary_poster already triaged them
  60 <Ursinha> sinzui, do you know what's going on in this oops? OOPS-1564B963
  61 <ubottu> Launchpad bug 553361 in launchpad-foundations "OOPS when accessing launchpad with no referrer, eg. via wget" [High,Triaged] https://launchpad.net/bugs/553361
  62 <ubottu> Launchpad bug 54005 in launchpad-foundations "bugs.launchpad.ubuntu.com email ending up in Launchpad mailbox" [Low,Triaged] https://launchpad.net/bugs/54005
  63 <ubottu> Launchpad bug 230106 in malone "emails interface oops reports need better error messages" [Low,Triaged] https://launchpad.net/bugs/230106
  64 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1564CEMAIL1
  65 <ubottu> Launchpad bug 563055 in launchpad-foundations "Mail parsing failing with ValueError" [Low,Triaged] https://launchpad.net/bugs/563055
  66 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1564CEMAIL1
  67 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1563CEMAIL101
  68 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1564B963
  69 <Ursinha> sinzui, hi :)
  70 <sinzui> Ursinha, it looks like an insane file name
  71 * matsubara (~matsubara@canonical/launchpad/matsubara) has joined #launchpad-meeting
  72 <Ursinha> sinzui, is it possible to upload files with insane names?
  73 <sinzui> I do not know
  74 <sinzui> nunitv2
  75 <sinzui> has one it seems
  76 <allenap> Ursinha: I've now also commented on bug 230106, https://bugs.edge.launchpad.net/malone/+bug/230106/comments/3
  77 <ubottu> Launchpad bug 230106 in malone "emails interface oops reports need better error messages" [Low,Triaged]
  78 <ubottu> Launchpad bug 230106 in malone "emails interface oops reports need better error messages" [Low,Triaged] https://launchpad.net/bugs/230106
  79 * sinzui looks on site
  80 <Ursinha> thanks allenap 
  81 <Ursinha> there's also another oops that I'm not sure who should take a look
  82 <Ursinha> it's OOPS-1565O1873
  83 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1565O1873
  84 <Ursinha> maybe sinzui or gary_poster 
  85 <gary_poster> looking
  86 <sinzui> Ursinha, the file is sane
  87 <sinzui> http://edge.launchpad.net/nunitv2/2.5/2.5.3/+download/NUnit-2.5.3.9345.msi
  88 <MootBot> LINK received:  http://edge.launchpad.net/nunitv2/2.5/2.5.3/+download/NUnit-2.5.3.9345.msi
  89 <Ursinha> sinzui, I saw that
  90 <gary_poster> Ursinha, I'll throw it to sinzui unless he throws it back.  we generally haven't worked on the gog stuff to my knowledge
  91 <gary_poster> gpg
  92 <Ursinha> I just didn't get why that happened
  93 <Ursinha> allenap, your comment is interesting, would that be too hard to do?
  94 <allenap> Ursinha: No, not too hard. Abel has made the statement logging code for webapps work well in a longer-running script context.
  95 <sinzui> gary_poster, I accepted the bug, but the issue still looks to be an issue between slave and master
  96 <Ursinha> allenap, cool! could you please check with deryck what does he thing about that idea? :)
  97 <allenap> Ursinha: Okay.
  98 <Ursinha> allenap, thanks
  99 <Ursinha> sinzui, what should we do about that?
 100 <sinzui> I do not know
 101 <sinzui> I saw this happen in the UI two weeks ago
 102 <sinzui> The CoC oopsed for 10 minutes then worked when the key was really visible
 103 <Ursinha> hm
 104 <Ursinha> sinzui, should I open a bug so that can be investigated properly?
 105 <gary_poster> sinzui: re bug that you accepted: would you bring it up with me out of meeting then?  I'm happy to help, or take it from you if appropriate, but my understanding is that the answer is to use the context manager I mentioned before, and that you chould just use it to always connect to the master for this purpose.  I May be off base, so I'd like to coordinate with you 
 106 .
 107 <sinzui> We have a fix for the page to step out of that condition, but I do not know how to make the master pass the data appear in a timely fashion
 108 <gary_poster> sinzui: do you mean you need info on the context manager I'm mentioning.  can find
 109 <Ursinha> gary_poster, sinzui, I'll file a bug for that, is that ok for you guys?
 110 <sinzui> These is a bug
 111 <gary_poster> Yeah thought so
 112 <sinzui> bug 562486
 113 <ubottu> Launchpad bug 562486 in launchpad-registry "accessing pending_gpg_keys using the api fails to some users" [High,Triaged] https://launchpad.net/bugs/562486
 114 <Ursinha> sinzui, okay, thanks
 115 <Ursinha> [action] gary_poster and sinzui to discuss about the issues between master and slave that could be causing oopses
 116 <MootBot> ACTION received:  gary_poster and sinzui to discuss about the issues between master and slave that could be causing oopses
 117 <Ursinha> [action] allenap to follow up with deryck about bug 230106
 118 <MootBot> ACTION received:  allenap to follow up with deryck about bug 230106
 119 <ubottu> Launchpad bug 230106 in malone "emails interface oops reports need better error messages" [Low,Triaged] https://launchpad.net/bugs/230106
 120 <Ursinha> I'm moving on 
 121 <Ursinha> we have three critical bugs, the two foundations ones are fix committed, one soyuz is in progress
 122 <Ursinha> jelmer is working on the soyuz one
 123 <Ursinha> I guess bug 514267 and bug 530002 are actually Fix Released, right gary_poster?
 124 <ubottu> Launchpad bug 514267 in launchpad-foundations "InternalError on clusters under busy load" [Critical,Fix committed] https://launchpad.net/bugs/514267
 125 <ubottu> Launchpad bug 530002 in launchpad-foundations "Launchpad edge 'devel' webservice serving 'beta' WADL" [Critical,Fix committed] https://launchpad.net/bugs/530002
 126 <sinzui> Ursinha, The filename oops was caused when a chinese user used the chinese name to access a file that does not exist
 127 <Ursinha> sinzui, oh boy
 128 <Ursinha> chinese name
 129 <sinzui> I think there was a transliteration problem for the user.
 130 <gary_poster> yes, and...
 131 <Ursinha> hehe
 132 <Ursinha> sinzui, you mean he typed the url by hand?
 133 <gary_poster> yes
 134 <sinzui> I think so
 135 <gary_poster> both are now marked as resolved
 136 <Ursinha> thanks gary_poster 
 137 <Ursinha> sinzui, well, so I guess there's almost nothing we could do about that..
 138 <Ursinha> in the failing scripts land, a fix for retry-depwait has landed production-devel, so the only script constantly failing has been fixed, it seems
 139 <Ursinha> thanks everyone
 140 <Ursinha> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
 141 <MootBot> New Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)
 142 <Ursinha> hi Chex 
 143 * Ursinha hands Chex the mic
 144 <Ursinha> or the keyboard :)
 145 * Ursinha pokes Chex in every possible channel
 146 <Chex> sorry
 147 <bigjools> oouch!
 148 <Chex> LP report for this week is short:
 149 <Ursinha> SNAFU? :P
 150 <Chex> - LP incidents of note:
 151 <Chex>         : 11-Apr: swap alerts. restarted some LPNet servers. Bug: 531071
 152 <Chex>         : 15-Apr: Librarian1 rapidly gobbling memory again, restarted to free up.  Bug 556245.
 153 <ubottu> Launchpad bug 531071 in launchpad-foundations "lpnet app servers are leaking memory again" [High,Triaged] https://launchpad.net/bugs/531071
 154 <ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/556245)
 155 <rockstar> Looks like Launchpad is timing out for ubottu as well as myself.
 156 <gary_poster> that's actually a different one
 157 <gary_poster> you mean...
 158 <Chex> these are existing issues, but I just bring them up as they occurred in the last week. any questions or comments?
 159 <gary_poster> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/556245
 160 <ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/556245)
 161 <gary_poster> oh
 162 <gary_poster> duh
 163 * gary_poster grins shamefacedly
 164 <Ursinha> rockstar, that's a private bug, maybe that's why?
 165 <gary_poster> yeah prob
 166 <rockstar> Ursinha, ah, maybe.
 167 <rockstar> I've been getting lots of timeouts on Launchpad recently...
 168 <Ursinha> rockstar, any specific pages?
 169 <rockstar> Ursinha, mostly bugs stuff.
 170 <Ursinha> well, lp is working pretty fine for me these days, I'm actually happy with that :)
 171 <Ursinha> rockstar, could you let me know when that happens, please?
 172 <Ursinha> and about Chex bugs, anyone has something to add?
 173 <bigjools> lp is sloooow right now
 174 <rockstar> bigjools, thank you for confirming what I just said.
 175 <bigjools> it just took about 13 seconds to load a simple page
 176 <Ursinha> ouch
 177 <Ursinha> Chex, do you know if is there anything going on right now that could be causing that?
 178 <Ursinha> bigjools, do you have a link so I can test it here, please?
 179 <bigjools> https://edge.launchpad.net/~launchpad/+archive/ppa/+build/1693771
 180 <Chex> Ursinha: I just quickly looked at the lpnet servers, and db-master, I dont see anything striking at the moment
 181 <Ursinha> bigjools, yes, that's really slow
 182 <Ursinha> I'll check the timeouts more closely
 183 <Ursinha> thanks bigjools, rockstar and Chex 
 184 <Chex> Ursinha: welcome
 185 <Ursinha> gary_poster, about that bug Chex mentioned, is there anything that would be done soon?
 186 <Ursinha> bug 556245 
 187 <ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/556245)
 188 <Ursinha> I guess not
 189 <Ursinha> I'll move on to finish this meeting today :)
 190 <gary_poster> Ursinha: I'm going to ask stub to investigate it.  That's all I can say for now.
 191 <Ursinha> gary_poster, that's fine, thanks
 192 <Ursinha> [action] gary_poster to ask stub to investigate bug 556245
 193 <MootBot> ACTION received:  gary_poster to ask stub to investigate bug 556245
 194 <ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/556245)
 195 <ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/556245)
 196 <Ursinha> [TOPIC] * DBA report (stub)
 197 <MootBot> New Topic:  * DBA report (stub)
 198 <Ursinha> [action] Ursinha to send an email to stub asking for the dba report
 199 <MootBot> ACTION received:  Ursinha to send an email to stub asking for the dba report
 200 <Ursinha> [TOPIC] * QA stats of the week
 201 <MootBot> New Topic:  * QA stats of the week
 202 <Ursinha> and here are the untriaged bugs of this week
 203 <Ursinha> launchpadlib: 13
 204 <Ursinha> lp-dev-utils: 9
 205 <Ursinha> launchpad-buildd: 9
 206 <Ursinha> launchpad-foundations: 6
 207 <Ursinha> launchpad-dev-moin-theme: 5
 208 <Ursinha> lp-qa-tools: 3
 209 <Ursinha> lpbuildbot: 2
 210 <Ursinha> trac-launchpad-migrator: 2
 211 <Ursinha> launchpad-loggerhead: 2
 212 <Ursinha> launchpad-news-wordpress-theme: 2
 213 <Ursinha> launchpad-help-moin-theme: 2
 214 <Ursinha> launchpad-cscvs: 2
 215 <Ursinha> launchpad-registry: 1
 216 <Ursinha> launchpad: 1
 217 <Ursinha> launchpad-documentation: 1
 218 <Ursinha> tickcount: 1
 219 <Ursinha> malone: 1
 220 <Ursinha> so, it seems soyuz managed to triage of their bugs :)
 221 <bigjools> \o/ \o/ \o/
 222 * rockstar gives Soyuz a cookie
 223 <Ursinha> bigjools, congratulations :)
 224 <gary_poster> :-)
 225 <bigjools> did the last 30 this morning :)
 226 <rockstar> You have to split between yourselves though.  I only had one cookie.
 227 <Ursinha> haha
 228 <Ursinha> :))
 229 <Ursinha> I'll talk to people about the other ones
 230 <Ursinha> [TOPIC] * Proposed items
 231 <MootBot> New Topic:  * Proposed items
 232 <Ursinha> anyone have anything else to add?
 233 <rockstar> I have one thing to say, which I'll express in a haiku (ala barry)
 234 <Ursinha> go ahead 
 235 <rockstar> Code team sprinting / this meeting is at 4 am / I am not waking up
 236 <Ursinha> lol
 237 <rockstar> That's for next week.
 238 <Ursinha> thanks for letting me know
 239 * Ursinha gives rockstar a cookie for his creativity
 240 <Ursinha> so I guess that's all for this week
 241 <Ursinha> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
 242 <Ursinha> #endmeeting