1 <matsubara> moo
   2 <sinzui> cluck
   3 <gary_poster> hee-haw
   4 <danilos> meow
   5 <Ursinha> hello
   6 <Ursinha> #startmeeting
   7 <Ursinha> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. 
   8 <MootBot> Meeting started at 11:01. The chair is Ursinha.
   9 <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
  10 <Ursinha> [TOPIC] Roll Call
  11 <Ursinha> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! 
  12 <MootBot> New Topic:  Roll Call
  13 <Ursinha> meeee
  14 <sinzui> me
  15 <gary_poster> smee
  16 <matsubara> me
  17 <Ursinha> rockstar, Chex, hello :)
  18 <Ursinha> I see no soyuz people :(
  19 * adeuring (~abel@93.217.153.185) has joined #launchpad-meeting
  20 <Ursinha> I know we'll have no bugs people here, so ok
  21 <Ursinha> well adeuring is here, but not sure if it's for our meeting :)
  22 <Ursinha> I'll move on and expect people to show up
  23 <Ursinha> [TOPIC] Agenda
  24 <Ursinha>  * Actions from last meeting
  25 <Ursinha>  * Oops report & Critical Bugs & Broken scripts
  26 <Ursinha>  * Operations report (mthaddon/Chex/spm/mbarnett)
  27 <Ursinha>  * DBA report (stub)
  28 <Ursinha>  * QA stats of the week
  29 <Ursinha>  * Proposed items
  30 <MootBot> New Topic:  Agenda
  31 <adeuring> Ursinha: yes, that's why I'm here, but I have another (mumble) meeting right now.
  32 <Ursinha> [TOPIC] * Actions from last meeting
  33 <Ursinha>  * DONE: matsubara to chase someone from Bugs about https://bugs.edge.launchpad.net/malone/+bug/583385
  34 <MootBot> New Topic:  * Actions from last meeting
  35 * salgado is now known as salgado-lunch
  36 <Ursinha> cool, all done
  37 <Ursinha> adeuring, thanks :)
  38 <ubot5> Error: Could not parse data returned by Launchpad: The read operation timed out (https://launchpad.net/bugs/583385)
  39 <Ursinha> [TOPIC] * Oops report & Critical Bugs & Broken scripts
  40 <MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
  41 <Ursinha> main offenders are bug 504124 (spam processing bug?) and bug 580181
  42 <ubot5> Launchpad bug 504124 in Launchpad Foundations "process-mail.py "need more than 1 value to unpack" (affected: 2, heat: 22)" [High,Triaged] https://launchpad.net/bugs/504124
  43 <ubot5> Launchpad bug 580181 in Soyuz "DistributionSourcePackageRelease page still oopsing with NotOneError/LocationError (affected: 1, heat: 6)" [High,Triaged] https://launchpad.net/bugs/580181
  44 <Ursinha> I see the latter should be worked by noodles as soon as he got rid of more urgent problems
  45 * rockstar is here...  had computer problems
  46 * Ursinha hugs rockstar 
  47 <Chex> Ursinha: All us LOSAs are unavailable this week as we are sprinting in Montreal.
  48 <Ursinha> Chex, oh, that's bad, LOSAs' report is so useful
  49 * Ursinha thinks
  50 <sinzui> the process-mail.py bug is a spam attack. Do we want to suppress oopses when the header is malformed?
  51 <Ursinha> this meeting is crippled :/
  52 <Ursinha> sinzui, good question
  53 <Ursinha> gary_poster, we're having annoying high oops counts of 504124, maybe we should do something about it now the counting seems to be raising?
  54 <Ursinha> gary_poster, even what sinzui suggested
  55 <matsubara> sinzui, I think we should not log an oops in that case as you suggested in the bug report
  56 <sinzui> Ursinha, gary_poster. The problem is not the app, but the data so we should gracefully recover I think
  57 <Ursinha> that's my opinion as well
  58 <matsubara> with a try:except when the code is processing the To: header
  59 <gary_poster> Ursinha: yeah, I had that one open already.  If it is really bothering us then let's do it.  Agree with sinzui's approach
  60 <Ursinha> cool
  61 <Ursinha> gary_poster, should I update the bug report with that? 
  62 <sinzui> well, that is now a trivial fix I think
  63 <gary_poster> Ursinha: I'll handle it, thank you :-)
  64 <Ursinha> thanks gary_poster :)
  65 <Ursinha> nice
  66 <Ursinha> thanks all :)
  67 <Ursinha> I'd need a soyuz people here now
  68 <Ursinha> I'll go with the code one then
  69 <Ursinha> rockstar, so, not sure if code team knows about it: OOPS-1607EA658 
  70 <ubot5> https://lp-oops.canonical.com/oops.py/?oopsid=1607EA658
  71 <sinzui> I think the soyuz bug has a deeper root cause...
  72 <sinzui> I looked at it last month and could not see what was happening. I think it goes back to changes made in Jan/Feb
  73 <Ursinha> sinzui, might have, this was "fixed" previously and now shows two different oopses
  74 <rockstar> Ursinha, so, I didn't know about it, but this area still hasn't been tested much...  :/
  75 <rockstar> Ursinha, it won't be released.
  76 <Ursinha> rockstar, oh, right
  77 <Ursinha> rockstar, was that QAed on edge? I'm pointing the oops as it might be a consequence of QAing the feature
  78 <rockstar> Ursinha, well, so no, we can't QA on edge much because we're blocked on two RTs...
  79 <Ursinha> rockstar, I see :/ do you know if the blockage is something that might last for long?
  80 <rockstar> Ursinha, well, the RT was filed as high priority more than a week ago, so your guess is as good as mine.
  81 <Ursinha> rockstar, maybe we should poke some people? :)
  82 * Ursinha cries for a losa
  83 <gary_poster> rockstar, Ursinha, is this ...
  84 <rockstar> gary_poster!
  85 <gary_poster> 39402?
  86 <rockstar> Ursinha, gary_poster is the one who's been chasing for us.
  87 <gary_poster> RT 39402?
  88 <Ursinha> ah, I see
  89 <rockstar> gary_poster, yes, that's it.
  90 <gary_poster> Ursinha, rockstar, mthaddon has said that he will look into it today
  91 <rockstar> gary_poster, ta
  92 <gary_poster> or to not misrepresent: "member:gary: I'll try and take a look at the latter one (the first one is marked resolved) sometime today" :-)
  93 <Ursinha> cool!
  94 <Ursinha> thanks gary_poster and rockstar :)
  95 <gary_poster> np
  96 <Ursinha> rockstar, I'll keep my eyes on that oopses and let you know if something shows up
  97 <Ursinha> thanks
  98 <danilos> rockstar, my suggestion is to also talk to bigjools and lamont to make sure your request is similar to what is being done for "unsupported" packages already
  99 <danilos> rockstar, about that RT at least
 100 <Ursinha> [action] talk to bigjools about the 300 timeouts in the DistroSeries:+index pages everyday, is that expected/known? e.g.: OOPS-1607C1282
 101 <MootBot> ACTION received:  talk to bigjools about the 300 timeouts in the DistroSeries:+index pages everyday, is that expected/known? e.g.: OOPS-1607C1282
 102 <ubot5> https://lp-oops.canonical.com/oops.py/?oopsid=1607C1282
 103 <ubot5> https://lp-oops.canonical.com/oops.py/?oopsid=1607C1282
 104 <matsubara> Ursinha, isn't that the one bigjools replied to recently?
 105 <matsubara> the one that is making an xmlrpc request to another server but is being blocked by a firewall
 106 <Ursinha> matsubara, not sure if that's the same, I guess that one was about builders
 107 <sinzui> Ursinha, That is my oops
 108 <matsubara> ok, just wondering
 109 <Ursinha> matsubara, thanks for that :)
 110 <Ursinha> sinzui, the DistroSeries:+index?
 111 <danilos> Ursinha, yeah, that sounds like registry
 112 <Ursinha> if so, even better, you're here :)
 113 <sinzui> the distroseries-portlet-packaging  is the root cause. I have a branch that add3 3 hours caching to it
 114 <danilos> and fwiw, storm taking too long to serialize stuff
 115 <Ursinha> [action] forget last action
 116 <MootBot> ACTION received:  forget last action
 117 <sinzui> Edwin's definitve SQL fix will not land until next release
 118 <gary_poster> Ursinha, so you know, thumper alerted me last night that this soft timeout is causing timeouts (hard, I assume) on the branch merge proposal page: https://lp-oops.canonical.com/oops.py/?oopsid=1607EA56 .  Therefore I need to make a bug and look into it.
 119 <gary_poster> (Which sucks because pretty much none of know librarian code, but oh well :-) )
 120 <Ursinha> gary_poster, thanks for that
 121 <Ursinha> sinzui, do you have the bug number?
 122 <sinzui> My oops is caused by a monster query. bug 535430 is the root cause
 123 <ubot5> Launchpad bug 535430 in Launchpad Registry "+needs-packaging link times out regularly (affected: 1, heat: 6)" [High,In progress] https://launchpad.net/bugs/535430
 124 <Ursinha> sinzui, right
 125 <sinzui> ^ this is really a model problem that manifests itself on three pages
 126 <Ursinha> sinzui, which ones?
 127 <sinzui> distroseries +index, +needs-packaging +packaging-links  (distroseries-portlet-packaging) plus two full pages
 128 <Ursinha> thank you 
 129 * Ursinha takes notes
 130 <Ursinha> thanks everyone
 131 <sinzui> ^ +packaging, not +packaging-links
 132 * Ursinha fixes her notes
 133 <sinzui> https://edge.launchpad.net/ubuntu/maverick shows that the issue centers on the Upstream packaging data
 134 <Ursinha> sinzui, so your fix will "fix" only part of the problem
 135 <sinzui> yes. timeouts will be reduced by 50% to 80%
 136 <sinzui> That is still disappointing
 137 <Ursinha> that's something
 138 <Ursinha> I guess the model problem won't be fixed (if fixed) anytime soon..
 139 <sinzui> The fix will be too complicated for a CP
 140 <Ursinha> sinzui, do you think that will be fixed soon?
 141 <sinzui> It will be fixed in week 1 on staging
 142 <Ursinha> oh, cool
 143 <Ursinha> better than I imagined
 144 <Ursinha> thanks sinzui 
 145 <Ursinha> I might move on, otherwise the meeting will last forever :)
 146 <Ursinha> we have no critical bugs! \o/
 147 <gary_poster> \o/
 148 <Ursinha> rockstar, upgrade_branches is still failing
 149 <Ursinha> gary_poster, :)
 150 <rockstar> Ursinha, yeah, I'll look into it.  I have some other P1 bugs that need to be fixed first.
 151 <Ursinha> rockstar, sure. Thanks for that. Do you know if there are any bugs opened for that? I can do that!
 152 <rockstar> Ursinha, can you please open one?
 153 <Ursinha> rockstar, sure :)
 154 <Ursinha> rockstar, I'll do that and let you know. Thanks
 155 <rockstar> Ursinha, thanks.
 156 <Ursinha> [action] Ursinha to open a bug for the failing upgrade_branches script
 157 <MootBot> ACTION received:  Ursinha to open a bug for the failing upgrade_branches script
 158 <Ursinha> cool, moving on
 159 <Ursinha> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
 160 <MootBot> New Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)
 161 <Ursinha> losas are sprinting, so no donut for us
 162 <Ursinha> :/
 163 <Ursinha> Chex, may I ask you to send any important news to the list, when you have time? please? :)
 164 * Ursinha values that report
 165 <Ursinha> it's more a wish than anything else, as you're sprinting
 166 <Ursinha> :)
 167 <Ursinha> moving on!
 168 <Ursinha> [TOPIC] * DBA report (stub)
 169 <MootBot> New Topic:  * DBA report (stub)
 170 <Ursinha> Email asking for the report sent!
 171 <Ursinha> [TOPIC] * QA stats of the week
 172 <MootBot> New Topic:  * QA stats of the week
 173 <Ursinha> untested stuff goes this way:
 174 <Ursinha> Code: 7
 175 <Ursinha> Foundations: 3
 176 <Ursinha> Registry: 4
 177 <Ursinha> Bugs: 3
 178 <Ursinha> Translations: 7
 179 <Ursinha> Soyuz: 3
 180 <Ursinha> Strategy: 0
 181 <Ursinha> I see we're kind of stable re. the number of untested items every week, which is good :)
 182 <Ursinha> right
 183 <Ursinha> about the untriaged bugs
 184 <Ursinha> I've sent one email to lp list asking your opinions about all projects we should take care of
 185 <Ursinha> I don't think pasting the numbers here will be useful now, I'd like you to reply that thread so we know what to do about those
 186 <Ursinha> pretty please?
 187 <danilos> stats for untested stuff seem incorrect for translations :)
 188 <Ursinha> danilos, it was like that 30 mins ago
 189 <danilos> Ursinha, I doubt it, maybe something like 1h or more ago :)
 190 <gary_poster> heh
 191 <Ursinha> danilos, how come? give me a paper bag, please
 192 * gary_poster is huungry :-D
 193 <Ursinha> ok ok
 194 <Ursinha> danilos, you're right, I see only three now
 195 <danilos> Ursinha, oh, the graph is probably slow to update, don't worry about it
 196 <Ursinha> danilos, thanks :)
 197 <Ursinha> so, please, answer the thread!
 198 <Ursinha> :_)
 199 <Ursinha> [TOPIC] * Proposed items
 200 <Ursinha> None from me.
 201 <MootBot> New Topic:  * Proposed items
 202 <Ursinha> anyone wants to say something?
 203 <Ursinha> okay :)
 204 <Ursinha> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
 205 <Ursinha> #endmeeting
 206 <gary_poster> I'm preparing an email to Ursinha and matsubara about orphan branches