1 <matsubara> #startmeeting
   2 <MootBot> Meeting started at 10:00. The chair is matsubara.
   3 <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
   4 <matsubara> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. 
   5 <matsubara> [TOPIC] Roll Call 
   6 <MootBot> New Topic:  Roll Call
   7 <matsubara> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! 
   8 <Ursinha-afk> me
   9 <gary_poster> me
  10 <intellectronica> me
  11 <barry> me
  12 * Ursinha-afk is now known as Ursinha
  13 <rockstar> ni!
  14 <bigjools> me
  15 <intellectronica> Ursinha: and i thought you're doing your telepathic typing trick again...
  16 <Ursinha> intellectronica, hehe
  17 <matsubara> Chex, hi
  18 <Chex> matsubara: hello
  19 <matsubara> ok, so we're missing stub and henning/danilo
  20 <danilos> me
  21 <matsubara> hi danilos 
  22 <matsubara> :-)
  23 <danilos> matsubara: no you are not :)
  24 <matsubara> so, from now on, you'll be attending the prod meeting, right?
  25 <danilos> matsubara: well, not necessarily, let's see :)
  26 <matsubara> ok, let's move on then
  27 <matsubara> [TOPIC] Agenda 
  28 <danilos> matsubara: but soon I will be replacing henninge as our QA contact
  29 <MootBot> New Topic:  Agenda
  30 <matsubara>  * Actions from last meeting
  31 <matsubara>  * Oops report & Critical Bugs & Broken scripts
  32 <matsubara>  * Operations report (mthaddon/Chex/spm/mbarnett)
  33 <matsubara>  * DBA report (stub)
  34 <matsubara>  * Proposed items
  35 <matsubara> danilos, ok
  36 <matsubara> [TOPIC] * Actions from last meeting
  37 <MootBot> New Topic:  * Actions from last meeting
  38 <matsubara>   * cprov to follow up on bug 413749
  39 <matsubara>   * stub to investigate garbo-hourly failure after spm adjusted script checking to 12h
  40 <matsubara>   * sinzui to poke barry about ExpatError OOPSes (bug 403606)
  41 <matsubara>   * danilos, Ursinha and matsubara to discuss oops summaries split per team
  42 <ubottu> Bug 413749 on http://launchpad.net/bugs/413749 is private
  43 <ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
  44 <matsubara> ok, I know that stub replied to the garbo-hourly failure email, so I think that's done
  45 <danilos> matsubara: we kind of did that, flacoste is involved as well, and it's all on track I'd say :)
  46 <matsubara> danilos, right. if you have more suggestions let me know
  47 <matsubara> barry, hi!
  48 <danilos> sinzui is on unavailable I believe
  49 * barry has some thoughts about the expat errors
  50 <barry> matsubara: hi!
  51 <matsubara> barry, what's up with those expat errors?
  52 <cprov> matsubara: I did reply on 413749 as well, needs discussion, though.
  53 <matsubara> thanks cprov 
  54 * noodles775 has quit (Read error: 110 (Connection timed out))
  55 <Ursinha> matsubara, the one with the expaterrors I'd bring in the oops section
  56 * noodles775_ (n=miken@f053074246.adsl.alicedsl.de) has joined #launchpad-meeting
  57 <barry> matsubara: well, some more investigation might be warranted, but they are intermittent and i've always chalked them up to problems with xmlrpc.  i believe they can be safely ignored though because the system will "catch up" the next time mailman talks to xmlrpc server.
  58 <Ursinha> barry, here's the problem, they happen every day
  59 <barry> matsubara: ultimately, we should get rid of the xmlrpc stuff.  it was a solution to a problem we no longer have (namely gpl pollution between mailman and launchpad)
  60 <matsubara> if it's safe to ignore them, can we add some code to catch the exception and not log the oops?
  61 * stub (n=stub@canonical/launchpad/stub) has joined #launchpad-meeting
  62 <matsubara> hi stub 
  63 <Ursinha> you said in the bug report that we have only a handful a week, but I see them everyday, and their numbers are the highest count
  64 <barry> Ursinha: i get many oops reports for mailman that are clean.  am i looking at the wrong reports?
  65 <Ursinha> barry, I think so :)
  66 <Ursinha> look at the lpnet reports
  67 <barry> Ursinha: meaning the daily emails
  68 <barry> hmm.  okay.  i see them only rarely in the mailman daily email.  i will double check next time i see the report, but i think ignoring the error is the right thing to do
  69 <Ursinha> barry, ok. it floods the summary, and that's annoying
  70 <barry> Ursinha: gotcha!  i hadn't realized that.  i'll do something about it then
  71 <gary_poster> When I discussed hiding the problems, given the frequency, Francis suggested that we not hide them but investigate a bit.  Maybe a small timebox would be appropriate?
  72 <Ursinha> thaaaanks muchly barry!
  73 <gary_poster> eh, nm
  74 <gary_poster> :-)
  75 <Ursinha> gary_poster, :)
  76 <barry> gary_poster: yes, we can do that.  it will take losa intervention and some cowboy debugging, but it's probably worth it
  77 <gary_poster> ok cool
  78 <matsubara> [action] barry to continue debug on bug 403606
  79 <MootBot> ACTION received:  barry to continue debug on bug 403606
  80 <ubottu> Launchpad bug 403606 in launchpad-registry "ExpatError errors should be handled to not generate the OOPSes" [High,Triaged] https://launchpad.net/bugs/403606
  81 <matsubara> thanks barry, gary_poster, Ursinha 
  82 <Ursinha> thanks
  83 <matsubara> let's move on
  84 <matsubara> [TOPIC] * Oops report & Critical Bugs & Broken scripts
  85 <MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
  86 <Ursinha> ok
  87 <matsubara> Ursinha, take the stage please
  88 <Ursinha> so the ExpatErrors were already discussed here
  89 <Ursinha> I have only one bug
  90 <Ursinha> intellectronica, bug 408738
  91 <ubottu> Launchpad bug 408738 in malone "OOPS when rendering bug activity" [Undecided,Triaged] https://launchpad.net/bugs/408738
  92 <Ursinha> that's another one that happens pretty often
  93 <intellectronica> really? if so then we should at least recover gracefully from it
  94 <Ursinha> intellectronica, I see those everyday as well
  95 <intellectronica> i mean in addition to figuring out why we have records like those
  96 <Ursinha> I agree
  97 <Ursinha> intellectronica, could you take a look or poke someone to do so, please?
  98 <intellectronica> yes yes
  99 <Ursinha> thanks intellectronica :)
 100 <Ursinha> matsubara, [action] intellectronica to take a look or find someone to take a look on bug 408738
 101 <ubottu> Launchpad bug 408738 in malone "OOPS when rendering bug activity" [Undecided,Triaged] https://launchpad.net/bugs/408738
 102 <matsubara> [action] intellectronica to take a look or find someone to take a look on bug 408738
 103 <MootBot> ACTION received:  intellectronica to take a look or find someone to take a look on bug 408738
 104 <Ursinha> thanks :)
 105 <matsubara> thanks Ursinha and intellectronica 
 106 <matsubara> we have 2 critical bugs
 107 <Ursinha> critical bugs are fix committed, and the process-pending-packagediffs script failure was already explained by cprov
 108 <matsubara> both fix committed
 109 <Ursinha> so, we're done!
 110 <Ursinha> you can move on, matsubara
 111 <Ursinha> thanks guys!
 112 <matsubara> thanks everyone
 113 <matsubara> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
 114 <MootBot> New Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)
 115 <Chex> hello everyone :
 116 <Ursinha> hi Chex :)
 117 <Chex> the report for this week:
 118 <Chex> Lots more cherry picks this week
 119 <Chex> More and more problems with codebrowse (per the Incident Log)
 120 <Chex> New losas getting up to speed (Chex and mbarnett)
 121 <Chex> Testing tarmac for oops-tools project - making progress but not quite there yet
 122 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=tools
 123 <Chex> Will begin process of implementing recommendations from SplitIt Sprint before too long
 124 <Chex> One of our top priority bugs is bug 416990 - others per https://bugs.launchpad.net/~canonical-losas
 125 <Chex> That's it unless there are any questions
 126 <ubottu> Launchpad bug 416990 in bzr "Memory usage of codehosting processes is excessive" [High,Confirmed] https://launchpad.net/bugs/416990
 127 <matsubara> Chex, re: tarmac setup for oops-tools, I'll submit a merge proposal to tarmac today with the remaining fixes needed.
 128 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=tools
 129 <matsubara> Chex, I'll coordinate with mthaddon later on
 130 <mthaddon> cool
 131 <Chex> matsubara: ok, great
 132 <matsubara> rockstar, can you poke people in your team about the codehosting memory issue?
 133 <danilos> matsubara: is it worth fixing ubottu not to choke on oops-tools mention? ;)
 134 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=tools
 135 <Ursinha> lol
 136 <matsubara> I can ask the maintainer
 137 <danilos> matsubara: thanks :)
 138 <matsubara> [action] matsubara to chase ubottu maintainer to fix regexp used to identify oopses
 139 <MootBot> ACTION received:  matsubara to chase ubottu maintainer to fix regexp used to identify oopses
 140 <matsubara> let's move on
 141 <matsubara> thanks Chex
 142 <matsubara> [TOPIC] * DBA report (stub)
 143 <MootBot> New Topic:  * DBA report (stub)
 144 <stub> The long running transaction killer is back set to 3 hours, thanks to the Rosetta team sorting out the language pack export issue quickly.
 145 <stub> oot
 146 <Ursinha> that was fast
 147 <matsubara> hehe
 148 <bigjools> like mrevell, it was short
 149 <matsubara> thanks stub and rosetta people :-)
 150 <matsubara> lol
 151 <Ursinha> lol
 152 <matsubara> [TOPIC] * Proposed items
 153 <MootBot> New Topic:  * Proposed items
 154 <matsubara> we have no proposed items
 155 <mrevell> haha bigjools
 156 <matsubara> anything else before I close?
 157 <matsubara> hi mrevell 
 158 <matsubara> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
 159 <mrevell> matsubara: nothing from me, just responding to bigjools... :)
 160 <matsubara> #endmeeting 
 161 <MootBot> Meeting finished at 10:24.