1 <matsubara> #startmeeting
2 <MootBot> Meeting started at 10:00. The chair is matsubara.
3 <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
4 <matsubara> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues.
5 <matsubara> [TOPIC] Roll Call
6 <MootBot> New Topic: Roll Call
7 <matsubara> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us!
8 <matsubara> Chex, gary_poster, rockstar, Ursinha, bigjools, danilos, sinzui1, allenap: hi
9 <sinzui1> me
10 <Ursinha> me
11 * sinzui1 is now known as sinzui
12 <danilos> me
13 <Chex> moi
14 <danilos> matsubara, hi :)
15 <bigjools> me, if it's faster than last week :)
16 <gary_poster> matsubara: on IS call as I mentioned.
17 <matsubara> gary_poster, all right.
18 <gary_poster> matsubara: added notes to the two bugs that were my action items
19 <matsubara> thanks gary_poster
20 <matsubara> so, let's move on, rockstar and allenap can join in along the way
21 <matsubara> [TOPIC] Agenda
22 <MootBot> New Topic: Agenda
23 <matsubara> * Actions from last meeting
24 <matsubara> * Oops report & Critical Bugs & Broken scripts
25 <matsubara> * Operations report (mthaddon/Chex/spm/mbarnett)
26 <matsubara> * DBA report (stub)
27 <matsubara> * Proposed items
28 <matsubara> [TOPIC] * Actions from last meeting
29 <MootBot> New Topic: * Actions from last meeting
30 <matsubara> * Ursinha to talk to mwhudson about bug 513412
31 <matsubara> * rockstar to investigate OOPS-1480CMP1 and fill in details on bug 517126
32 <matsubara> * gary_poster to check status on work for DisconnectionErrors (bug 450593) and assign bug 403618 to someone in foundations
33 <matsubara> * Ursinha to set up StrategyTeamTestPlan wiki for jml,mrevell
34 <ubottu> Bug 513412 on http://launchpad.net/bugs/513412 is private
35 <ubottu> Launchpad bug 517126 in launchpad-code "BzrCheckError raised creating a merge proposal" [Undecided,New] https://launchpad.net/bugs/517126
36 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1480CMP1
37 <ubottu> Launchpad bug 450593 in launchpad-foundations "Still seeing non-informational DisconnectionErrors on login servers" [High,Triaged] https://launchpad.net/bugs/450593
38 <ubottu> Launchpad bug 403618 in launchpad-foundations "Launchpad should return a 404 instead of ForbiddenAttribute OOPS" [High,Triaged] https://launchpad.net/bugs/403618
39 <Ursinha> matsubara: I've set up the testplan for Strategy team
40 <matsubara> cool.
41 <Ursinha> matsubara: and I can't access the bug to see if I commented on it
42 <Ursinha> :/
43 <matsubara> and gary_poster commented on both bugs
44 <Ursinha> matsubara: I'll check
45 <matsubara> Ursinha, it's private for me too
46 <matsubara> Ursinha, ask one of the losas to subscribe the lp team to it
47 * mrjazzcat (~mrjazzcat@c-24-9-91-150.hsd1.co.comcast.net) has joined #launchpad-meeting
48 <Ursinha> matsubara: I'll do that
49 <matsubara> [action] * rockstar to investigate OOPS-1480CMP1 and fill in details on bug 517126
50 <ubottu> Launchpad bug 517126 in launchpad-code "BzrCheckError raised creating a merge proposal" [Undecided,New] https://launchpad.net/bugs/517126
51 <MootBot> ACTION received: * rockstar to investigate OOPS-1480CMP1 and fill in details on bug 517126
52 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1480CMP1
53 <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1480CMP1
54 <mrjazzcat> matsubara: whew, made it
55 <matsubara> hi mrjazzcat, welcome
56 <Ursinha> :)
57 <matsubara> Ursinha, shall I re-add the action item for the one you need to talk to mwhudson about?
58 <Ursinha> matsubara: please
59 <matsubara> [action] * Ursinha to talk to mwhudson about bug 513412
60 <MootBot> ACTION received: * Ursinha to talk to mwhudson about bug 513412
61 <ubottu> Bug 513412 on http://launchpad.net/bugs/513412 is private
62 <ubottu> Bug 513412 on http://launchpad.net/bugs/513412 is private
63 <Ursinha> matsubara: as soon as I confirm I'll leave a comment there
64 <matsubara> ok, thanks
65 <matsubara> [TOPIC] * Oops report & Critical Bugs & Broken scripts
66 <MootBot> New Topic: * Oops report & Critical Bugs & Broken scripts
67 <Ursinha> okay
68 <Ursinha> so I'd like to bring up a timeout weirdness I saw, I've filed bug 523904 for that
69 <ubottu> Launchpad bug 523904 in malone "Distribution:+bugs timing out" [Undecided,New] https://launchpad.net/bugs/523904
70 <Ursinha> allenap: this bug is about a bunch of timeouts in Distro:+bugs
71 <Ursinha> allenap: but they apparently happened in a short period of time, and I couldn't find anything in the LP incident log
72 <Ursinha> allenap: could you take a look, please?
73 <matsubara> can you reproduce that bug Ursinha ?
74 <Ursinha> matsubara: not consistently
75 <matsubara> [action] allenap to investigate bug 523904
76 <ubottu> Launchpad bug 523904 in malone "Distribution:+bugs timing out" [Undecided,New] https://launchpad.net/bugs/523904
77 <MootBot> ACTION received: allenap to investigate bug 523904
78 <Ursinha> matsubara: but it times out from time to time and given the number of timeouts it would be great to investigate what happened
79 <sinzui> We are seeing a frequent timeouts on edge /ubuntu/lucid/+packaging and +needs-packaging. I reported a bug and will try to land a fix today
80 <Ursinha> thanks sinzui
81 <matsubara> any other oopses Ursinha ?
82 <Ursinha> the other bugs with most oopses are fix committed
83 <Ursinha> so, no
84 <matsubara> we have 4 critical bugs, all fix committed
85 <matsubara> yay!
86 <Ursinha> :)
87 <Ursinha> there are two scripts I've seen failing a couple of times lately
88 <Ursinha> productreleasefinder and allocate-revision-karma
89 <Ursinha> sinzui: do you know if prf one is legit?
90 <Ursinha> the failures, that is
91 <Ursinha> the other one I guess rockstar would be the best person to ask
92 <Ursinha> after that, I'm done
93 <sinzui> Ursinha: I am waiting to see if it fails again. It should be running right now
94 <Ursinha> sinzui: right
95 <Ursinha> matsubara: [action] Ursinha to ask rockstar about allocate-revision-karma failures
96 <matsubara> [action] Ursinha to ask rockstar about allocate-revision-karma failures
97 <MootBot> ACTION received: Ursinha to ask rockstar about allocate-revision-karma failures
98 <matsubara> ok, let's move on
99 <matsubara> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
100 <MootBot> New Topic: * Operations report (mthaddon/Chex/spm/mbarnett)
101 <Chex> hello everyone
102 <Chex> quick report this week:
103 <Chex> - DB Replication Lag/Slave-master issue: We are still running on the fake lag script pointing all traffic at
104 <sinzui> did edge just fall off the earth?
105 <Chex> the main production db. Has there been any new progress with this issue?
106 <Chex> - LP incidents of note:
107 <Chex> ; Codebrowse restarts, we are getting better debugging info with mwhudson's help.
108 <Chex> anyone have any questions or comments?
109 <Ursinha> Chex: I have one request: could you subscribe lp team to bug 513412, please?
110 <ubottu> Bug 513412 on http://launchpad.net/bugs/513412 is private
111 <Chex> Ursinha: yes sure
112 <Ursinha> sinzui: what do you mean?
113 <Ursinha> Chex: thanks!
114 <sinzui> edge is slow, and users on #launchpad are reporting lots of timeouts. The I got a server is down from edge
115 <Chex> Ursinha: done
116 <Ursinha> Chex: great, thanks a lot
117 <Ursinha> sinzui: hm, it's working quite well here... anyone else noticing that?
118 <sinzui> I wonder if we have a bad server is rotation. I can have great loads and Please try again pages
119 <Ursinha> Chex: do you know if there's an incident going on?
120 <Chex> Ursinha: I am not seeing anything specific in our Lp monitoring, checking some more
121 <matsubara> ok, I guess we can move on and see if the timeouts are worrisome outside the meeting
122 <matsubara> thanks Chex, Ursinha and sinzui
123 <matsubara> [TOPIC] * DBA report (stub)
124 <MootBot> New Topic: * DBA report (stub)
125 <matsubara> I'll ask stub to email the report
126 <matsubara> [TOPIC] * Proposed items
127 <MootBot> New Topic: * Proposed items
128 <matsubara> no new proposed items
129 <matsubara> anything else before I close?
130 <matsubara> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
131 <matsubara> #endmeeting
132 <MootBot> Meeting finished at 10:26.