1 [13:01] <Ursinha> #startmeeting
2 [13:01] <Ursinha> Happy new year!
3 [13:01] <Ursinha> Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues.
4 [13:01] <MootBot> Meeting started at 10:01. The chair is Ursinha.
5 [13:01] <MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
6 [13:01] <noodles775> me
7 [13:01] <Ursinha> [TOPIC] Roll Call
8 [13:01] <Ursinha> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us!
9 [13:01] <MootBot> New Topic: Roll Call
10 [13:01] <allenap> me
11 [13:01] <sinzui> me
12 [13:01] <Ursinha> hehe :)
13 [13:01] <Ursinha> me
14 [13:02] <Ursinha> stub isn't here, that's kinda fine
15 [13:02] <Ursinha> rockstar, hi
16 [13:02] <rockstar> ni!
17 [13:02] <Ursinha> :)
18 [13:02] <salgado> me
19 [13:02] <Ursinha> Chex, hello
20 [13:02] <jtv> I'm keeping one eye on the meeting, but not really here :P
21 [13:03] <Ursinha> jtv, thanks :)
22 [13:03] <Ursinha> let
23 [13:03] <Ursinha> let's go then
24 [13:03] <Ursinha> [TOPIC] Agenda
25 [13:03] <Ursinha> * Actions from last meeting
26 [13:03] <Ursinha> * Oops report & Critical Bugs & Broken scripts
27 [13:03] <Ursinha> * Operations report (mthaddon/Chex/spm/mbarnett)
28 [13:03] <Ursinha> * DBA report (stub)
29 [13:03] <Ursinha> * Proposed items
30 [13:03] <MootBot> New Topic: Agenda
31 [13:03] <Ursinha> [TOPIC] * Actions from last meeting
32 [13:03] <Ursinha> * Chex to file a bug about the multiple git import failures on the importd
33 [13:03] <Ursinha> * Ursinha to try to come up with an older working version of oops-tools (non oops-db version)
34 [13:03] <Ursinha> * Done, it's running on devpad and generating the partial summaries since before holidays. Used to generate the daily summaries as well when the db version fails for any reason.
35 [13:03] <Ursinha> * Ursinha to send one email to stub about the db report
36 [13:03] <Ursinha> * Sent. stub sent the report.
37 [13:03] <MootBot> New Topic: * Actions from last meeting
38 [13:03] <ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=db
39 [13:04] <Chex> here.. sorry
40 [13:04] <Ursinha> Chex, hey :)
41 [13:04] <Ursinha> Chex, so, did you file that bug?
42 [13:04] <Chex> Ursinha: yes I did, let me get the bug number for you
43 [13:05] <Ursinha> thanks Chex
44 [13:05] <Ursinha> I'll keep the ball rolling while Chex finds the bug number :)
45 [13:05] <Ursinha> [TOPIC] * Oops report & Critical Bugs & Broken scripts
46 [13:06] <MootBot> New Topic: * Oops report & Critical Bugs & Broken scripts
47 [13:06] <Ursinha> the failing scripts are being handled already, right jtv? :)
48 [13:06] <jtv> Ursinha: yes, I got distracted for a while by something about the database, but back on it now
49 [13:06] <Ursinha> jtv, haha thanks :)
50 [13:06] <Ursinha> and, we had a bunch of DisconnectionErrors on 4th, on staging, and are having them now on lpnet and edge
51 [13:06] <Ursinha> I've filed a bug for them, bug 504291, jtv kindly did his analysis
52 [13:06] <Ursinha> salgado, could you take this to foundations, please?
53 [13:06] <ubottu> Launchpad bug 504291 in launchpad-foundations "DisconnectionErrors (already disconnected) happening again" [Undecided,New] https://launchpad.net/bugs/504291
54 [13:07] <salgado> again!?
55 [13:07] <salgado> yeah, I'll have a look at it
56 [13:07] <Ursinha> salgado, unfortunately :(
57 [13:07] <Ursinha> another issue that needs to be investigated by foundations is why is staging not being updated
58 [13:07] <Ursinha> mthaddon filed a bug with the error
59 [13:08] <Ursinha> bug 504300
60 [13:08] <salgado> Ursinha, do you have the number?
61 [13:08] <ubottu> Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300
62 [13:08] <Ursinha> salgado, sorry, was searching the tab
63 [13:08] <Chex> Ursinha: I filed a bug, and was immediately redirected to this as a duplicate bug: bug #486076.
64 [13:08] <ubottu> Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076
65 [13:08] <Ursinha> right Chex, thanks
66 [13:09] <Ursinha> I'll take a look
67 [13:09] <Chex> Ursinha: which, when I look at it, I am not sure directly addresses our issue, but its a start, I think.
68 [13:09] <Ursinha> [action] salgado to take bugs 504300 and 504291 to foundations discussion
69 [13:09] <MootBot> ACTION received: salgado to take bugs 504300 and 504291 to foundations discussion
70 [13:09] <ubottu> Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300
71 [13:09] <ubottu> Launchpad bug 504291 in launchpad-foundations "DisconnectionErrors (already disconnected) happening again" [Undecided,New] https://launchpad.net/bugs/504291
72 [13:10] <Ursinha> salgado, is that ok for you?
73 [13:10] <salgado> yep
74 [13:10] <Ursinha> right, thanks
75 [13:11] <Ursinha> Chex, right, do you think it's not a duplicate? maybe a code guy could enlighten us
76 [13:11] <Ursinha> rockstar, ^
77 [13:11] <Chex> Ursinha: I was under the impression that if the code tree was very large and big, it overwhelmed the import process, with memory usage.. that doesn't seem like a memory leak to me, but I could be mistaken about that.
78 [13:12] <-- andrea-bs
79 [13:12] <Ursinha> rockstar doesn't love us anymore
80 [13:13] <rockstar> Chex, we can probably sleuth whether or not it's a duplicate. I'd suggest filing the bug.
81 [13:13] <Chex> Ursinha: here is the orginal bug I filed: Bug #497945 : Large git imports into bzr tree uses lots of memory
82 [13:13] <ubottu> Launchpad bug 497945 in bzr "Large git imports into bzr tree uses lots of memory (dup-of: 486076)" [Undecided,New] https://launchpad.net/bugs/497945
83 [13:13] <ubottu> Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076
84 [13:13] <Ursinha> rockstar, ^
85 [13:14] <Chex> Ursinha: I can discuss this offline with you and/or rockstar later, not to hold up the meeting
86 [13:14] * rockstar agrees
87 [13:14] <Ursinha> Chex, thanks :)
88 [13:14] <Ursinha> rockstar, thanks :)
89 [13:14] <Ursinha> moving on then
90 [13:14] <rockstar> These kinds of bugs are harder to track down, but we know they exist.
91 [13:15] <Ursinha> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
92 [13:15] <MootBot> New Topic: * Operations report (mthaddon/Chex/spm/mbarnett)
93 [13:15] <Chex> here is our report for this week, folks:
94 [13:15] <Chex> - LP edge updates broke today (CSS broken) because make build was relying
95 [13:15] <Chex> on python2.4, salgado is helping to fix this issue now.
96 [13:15] <Chex> - LP staging updates are broken at the moment, Bug 504300 has been filed
97 [13:15] <Chex> addressing this issue.
98 [13:15] <ubottu> Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300
99 [13:16] <Chex> - LP seeing intermittent 500s on shipit servers, we are
100 [13:16] <Chex> investigating that more to see exactly what is causing those.
101 [13:16] <Chex> - LP incidents of note:
102 [13:16] <Chex> ; LP Cherry-picks: revno 8809 rolled out to lpnet 30-Dec,
103 [13:16] <Chex> revno 8810 rollout to lpnet 07-Jan;
104 [13:16] <Chex> ; Crowberry upgraded from 8Gb to 32GB RAM 23-Dec
105 [13:16] <Chex> and thats our report for this week, any questions/comments?
106 [13:16] <Ursinha> not here
107 [13:16] <Ursinha> anyone?
108 [13:16] <Ursinha> [action] Chex to discuss with rockstar about bug 497945
109 [13:16] <MootBot> ACTION received: Chex to discuss with rockstar about bug 497945
110 [13:16] <ubottu> Launchpad bug 497945 in bzr "Large git imports into bzr tree uses lots of memory (dup-of: 486076)" [Undecided,New] https://launchpad.net/bugs/497945
111 [13:17] <ubottu> Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076
112 [13:17] <Ursinha> all right
113 [13:17] <Ursinha> thanks Chex!
114 [13:17] <Ursinha> [TOPIC] * DBA report (stub)
115 [13:17] <Ursinha> sent by email to the list
116 [13:17] <MootBot> New Topic: * DBA report (stub)
117 [13:17] <Ursinha> [TOPIC] * Proposed items
118 [13:17] <MootBot> New Topic: * Proposed items
119 [13:17] <Ursinha> anyone?
120 [13:17] <Chex> Ursinha: your welcome, thanks..
121 [13:17] <Ursinha> no items proposed in the meeting agenda page
122 [13:18] <Ursinha> 5
123 [13:18] <Ursinha> 4
124 [13:18] <Ursinha> 3
125 [13:18] <Ursinha> 2
126 [13:18] <Ursinha> 1
127 [13:18] <Ursinha> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
128 [13:18] <Ursinha> #endmeeting
129 [13:18] <MootBot> Meeting finished at 10:18.