## Template for LP Production Meeting logs. Just paste your irc client log below and the format IRC line will take care of formatting correctly #format IRC [13:01] #startmeeting [13:01] Happy new year! [13:01] Welcome to this week's Launchpad Production Meeting. For the next 45 minutes or so, we'll be coordinating the resolution of specific Launchpad bugs and issues. [13:01] Meeting started at 10:01. The chair is Ursinha. [13:01] Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE] [13:01] me [13:01] [TOPIC] Roll Call [13:01] Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! [13:01] New Topic: Roll Call [13:01] me [13:01] me [13:01] hehe :) [13:01] me [13:02] stub isn't here, that's kinda fine [13:02] rockstar, hi [13:02] ni! [13:02] :) [13:02] me [13:02] Chex, hello [13:02] I'm keeping one eye on the meeting, but not really here :P [13:03] jtv, thanks :) [13:03] let [13:03] let's go then [13:03] [TOPIC] Agenda [13:03] * Actions from last meeting [13:03] * Oops report & Critical Bugs & Broken scripts [13:03] * Operations report (mthaddon/Chex/spm/mbarnett) [13:03] * DBA report (stub) [13:03] * Proposed items [13:03] New Topic: Agenda [13:03] [TOPIC] * Actions from last meeting [13:03] * Chex to file a bug about the multiple git import failures on the importd [13:03] * Ursinha to try to come up with an older working version of oops-tools (non oops-db version) [13:03] * Done, it's running on devpad and generating the partial summaries since before holidays. Used to generate the daily summaries as well when the db version fails for any reason. [13:03] * Ursinha to send one email to stub about the db report [13:03] * Sent. stub sent the report. [13:03] New Topic: * Actions from last meeting [13:03] https://lp-oops.canonical.com/oops.py/?oopsid=db [13:04] here.. sorry [13:04] Chex, hey :) [13:04] Chex, so, did you file that bug? [13:04] Ursinha: yes I did, let me get the bug number for you [13:05] thanks Chex [13:05] I'll keep the ball rolling while Chex finds the bug number :) [13:05] [TOPIC] * Oops report & Critical Bugs & Broken scripts [13:06] New Topic: * Oops report & Critical Bugs & Broken scripts [13:06] the failing scripts are being handled already, right jtv? :) [13:06] Ursinha: yes, I got distracted for a while by something about the database, but back on it now [13:06] jtv, haha thanks :) [13:06] and, we had a bunch of DisconnectionErrors on 4th, on staging, and are having them now on lpnet and edge [13:06] I've filed a bug for them, bug 504291, jtv kindly did his analysis [13:06] salgado, could you take this to foundations, please? [13:06] Launchpad bug 504291 in launchpad-foundations "DisconnectionErrors (already disconnected) happening again" [Undecided,New] https://launchpad.net/bugs/504291 [13:07] again!? [13:07] yeah, I'll have a look at it [13:07] salgado, unfortunately :( [13:07] another issue that needs to be investigated by foundations is why is staging not being updated [13:07] mthaddon filed a bug with the error [13:08] bug 504300 [13:08] Ursinha, do you have the number? [13:08] Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300 [13:08] salgado, sorry, was searching the tab [13:08] Ursinha: I filed a bug, and was immediately redirected to this as a duplicate bug: bug #486076. [13:08] Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076 [13:08] right Chex, thanks [13:09] I'll take a look [13:09] Ursinha: which, when I look at it, I am not sure directly addresses our issue, but its a start, I think. [13:09] [action] salgado to take bugs 504300 and 504291 to foundations discussion [13:09] ACTION received: salgado to take bugs 504300 and 504291 to foundations discussion [13:09] Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300 [13:09] Launchpad bug 504291 in launchpad-foundations "DisconnectionErrors (already disconnected) happening again" [Undecided,New] https://launchpad.net/bugs/504291 [13:10] salgado, is that ok for you? [13:10] yep [13:10] right, thanks [13:11] Chex, right, do you think it's not a duplicate? maybe a code guy could enlighten us [13:11] rockstar, ^ [13:11] Ursinha: I was under the impression that if the code tree was very large and big, it overwhelmed the import process, with memory usage.. that doesn't seem like a memory leak to me, but I could be mistaken about that. [13:12] <-- andrea-bs has left this server (Remote closed the connection). [13:12] rockstar doesn't love us anymore [13:13] Chex, we can probably sleuth whether or not it's a duplicate. I'd suggest filing the bug. [13:13] Ursinha: here is the orginal bug I filed: Bug #497945 : Large git imports into bzr tree uses lots of memory [13:13] Launchpad bug 497945 in bzr "Large git imports into bzr tree uses lots of memory (dup-of: 486076)" [Undecided,New] https://launchpad.net/bugs/497945 [13:13] Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076 [13:13] rockstar, ^ [13:14] Ursinha: I can discuss this offline with you and/or rockstar later, not to hold up the meeting [13:14] * rockstar agrees [13:14] Chex, thanks :) [13:14] rockstar, thanks :) [13:14] moving on then [13:14] These kinds of bugs are harder to track down, but we know they exist. [13:15] [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett) [13:15] New Topic: * Operations report (mthaddon/Chex/spm/mbarnett) [13:15] here is our report for this week, folks: [13:15] - LP edge updates broke today (CSS broken) because make build was relying [13:15] on python2.4, salgado is helping to fix this issue now. [13:15] - LP staging updates are broken at the moment, Bug 504300 has been filed [13:15] addressing this issue. [13:15] Launchpad bug 504300 in launchpad "Staging updates broken with "ImportError: No module named _gpgme"" [Undecided,New] https://launchpad.net/bugs/504300 [13:16] - LP seeing intermittent 500s on shipit servers, we are [13:16] investigating that more to see exactly what is causing those. [13:16] - LP incidents of note: [13:16] ; LP Cherry-picks: revno 8809 rolled out to lpnet 30-Dec, [13:16] revno 8810 rollout to lpnet 07-Jan; [13:16] ; Crowberry upgraded from 8Gb to 32GB RAM 23-Dec [13:16] and thats our report for this week, any questions/comments? [13:16] not here [13:16] anyone? [13:16] [action] Chex to discuss with rockstar about bug 497945 [13:16] ACTION received: Chex to discuss with rockstar about bug 497945 [13:16] Launchpad bug 497945 in bzr "Large git imports into bzr tree uses lots of memory (dup-of: 486076)" [Undecided,New] https://launchpad.net/bugs/497945 [13:17] Launchpad bug 486076 in bzr-git "memory leaks" [High,Triaged] https://launchpad.net/bugs/486076 [13:17] all right [13:17] thanks Chex! [13:17] [TOPIC] * DBA report (stub) [13:17] sent by email to the list [13:17] New Topic: * DBA report (stub) [13:17] [TOPIC] * Proposed items [13:17] New Topic: * Proposed items [13:17] anyone? [13:17] Ursinha: your welcome, thanks.. [13:17] no items proposed in the meeting agenda page [13:18] 5 [13:18] 4 [13:18] 3 [13:18] 2 [13:18] 1 [13:18] Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs. [13:18] #endmeeting [13:18] Meeting finished at 10:18.