#format IRC
<Ursinha> #startmeeting
<MootBot> Meeting started at 11:02. The chair is Ursinha.
<MootBot> Commands Available: [TOPIC], [IDEA], [ACTION], [AGREED], [LINK], [VOTE]
<sinzui> me
<Ursinha> sigh
<matsubara> me
<Ursinha> [TOPIC] Roll Call
<Ursinha> Not on the Launchpad Dev team? Welcome! Come "me" with the rest of us! 
<MootBot> New Topic:  Roll Call
<Ursinha> meeeeee
<sinzui> me
<Chex> me
<matsubara> me
<Ursinha> sinzui is here several times
<Ursinha> bigjools, 
<Ursinha> hi
* bigjools excuses himself from the meeting
<Ursinha> I guess we don't have a bugs' person here
<bigjools> UDS...
<Ursinha> bigjools, I  have one question for you in the oops section, should be easy for you :)
<Ursinha> ok
<Ursinha> [TOPIC] Agenda
<Ursinha>  * Actions from last meeting
<Ursinha>  * Oops report & Critical Bugs & Broken scripts
<Ursinha>  * Operations report (mthaddon/Chex/spm/mbarnett)
<Ursinha>  * DBA report (stub)
<Ursinha>  * QA stats of the week
<Ursinha>  * Proposed items
<MootBot> New Topic:  Agenda
<Ursinha> [TOPIC] * Actions from last meeting
<MootBot> New Topic:  * Actions from last meeting
<Ursinha>     * Ursinha to triage lp-qa-tools bugs
<Ursinha>        * done.
<Ursinha>     * matsubara to talk to foundations about OOPS-1585G1243
<Ursinha>         * Talked to Gary about it and filed https://launchpad.net/bugs/579987
<Ursinha>     * adeuring to follow up on calculate_bug_heat script failure
<Ursinha>     * matsubara to chase landing for bug 574493
<Ursinha>         * Landed and CP'ed
<Ursinha>     * matsubara to chase landing for bug 575426
<Ursinha>         * Landed and CP'ed
<Ursinha>     * matsubara to ask for the DBA report
<Ursinha>         * emailed stub asking for it
<Ursinha>     * all QA contacts to take orphaned commits' lists to their teams to check if were left untested items behind because they were orphaned commits
<Ursinha> bot is sleeping 
<Ursinha> my stuff is ok, so is matsubara's
<Ursinha> adeuring is not here to follow up on his item
<Ursinha> [action] Ursinha to chase adeuring abour calculate_bug_heat failing script
<MootBot> ACTION received:  Ursinha to chase adeuring abour calculate_bug_heat failing script
* ubottu has quit (Ping timeout: 260 seconds)
<Ursinha> about the orphaned commits, we can discuss that in the QA section
<Ursinha> moving on
<Ursinha> [TOPIC] * Oops report & Critical Bugs & Broken scripts
<MootBot> New Topic:  * Oops report & Critical Bugs & Broken scripts
<Ursinha> One bug and one oops for code team, and one oops to soyuz
<Ursinha> the code bug is bug 579004, triaged high and targetted to 10.05, seems to affect only staging
<Ursinha> fine
<Ursinha> the oops: rockstar, any ideas? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1593S580
<Ursinha> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1593EC1021, NotOneError seems to be bug 523346, already fix released, but I still see that happening
<Ursinha> bigjools, is that the same thing? ^
<bigjools> Ursinha: gah
<rockstar> Ursinha, I'll file a bug.
<bigjools> it's related
<Ursinha> thanks rockstar
<Ursinha> bigjools, may I file another bug?
<bigjools> Ursinha: yes please, and mention that other one
* ubottu (~supybot@ubuntu/bot/ubottu) has joined #launchpad-meeting
<Ursinha> bigjools, sure
<bigjools> thanks
<Ursinha> [action] Ursinha to file a bug for OOPS-1593EC1021, related to bug 523346
<MootBot> ACTION received:  Ursinha to file a bug for OOPS-1593EC1021, related to bug 523346
<Ursinha> thanks rockstar and bigjools, I love you both
<Ursinha> moving on
<Ursinha> calculate_bug_heat and upgrade_branches were the last scripts to fail
<Ursinha> rockstar, do you know about the upgrade_branches? is that related to code as it seems?
<bigjools> heh :)
<rockstar> Ursinha, yes.  That script is my baby.
<Ursinha> rockstar, oh, it seems it's been crying on his last runs
<Ursinha> or its :)
<rockstar> Ursinha, I'll look into it.
<Ursinha> thanks rockstar 
<Ursinha> much appreciated
<ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/523346)
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1593EC1021
<ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/523346)
<ubottu> https://lp-oops.canonical.com/oops.py/?oopsid=1593EC1021
<rockstar> Ursinha, the conjugated form was correct, since it's a conjugation of "it" and "has"
<Ursinha> [action] rockstar to look into upgrade_branches failures
<MootBot> ACTION received:  rockstar to look into upgrade_branches failures
<Ursinha> rockstar, I was referring to s/his/its/ ;)
<Ursinha> s/;)/:)
<Ursinha> thanks :)
<Ursinha> we have three critical bugs, two already fix committed: bug 574142 and bug 574493
<Ursinha> gary_poster, sinzui, are those two going to be or were CPed? 
<ubottu> Launchpad bug 574142 in launchpad-registry "Could not adapt ProductSeries to IBranchTarget" [Critical,Fix committed] https://launchpad.net/bugs/574142
<ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/574493)
<Ursinha> there's one in progress, code, bug 578331, last comment was danilos'
<Ursinha> how is that?
<ubottu> Launchpad bug 578331 in launchpad-code "exporting to bzr seems broken since a few days" [Critical,In progress] https://launchpad.net/bugs/578331
<mwhudson> i think it's fix committed now?
<danilos> Ursinha, it's on production-stable, needs CPing
<Ursinha> oh, so mwhudson is here
<mwhudson> oops
<Ursinha> camper :)
<mwhudson> my cover is busted
<danilos> https://lpbuildbot.canonical.com/builders/prod_lp/builds/62
<sinzui> Ursinha, it is fix released, I updated it
<sinzui> now
<gary_poster> Ursinha, I'm remebering only one critical and it has been CP'd--the log-out one
<Ursinha> thanks sinzui 
<Ursinha> gary_poster, that one seems private, let me see
<matsubara> Ursinha, 574493 was CP'ed
<gary_poster> bug 574493, right?
<ubottu> Error: Could not parse data returned by Launchpad: list index out of range (https://launchpad.net/bugs/574493)
<gary_poster> that's CP'd
<Ursinha> gary_poster, yes
<gary_poster> cool
<Ursinha> matsubara, gary_poster, so, it's fix released :) could you mark it as such, please?
<Ursinha> or I can do that :)
<gary_poster> will do
<Ursinha> thank you
<Ursinha> danilos, are you taking care of that CP?
<matsubara> I didn't mark as fix released before because of gary's comment saying the fix was CP'd but not landed on devel. I was waiting for it to reach devel a go through the usual process
<matsubara> s/a/and/
<danilos> Ursinha, will do
<gary_poster> matsubara, ack.  It happened early this morning
<Ursinha> matsubara, that's ok, I was just wondering if they could be closed now
<Ursinha> danilos, thanks
<Ursinha> [action] danilos to take care of CPing bug 578331 fix
<MootBot> ACTION received:  danilos to take care of CPing bug 578331 fix
<ubottu> Launchpad bug 578331 in launchpad-code "exporting to bzr seems broken since a few days" [Critical,Fix committed] https://launchpad.net/bugs/578331
<Ursinha> good
<Ursinha> [TOPIC] * Operations report (mthaddon/Chex/spm/mbarnett)
<MootBot> New Topic:  * Operations report (mthaddon/Chex/spm/mbarnett)
<Ursinha> Chex, stage is all yours
<Chex> hello everyone, here is this weeks LOSA report:
<Chex> - Applied LP security fix for Bug# 578914
<Chex>         : Rolled out in security mode, applying CP's to codebrowse & codehost, and cowboy edge.
<Chex> - Bug 531071: app servers leaking memory
<Chex>         : We have a few dump-logs now ( LPIncidentLog ) , has anyone looked at or could look at these to debug the issues here?
<ubottu> Launchpad bug 531071 in launchpad-foundations "lpnet app servers are leaking memory again" [High,Triaged] https://launchpad.net/bugs/531071
<Chex>  - LP incidents of note: : Codebrowse fixed from hourly restarts after last week rollout, but still needs restarting, daily, or more. : ftp/poppy service crashed a few times this week, Bug 414482
<ubottu> Launchpad bug 414482 in soyuz "upload daemon poppy dies often" [Medium,Triaged] https://launchpad.net/bugs/414482
<Chex>         : Apr-11: CP 9339 App Servers & codehost/crowberry
<Chex>         : Apr-12: CP 9341 to germanium/cocoplum
<Chex>         : Apr-12: CP 9342 to lpnet/cocoplum & loganberry-bzrsyncd
<Chex> questions/comments on the report?
<danilos> I guess that's May, or an abbreviation for something like "Anonymous Public Report" :)
<Chex> wow, yes sorry
<Chex> .. s/Apr/May/ my mistake.
<danilos> Chex, should we try to get bigjools to up the priority on poppy bug?
<danilos> Chex, if that's bigjools territory at all?
<bigjools> it is
<bigjools> but it's a bug in zope
<bigjools> so we're working around that by writing a new sftp server :)
<Ursinha> what a workaround :)
<danilos> woohoo, simple workarounds :)
<gary_poster> probably late for this, but I'm always happy to try and apply pressure withing Zope community if necessary
<bigjools> good to know, thanks
<Ursinha> ok, anyone else?
<Ursinha> bug 531071 maybe?
<ubottu> Launchpad bug 531071 in launchpad-foundations "lpnet app servers are leaking memory again" [High,Triaged] https://launchpad.net/bugs/531071
<Ursinha> gary_poster, ^
<Ursinha> this is already targeted to 10.04
<gary_poster> we now have some meliae outputs as of the past week.  I'll be investigating them soon.
<Chex> Ursinha: gary_poster: we have crash dumps of what was requested now on the app servers, if we can work with someone who wants to look at this
<Ursinha> s/already/still/
<Chex> gary_poster: ah fair enough, thank you
<gary_poster> Chex: (I assume they are the same ones you are talking about--spm sent us some emails)
<Ursinha> gary_poster, may I target that to this cycle? 
<Chex> gary_poster: yes I think so, what I saw in the incident logs was in spm
<Chex> 's timeframe, so.
<gary_poster> Chex: cool
<gary_poster> Ursinha: too late ;-) but thanks
<Ursinha> haha, thanks gary_poster :)
<Ursinha> moving on then :)
<Ursinha> [TOPIC] * DBA report (stub)
<Ursinha> [action] send email to stub about the dba report of the week
<MootBot> New Topic:  * DBA report (stub)
<MootBot> ACTION received:  send email to stub about the dba report of the week
<Ursinha> [TOPIC] * QA stats of the week
<MootBot> New Topic:  * QA stats of the week
<Ursinha> Untested items per team:
<Ursinha> Code: 3
<Ursinha> Foundations: 2
<Ursinha> Registry: 0
<Ursinha> Bugs: 3
<Ursinha> Translations: 2
<Ursinha> Soyuz: 3
<Ursinha> Strategy: 0
<gary_poster> Untested == QA?
<gary_poster> duh
<gary_poster> yes
<Ursinha> :)
<Ursinha> one thing I notice is that since we started using tags, we've been having less untested items at once
<Ursinha> which is cool
<Ursinha> :)
<Ursinha> Untriaged bugs per project:
<Ursinha> malone: 17
<Ursinha> launchpad-buildd: 12
<Ursinha> lp-dev-utils: 9
<Ursinha> launchpadlib: 9
<Ursinha> launchpad-dev-moin-theme: 6
<Ursinha> launchpad-code: 5
<Ursinha> launchpad-foundations: 3
<Ursinha> soyuz: 3
<Ursinha> launchpad-help-moin-theme: 3
<Ursinha> lpbuildbot: 2
<Ursinha> trac-launchpad-migrator: 2
<Ursinha> launchpad-news-wordpress-theme: 2
<Ursinha> launchpad-cscvs: 2
<Ursinha> launchpad: 1
<Ursinha> launchpad-web: 1
<Ursinha> launchpad-loggerhead: 1
<Ursinha> launchpad-documentation: 1
<Ursinha> tickcount: 1
<gary_poster> agreed
<Ursinha> we're not that bad triaging bugs as well
<Ursinha> about the orphaned commits
<Ursinha> last week I asked the QA contacts to check with their teams all OCs that are yours, to see if we're leaving untested things behind
<Ursinha> sinzui answered me, what about the others?
<matsubara> I suck. I forgot to bring that up with my team
<Ursinha> https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.04-devel and https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.04-db-devel
<Ursinha> rockstar, and you?
<rockstar> We talked about it.  I don't know if anyone did anything about it.
<sinzui> I will bring up the branch-link-to-bug issue in the next reviewers meeting. Many contributors do not do this
<danilos> Ursinha, I totally don't remember the discussion, which means that I suck
<Ursinha> :)
<sinzui> We could make ec2 land require a linked bug
<Ursinha> c'mon people :)
<Ursinha> sinzui, guess the problem is that not all commits really require bugs
<rockstar> Oh no, I think we did talk about it, and determined that there were no orphaned commits.
<Ursinha> and not everyone uses ec2
<wgrant> What about refactorings?
<sinzui> right
<Ursinha> ah, hello wgrant :)
<rockstar> However, we're also not linking bugs to branches in many cases.
<wgrant> Ursinha: Yeah, I'm not asleep for once :P
<sinzui> pqm submit certain is not
<Ursinha> wgrant, :P
<rockstar> Ursinha, we haven't really been tracking QA in the bugtracker - We've been using our Kanban board.
<matsubara> if you use --fixes=lp:NNNN when you commit, then when you push the branch to LP, it'll be linked automatically
<Ursinha> rockstar, I see. this is not good, but we fall again in the kanban board/lp integration
<Ursinha> problem
<matsubara> and the tagging script will find out which bug was fixed by that commit, even if it's not mentioned in the commit message
<rockstar> Ursinha, well, we have two tools essentially doing the same thing.
<Ursinha> rockstar, also what matsubara said
<sinzui> I still use the milestone because I am tracking every registry related blueprint and bug that I may be required to land a RC for
<danilos> Ursinha, fwiw, I've looked through the orphan commits now and there is only one by a translations dev and that one was qa-ok :)
<Ursinha> I don't know what everyone else agrees, but I think we should maintain lp updated so when we integrate that with kanban we'll all be used to the lp dance
<Ursinha> thanks danilos 
<sinzui> Every contributor or even developer from another team gets their in progress bugs targets to a milestone and I verify the number of fix committed is the same number as qa-ok
<rockstar> Ursinha, asking us to keep two duplicate tools in sync manually seems crazy.
<gary_poster> Foundations often has multiple commits as we pursue a larger bug
<Ursinha> rockstar, but is that hard to use --fixes or link a bug to a branch?
<sinzui> danilos, the registry orphaned commits were for contributor branches that we landed and for testfix
<rockstar> Ursinha, no, but I don't want to create a kanban task AND a bug for something I'm about to do.
<danilos> sinzui, I count adiroiban as a translations developer as well :)
<rockstar> And then move them both through their own separate processes.
<gary_poster> We also have kanban cards for our tasks
<rockstar> I can't use --fixes if there isn't a bug...
<Ursinha> rockstar, I thought that the whole team had agreed on using lp as qa tracker
<sinzui> He applied to be an lp engineer. He should be counted on someone's team
<Ursinha> did I get this wrong?
<rockstar> Ursinha, maybe, but the code team isn't doing it that way, and we've found that the kanban board has been the tool we've preferred.
<Ursinha> I guess that needs to be discussed then
<rockstar> Ursinha, we get our stuff QA'd, so it's not like things are falling through the cracks, but it doesn't have great visibility.
<Ursinha> if you're preferring another tool, we should discuss the problems
<rockstar> Ursinha, the problem is that we have two tools...
<gary_poster> Ursinha: do I understand correctly that you are saying that orphan commits are bad?
<Ursinha> rockstar, imo the problem is that we don't have kanban integration
<danilos> Ursinha, rockstar: it's a recognized problem, and flacoste promises us we'll do something about it (two tools for the same job)
<Ursinha> gary_poster, only if they are really orphaned
<gary_poster> because of the QA-through-LP initiative?
<danilos> Ursinha, let's move on
<danilos> at least that's my take on it
<matsubara> gary_poster, not necessarily bad, we just don't want those falling through the cracks
<Ursinha> gary_poster, in the sense that it should have been qaed and it isn't 
<Ursinha> what matsubara said
<gary_poster> ...and if we use LP for qa then how do we know?
<gary_poster> right
<matsubara> if you use LP as the QA tracker, then we have tools to help remind you that some QA is needed
<gary_poster> I think this is a matter for discussion, but maybe not here.  mailing list?
* mwhudson has quit (Ping timeout: 276 seconds)
<Ursinha> doesn't matter if it doesn't have a bug when it doesn't need to have, the problem is not having a bug when it should
<Ursinha> agreed gary_poster 
<gary_poster> cool
<matsubara> if you start landing OCs, then the tools can't help remind you that QA needs to be done
<Ursinha> so, in this meantime, I'd like to ask you to once in a while take a look in the OC list
<Ursinha> and see if any important things are missing
<Ursinha> https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.05-devel and https://wiki.canonical.com/Launchpad/QATeam/OrphanedCommits/10.05-db-devel this cycle
<Ursinha> anyone else want to say something?
<Ursinha> moving on :)
<Ursinha> [TOPIC] * Proposed items
<Ursinha> Thank you all for attending this week's Launchpad Production Meeting. See https://dev.launchpad.net/MeetingAgenda for the logs.
<Ursinha> #endmeeting
<MootBot> New Topic:  * Proposed items
<MootBot> Meeting finished at 11:39.
<Ursinha> errrrr bad paste :)
<Ursinha> but we didn't have any other proposed items, so, nevermind :)
<gary_poster> :-)
<matsubara> thanks Ursinha
<gary_poster> thank you
<Ursinha> thanks everyone for this week's meeting :)