1 <INTRO> IRC session "Getting started with Launchpad development"
   2 
   3 <INTRO> Given 2 September 2009 by Graham Binns (gmb) as part of Ubuntu Dev Week; see https://wiki.ubuntu.com/MeetingLogs/devweek0909/LPDevelopment.
   4 
   5 <INTRO> Take it away, Graham!...
   6 
   7 <gmb> My name's Graham Binns. I'm a member of the Launchpad Bugs
   8       development team.
   9 
  10 <gmb> I'm going to talk today about getting started with Launchpad
  11       development, in the hope that it might make it easier for you guys
  12       to contribute patches to scratch your least favourite itches.
  13 
  14 <gmb> Hopefully you'll have all completed the instructions at
  15       http://dev.launchpad.net/Getting so that you can follow along with
  16       this session. If not, you might struggle a bit, but you can always
  17       go back once the session is over and follow it through on your own
  18       time.
  19 
  20 <gmb> If you've any questions, please shout them out in
  21       #ubuntu-classroom-chat and prefix them with QUESTION so that I can
  22       see them easier :)
  23 
  24 <gmb> Okay, so, first things first, we need to find us a bug to fix. For
  25       the purposes of this session I've filed a made-up bug on staging
  26       for us to fix https://staging.launchpad.net/bugs/422299. I've gone
  27       with this because:
  28 
  29 <gmb> 1) It's fairly simple to fix. 2) It's easy to demonstrate our
  30       test-driven development process whilst we fix it, which is why I
  31       didn't pick a bug in the UI. 3) There were no really trivial bugs
  32       available for us to try this out on :).
  33 
  34 <gmb> When you're working on fixing a bug in Launchpad, you nearly
  35       always want to be doing it in a new branch.
  36 
  37 <gmb> We try to keep to one bug per branch, because that means that it's
  38       much easier to review the patches when they're done (because
  39       they're smaller, natch :))
  40 
  41 <gmb> So, let's create a branch in which to fix the bug.
  42 
  43 <gmb> If you've set up the Launchpad development environment properly
  44       according to http://dev.launchpad.net/Getting, you should be able
  45       to run the following command:
  46 
  47 <gmb> $ rocketfuel-branch getting-started-with-lp-bug-422299
  48 
  49 <gmb> Note that I've appended the bug number to the branch
  50 
  51 <gmb> so that I can always refer to it if I need to
  52 
  53 <gmb> but I've also given the branch a useful name to help me remember
  54       what it's for if I have to leave it for a while.
  55 
  56 <gmb> rocketfuel-branch takes a few seconds, so I'll just wait a minute
  57       for everyone to catch up.
  58 
  59 <gmb> (By the way, if anyone has any problems with rocketfuel-get or any
  60       other part of this lesson, please come find me afterwards in
  61       #launchpad and I'll try to help you out)
  62 
  63 <gmb> s/-get/-branch/ there, sorry.
  64 
  65 <gmb> Okay.
  66 
  67 <gmb> Now, at this point, once you'd decided how to fix the bug
  68 
  69 <gmb> but - importantly - before you start coding
  70 
  71 <gmb> you'd ideally have a chat with a member of the Launchpad
  72       development team about your intended fix.
  73 
  74 <gmb> We normally do this either on IRC or on Skype, depending on your
  75       preference.
  76 
  77 <gmb> You can usually find a Launchpad developer in #launchpad-dev on
  78       Freenode who'll be available for one of these calls.
  79 
  80 <gmb> The call gives you a chance to ensure that what you're doing is
  81       actually sane.
  82 
  83 <gmb> For some bugs there's only one possible fix, complex or
  84       otherwise. For others there may be many ways to do it, and it's
  85       important to pick the right one.
  86 
  87 <gmb> If your solution is particularly complex or you need to
  88       demonstrate *why* you want to do things the way you do, it may
  89       help to write some tests to reproduce the bug before you have the
  90       call.
  91 
  92 <gmb> Note that the tests should always fail at this point;
  93 
  94 <gmb> you shouldn't make any changes to the actual code until you've had
  95       the pre-implementation call or chat with an LP developer.
  96 
  97 <gmb> Okay, so that's the info-dumpy bit of this session over for now :)
  98 
  99 [*** gmb accidentally drops from channel for momentarily ***]
 100 
 101 <gmb> Sorry about that, all.
 102 
 103 <gmb> I have a rather flaky connection today :)
 104 
 105 <gmb> As I was saying...
 106 
 107 <gmb> Under lib/lp you'll find most of the Launchpad code, split up into
 108       its applications.
 109 
 110 <gmb> So, `ls lib/lp` in your new getting-started-with-lp-bug-422299
 111       branch should give you something like this:
 112 
 113 <gmb> $ ls lib/lp
 114 
 115 <gmb> answers           archiveuploader  buildmaster  coop
 116       registry  soyuz
 117 
 118 <gmb> app               blueprints       code         __init__.py
 119       scripts   testing
 120 
 121 <gmb> archivepublisher  bugs             codehosting  __init__.pyc
 122       services  translations
 123 
 124 <gmb> Now, we know that we're working in the bugs application, so lets
 125       take a look in there to see where to put our tests:
 126 
 127 <gmb> $ ls lib/lp/bugs
 128 
 129 <gmb> adapters        emailtemplates      help          model
 130       stories      windmill
 131 
 132 <gmb> browser         event               __init__.py   notifications
 133       subscribers  xmlrpc
 134 
 135 <gmb> configure.zcml  externalbugtracker  __init__.pyc  pagetests
 136       templates
 137 
 138 <gmb> doc             feed                interfaces    scripts
 139       tests
 140 
 141 <gmb> There are three types of test in Launchpad: doctests, which live
 142       in lib/lp/$app/doc; stories, which live in lib/lp/$app/stories and
 143       unittests, which live in lib/lp/$app/tests.
 144 
 145 <gmb> In this case we want to add to an existing doctest, so I'll stick
 146       with that for now and we can come back to what the others are for
 147       later.
 148 
 149 <gmb> So, in lib/lp/bugs/doc/ you'll find a file called
 150       externalbugtracker-trac.txt.
 151 
 152 <gmb> This is the test we want to modify, so feel free to open it in
 153       your text editor and take a look at line 110, which is where we're
 154       going to add our test.
 155 
 156 <gmb> For the sake of making this quicker, I've already created a diff
 157       of the change that I'd make here:
 158       http://pastebin.ubuntu.com/263869/plain/
 159 
 160 <gmb> You can save that to disk somewhere (e.g. /tmp/diff) and then
 161       apply it as a patch using `bzr patch /tmp/diff` in the root of
 162       your new Launchpad branch.
 163 
 164 <gmb> The test we've just added is really simple.
 165 
 166 <gmb> It passes 'frobnob' to the convertRemoteStatus() method of a Trac
 167       instance (which is just an abstraction that lets us talk to an
 168       actual Trac server)
 169 
 170 <gmb> and expects to get "Fix Released" back.
 171 
 172 <gmb> Of course, it doesn't since we haven't implemented that yet :).
 173 
 174 <gmb> Once we've written the test, we run it to make sure it fails.
 175 
 176 <gmb> This part is very important: your tests should always fail first
 177       and only after they fail do you write the code to make them pass.
 178 
 179 <gmb>  That means that you can use the tests to build a good spec of how
 180       your module / class / function / whatever should behave.
 181 
 182 <gmb> It also means that, like I said before, you can use the failing
 183       tests to demonstrate what your fix will actually change to whoever
 184       you have a call with.
 185 
 186 <gmb> To run this specific test only, we use the `bin/test` command:
 187 
 188 <gmb> $ bin/test -vvt externalbugtracker-trac.txt
 189 
 190 <gmb> That might take a short while to run (Launchpad's test suite can
 191       be frustratingly slow sometimes, but don't let that put you off;
 192       the payoff is worth it)
 193 
 194 <gmb> The output from which should look something like this:
 195       http://pastebin.ubuntu.com/263874/
 196 
 197 <gmb> Note the important bit:
 198 
 199 <gmb>     File "lib/lp/bugs/tests/../doc/externalbugtracker-trac.txt",
 200       line 111, in externalbugtracker-trac.txt
 201 
 202 <gmb>     Failed example:
 203 
 204 <gmb>         trac.convertRemoteStatus('frobnob').title
 205 
 206 <gmb>     Exception raised:
 207 
 208 <gmb>         Traceback (most recent call last):
 209 
 210 <gmb>           File
 211       "/home/graham/canonical/lp-sourcedeps/eggs/zope.testing-3.8.1-py2.4.egg/zope/testing/doctest.py",
 212       line 1361, in __run
 213 
 214 <gmb>             compileflags, 1) in test.globs
 215 
 216 <gmb>           File "<doctest externalbugtracker-trac.txt[line 111,
 217       example 35]>", line 1, in ?
 218 
 219 <gmb>           File
 220       "/home/graham/canonical/lp-branches/lesson/lib/lp/bugs/externalbugtracker/trac.py",
 221       line 265, in convertRemoteStatus
 222 
 223 <gmb>             raise UnknownRemoteStatusError(remote_status)
 224 
 225 <gmb>         UnknownRemoteStatusError: frobnob
 226 
 227 <gmb> This tells us that the test failed, which is exactly what we
 228       wanted.
 229 
 230 <gmb> (Yes, copying and pasting in IRC makes me a bad man.)
 231 
 232 <gmb> nvertRemoteStatus() raised an UnknownRemoteStatusError instead of
 233       giving us back the status we wanted.
 234 
 235 <gmb> Which was, of course, the 'Fix Released' status.
 236 
 237 <gmb> At this point, you might want to commit the changes:
 238 
 239 <gmb> $ bzr commit -m "Added tests for bug 422299."
 240 
 241 <gmb> Again - I can't emphasise this enough - the fact that your test
 242       fails is a Good Thing. If it didn't fail, it wouldn't be a good
 243       test, since we know that the bug actually exists in the code.
 244 
 245 <gmb> Now that we have a test that fails, we want to add some code to
 246       make it pass
 247 
 248 <gmb> We want to add this to lib/lp/bugs/externalbugtracker/trac.py.
 249 
 250 <gmb> Now, as it happens, I knew that before I started, but you can work
 251       it out by looking at the top of the doctest file that we just
 252       edited.
 253 
 254 <gmb> So, open lib/lp/bugs/externalbugtracker/trac.py now and take a
 255       look at line 258. We'll add our fix here.
 256 
 257 <gmb> The fix is really simple, and we can pretty much copy line 255 and
 258       alter it to suit our needs.
 259 
 260 <gmb> We want 'frobnob' to map to 'Fix Released', so we add the
 261       following line:
 262 
 263 <gmb>     ('frobnob', BugTaskStatus.FIXRELEASED),
 264 
 265 <gmb> I'll not go into the nitty-gritty of how status lookups work here,
 266       because it's unimportant.
 267 
 268 <gmb> Suffice it to say that in Trac's case it's a simple pair of
 269       values, (remote_status, launchpad_status).
 270 
 271 <gmb> Here's a diff of that change: http://pastebin.ubuntu.com/263882/
 272 
 273 <gmb> Now that we've added a fix for the bug, we run the test again:
 274 
 275 <gmb> $ bin/test -vvt externalbugtracker-trac.txt
 276 
 277 <gmb> This time, it should pass without any problems...
 278 
 279 <gmb> and it does
 280 
 281 <gmb> http://pastebin.ubuntu.com/263885/
 282 
 283 <gmb> So, now we commit our changes:
 284 
 285 <gmb> $ bzr ci -m "Fixed bug 422299"
 286 
 287 <gmb> (Note that this is a lame description of the fix; you should use
 288       something more descriptive).
 289 
 290 <gmb> So, we now have a branch that fixes a bug. Hurrah and all that.
 291 
 292 <gmb> Now we need to get it into the Launchpad tree.
 293 
 294 <gmb> Launchpad developers use the Launchpad code review system to
 295       review Launchpad branches.
 296 
 297 <gmb> You can't land a branch without having it reviewed first
 298 
 299 <gmb> This allows us to ensure that code quality stays high
 300 
 301 <gmb> And it also acts as a  sanity check to make sure that the
 302       developer hasn't done something unnecessarily odd in their fix.
 303 
 304 <gmb> So at this point, you need to push your branch to Launchpad using
 305       the `bzr push` command:
 306 
 307 <gmb> $ bzr push
 308 
 309 <gmb> Once the branch has been pushed up to Launchpad it gets its own
 310       page in the Launchpad web interface, which you can look at by
 311       running:
 312 
 313 <gmb> $ bzr lp-open
 314 
 315 <gmb> This should open the page in your default browser.
 316 
 317 <gmb> Now that you've fixed the bug and pushed the branch to Launchpad
 318       you need to request a review for it.
 319 
 320 <gmb> To do this, go to the branch page in your browser and click the
 321       "Propose for merging into another branch" link.
 322 
 323 <gmb> This will take you to a page that looks like this:
 324 
 325 <gmb> http://people.ubuntu.com/~gbinns/propose-merge.png
 326 
 327 <gmb> In the "Initial comment" box, you need to type a description of
 328       the branch.
 329 
 330 <gmb> For example, for this branch I'd write something like:
 331 
 332 <gmb> "This branch fixes bug 422299 by making Trac.convertRemoteStatus()
 333       map the "frobnob" status to Launchpad's Fix Released status."
 334 
 335 <gmb> After you've typed in your description, hit the "Propose merge"
 336       button and you should see a page that looks something like this:
 337       https://code.edge.launchpad.net/~gmb/launchpad/lesson/+merge/11068
 338 
 339 <gmb> You then need to head on over to #launchpad-reviews on Freenode
 340       and ask if anyone's available to review your branch.
 341 
 342 <gmb> If there's no-one available at the time, don't worry.
 343 
 344 <gmb> We have a reviewer schedule:
 345       http://dev.launchpad.net/ReviewerSchedule, so someone should take
 346       a look at it withing 24 hours.
 347 
 348 <gmb> The reviewer may ask you to make changes to your branch
 349 
 350 <gmb> To bring your fix into line with our coding standards
 351 
 352 <gmb> Or maybe to fix a bug that they've spotted in your fix.
 353 
 354 <gmb> Once the reviewer has signed off on the changes, they'll submit
 355       the branch for merging for you.
 356 
 357 <gmb> When a branch gets merged, the entire test suite is run against it
 358 
 359 <gmb> If any of the tests fail
 360 
 361 <gmb> The reviewer may ask you to help fix them
 362 
 363 <gmb> But it's likely that someone else will take care of it if you're
 364       not around at the time
 365 
 366 <gmb> And that's about all there is to simple Launchpad development :)
 367 
 368 <gmb> Are there any questions? Please shout them out in
 369       #ubuntu-classroom-chat
 370 
 371 <gmb> < ahe> QUESTION: When will launchpad be available as a package in
 372       the standard distribution?
 373 
 374 <gmb> ahe: At this point, there aren't any plans for that. We released
 375       the code for Launchpad because we wanted to let people help to
 376       improve the service, but we've no plans as far as I'm aware to
 377       distribute it as a package.
 378 
 379 <gmb> < Andphe> question: have you planned guys, offer launchpad in
 380       another languages than english, example spanish ?
 381 
 382 <gmb> Andphe: It's something that we've considered and that we would
 383       like to do at some point, at least for certain parts of the
 384       interface.
 385 
 386 <gmb> The problem is that launchpad is meant to be a global
 387       collaboration tool, and if we translate it wholesale into other
 388       languages that automatically means that a certain amount of
 389       collaboration will be lost
 390 
 391 <gmb> For exampel, if a user reads the interface in Spanish and files a
 392       bug in Spanish, how am I, an non-Spanish speaker, going to be able
 393       to deal with that bug report?
 394 
 395 <gmb> However, internationalisation would work quite well for the
 396       Answers application, and it's already built with that in mind.
 397 
 398 <gmb> < ahe> QUESTION: Do you deploy launchpad manually or are there
 399       some helper scripts or stuff like that to ease the deployment in a
 400       production environment?
 401 
 402 <gmb> It's a combination of the two.
 403 
 404 <gmb> edge.launchpad.net is deployed by a script every night, as is
 405       staging.launchpad.net.
 406 
 407 <gmb> The production servers are updated manually by our sysadmins at
 408       least once per cycle (though it's usually more than that since we
 409       discover urgent bugs that need to be fixed).
 410 
 411 <gmb> < Andphe> question: if answers already support another languages,
 412       how can we help to translate it ?
 413 
 414 <gmb> Andphe: It's built with translation in mind, but I don't know what
 415       work needs doing to make it translatable.
 416 
 417 <gmb> Andphe: Your best bet would be to join the Launchpad Developers
 418       mailing list (http://launchpad.net/~launchpad-dev) and post a
 419       question about it there.
 420 
 421 <gmb> I think that's about all we've got time for.
 422 
 423 <gmb> If you've any further questions, please feel free to join the
 424       Launchpad Dev list (above)
 425 
 426 <gmb> And ask there.
 427 
 428 <gmb> Everyone's welcome to contribute.
 429 
 430 <gmb> Thanks very much for your time.