| Nahsei | good night | 00:03 |
|---|---|---|
| === IdleOne_ is now known as IdleOne | ||
| Bree | hi all | 10:35 |
| === deegee_1 is now known as deegee | ||
| === Knightlust is now known as Igorots | ||
| danfish | quit | 13:23 |
| === Igorots is now known as Knightlust | ||
| === Hutley_ is now known as Hutley | ||
| dholbach | WELCOME TO THE LAST DAY OF UBUNTU DEVELOPER WEEK! | 15:59 |
| dholbach | As you can see on https://wiki.ubuntu.com/UbuntuDeveloperWeek we have a bunch of GREAT sessions lined up for today | 15:59 |
| dholbach | If you have any questions, please ask them in #ubuntu-classroom-chat and prefix them with QUESTION: | 16:00 |
| dholbach | ie: QUESTION: Does Jono really just listen to Death Metal music? | 16:00 |
| dholbach | and if you'd prefer not to ask questions in English have a look at the bottom of https://wiki.ubuntu.com/UbuntuDeveloperWeek - we have a couple of channels where you can ask questions in your native language :) | 16:01 |
| dholbach | (and consider helping out there too) | 16:01 |
| dholbach | ok... I just learned that our first speaker has trouble making it to the session | 16:02 |
| dholbach | I'll make sure that he'll repeat the session some time later | 16:02 |
| dholbach | I'm sorry | 16:02 |
| dholbach | I'd suggest we make this a Question and Answer session | 16:02 |
| dholbach | feel free to ask every development-related question you can think of during this session and I'll try to get you an answer :) | 16:03 |
| dholbach | <Daviey> QUESTION: Does Jono really just listen to Death Metal music? | 16:03 |
| === godzilla is now known as Guest57848 | ||
| dholbach | Daviey: I'm afraid he almost exclusively listens to Death Metal. Sometimes there's some AC/DC too and sometimes some ABBA, but 95% Death Metal :) | 16:03 |
| dholbach | I'm sure that jcastro can confirm that ^ | 16:04 |
| jcastro | I can't neither confirm nor deny any ABBA listenage | 16:04 |
| dholbach | do we have more questions about Ubuntu and Ubuntu development? anything that was unclear during the week? | 16:05 |
| dholbach | <mhall119|work> QUESTION: should tabs be represented as 4 spaces or 8? Vi or Emacs? Hungarian notation? | 16:05 |
| dholbach | mhall119|work: I guess that depends who you ask :-) | 16:05 |
| dholbach | mhall119|work: if you mainly work with python, you might want to check out http://www.python.org/dev/peps/pep-0008/ which at least has something to say about spaces | 16:05 |
| dholbach | <lbrinkma> QUESTION: Is there any advanced guide to packaging, for example many banarie packages out of one source package? | 16:06 |
| dholbach | lbrinkma: if you worked your way through https://wiki.ubuntu.com/PackagingGuide I suggest check out http://www.debian.org/doc/debian-policy/ (Debian Policy) and http://www.debian.org/doc/maint-guide/ (Debian New Maintainer's Guide) | 16:06 |
| dholbach | plus checking out a lot of source packages and see how they do it | 16:07 |
| dholbach | I'm no expert, but a lot I know today I learnt from other packages and checking out what other maintainers did | 16:07 |
| dholbach | that's one of the things I LOVE about Ubunt, everything is just an "apt-get source ..." away :) | 16:07 |
| dholbach | <duanedesign56> <QUESTION> Have you used 'Groundcontrol' yet and how do you see that being integrated into the community? | 16:08 |
| dholbach | duanedesign56: I personally didn't | 16:08 |
| dholbach | jcastro: ^ did you? | 16:08 |
| jcastro | I saw the video | 16:08 |
| jcastro | it's a nautilus integration thing for bzr | 16:08 |
| dholbach | oh nice | 16:08 |
| dholbach | does it integrate with olive (bzr-gtk)? | 16:08 |
| dholbach | <duanedesign56> dholbach: he has mentioned bzr-gtk integration in a future release | 16:09 |
| dholbach | I guess that answers the question :) | 16:09 |
| dholbach | any more questions? | 16:10 |
| dholbach | maybe something that wasn't explained due to time constraints? | 16:10 |
| dholbach | <amichair> QUESTION: in utilities that are developed "in-house" by (k)ubuntu, rather than just packaged, is there a proper QA cycle, or is it just an 'ok' from the dev followed by waiting for bug reports to flow in? | 16:14 |
| dholbach | amichair: yes, although there's no strict one-QA-process-for-everything there's quite a lot of QA being done | 16:15 |
| dholbach | tools like bazaar and launchpad which are mainly developed by Canonical engineers are developed through test-driven development and have extensive test-suites that are run for every merge | 16:16 |
| dholbach | (and run several hours in the case of Launchpad AFAIK) | 16:16 |
| dholbach | tools that actually are in Ubuntu, like upstart have testsuites too | 16:16 |
| dholbach | http://mago.ubuntu.com/ is an approach to test desktop applications by test-driving them through the accessibility interface (IIRC) | 16:17 |
| dholbach | the security and SRU team add test-cases for stuff that broke to ensure it doesn't break again | 16:17 |
| dholbach | and we do quite a lot of certification testing and ISO testing of CD images | 16:18 |
| dholbach | I'm sure I forgot heaps of things | 16:18 |
| dholbach | jcastro: anything I missed? | 16:18 |
| jcastro | I don't know anything about testing | 16:19 |
| dholbach | #ubuntu-testing and #ubuntu-bugs and #ubuntu-qa are good places to talk about that | 16:19 |
| dholbach | (and we had a session about automated server testing too! :) | 16:19 |
| dholbach | <donaldharvey> QUESTION: I'm a Git/Github user, and am really comfortable with the system. I maintain a project on Launchpad and use bzr-git to import the existing repository. Should I switch to bzr, and if so, why? | 16:19 |
| dholbach | donaldharvey: what you could easily do without having to switch tools is setting up a bzr in code.launchpad.net | 16:20 |
| dholbach | erm sorry | 16:20 |
| dholbach | a bzr import | 16:20 |
| dholbach | that'd give people the opportunity to use code.launchpad.net to propose merges when they're used to using that already | 16:20 |
| dholbach | I love using code.launchpad.net and made great experiences with new contributors who quickly "got it" and knew how it worked | 16:21 |
| dholbach | <donaldharvey> That's what I'm doing currently, yeah | 16:21 |
| dholbach | I have little experience with git to be honest, but a lot of good with bzr, so I'm probably the wrong person to talk to :-) | 16:21 |
| dholbach | but the fine people in #bzr and #launchpad can probably give you some further advice | 16:22 |
| dholbach | <duanedesign56> <QUESTION> Are there any charts or graphs that show bug stats plotted over time? | 16:23 |
| dholbach | jcastro: ^? | 16:23 |
| dholbach | I know there are some on qa.ubuntu.com | 16:23 |
| dholbach | let me see if I can spot them easily | 16:24 |
| jcastro | not anything that isn't on qa.ubuntu.com | 16:24 |
| jcastro | for large packages they have this | 16:24 |
| jcastro | http://status.qa.ubuntu.com/qapkgstatus/knetworkmanager | 16:24 |
| jcastro | as an example | 16:24 |
| jcastro | but nothing that summarizes everything. | 16:25 |
| jcastro | this would be good info to get started on | 16:25 |
| jcastro | for example look at this one: http://status.qa.ubuntu.com/qapkgstatus/gnome-power-manager | 16:26 |
| jcastro | oldest bug with status NEW: 446 days | 16:26 |
| jcastro | that means someone either can't confirm the bug or it's been forgotten | 16:26 |
| jcastro | jml mentions that he has this: http://people.canonical.com/~jml/lp-bugs/ | 16:27 |
| jml | jcastro, it's a prototype that uses out-of-date data and is a chart of the bugs on Launchpad itself | 16:28 |
| jml | jcastro, I'd be over the moon with ecstasy and joy if someone wanted to turn that into an actual page on Launchpad, and would be more than happy to help whoever wants to do it | 16:29 |
| jcastro | graphing in general would be useful I think | 16:29 |
| jml | heck yes. | 16:29 |
| dholbach | <breinera> <QUESTION> as a newbie to Ubuntu developer, but someone who can program, where can I make the most impact and help Ubuntu? | 16:33 |
| dholbach | breinera: awesome question! | 16:33 |
| === R_ is now known as Guest24303 | ||
| dholbach | the most important thing you need to have as somebody who wants to help out with Ubuntu is: a knack for making things work again, some patience, some humour and being a team player | 16:34 |
| dholbach | I'd recommend having a look at https://wiki.ubuntu.com/MOTU/TODO for now until we managed to give http://daniel.holba.ch/harvest a facelift | 16:34 |
| dholbach | we soon will have a nice overview over tasks that need some help and group them by packages and everything | 16:35 |
| dholbach | until then it's this a bit ugly list of lists | 16:35 |
| dholbach | so... what do we have there? | 16:35 |
| dholbach | jcastro mentions 'bitesize' bugs | 16:35 |
| dholbach | we tag bugs in Launchpad to indicate what kind of task there are | 16:35 |
| dholbach | 'bitesize' for example indicates "this is a job to get started with" | 16:36 |
| dholbach | 'update' means "this package needs updating to a new upstream version" | 16:36 |
| dholbach | 'packaging' means "here's something wrong with the packaging" | 16:36 |
| dholbach | etc. | 16:36 |
| cjwatson | I'd also be inclined to say that it's always most effective to follow your own interests | 16:36 |
| cjwatson | people tend to be more productive with things they care about | 16:36 |
| dholbach | the list of official tags can be found here: https://wiki.ubuntu.com/Bugs/Tags | 16:36 |
| dholbach | thanks cjwatson - absolutely agreed | 16:37 |
| dholbach | Ok, maybe we should broaden the scope of the session somewhat and ask for some feedback about the whole week... what do you think about the week? what do you think went well, what didn't? | 16:38 |
| dholbach | (if you still have questions, that's fine too) | 16:39 |
| dholbach | Any feedback? | 16:41 |
| dholbach | Any questions? :) | 16:41 |
| dholbach | maybe requests for the next Ubuntu Developer Week? | 16:41 |
| dholbach | or other development related classes outside of UBuntu Developer Week? | 16:43 |
| dholbach | <SevenMachines> [QUESTION] i was wondering if theres a difference in focus over the development cycle, ie, work on merging, fixing bugs in ubuntu, or getting bugs fixed in debian | 16:43 |
| dholbach | SevenMachines: definitely | 16:43 |
| dholbach | one key decision is to try to land as many earth-shattering changes as early as possible so they can get tested and debugged most appropriately | 16:44 |
| dholbach | also if you find out that Debian suffers from the same problem and you want to fix it there too, then early in the cycle is a good time for that | 16:44 |
| dholbach | the later it gets in our cycle the more we exclusively focus on just fixing stuff | 16:45 |
| dholbach | it doesn't make sense to first push the critical fix to Debian, wait for it to get included, then maybe sync or merge it again | 16:45 |
| dholbach | http://people.canonical.com/~dholbach/cheatsheet.jpg tries to give a bit of an overview over release-cycle decisions | 16:45 |
| dholbach | <dscassel> QUESTION: Any suggestions for running a successful Global Jam? Focus more on giving people the tools to help, versus actual triage and packaging? | 16:46 |
| dholbach | jcastro: ^ you want to take that one? | 16:46 |
| dholbach | dscassel: I'd try to do everything that the people want to do | 16:48 |
| dholbach | dscassel: if some of them are very new to the processes and to Ubuntu they will need all the help they can get | 16:48 |
| jcastro | https://wiki.ubuntu.com/Jams is a place to start | 16:48 |
| jcastro | the key is to ask around for groups that have done jams in the past | 16:49 |
| jcastro | so you don't make a simple mistake | 16:49 |
| dholbach | I'm in the Ubuntu Berlin team and we made good experiences with splitting up stuff among team members: one organises bug triage efforts, somebody else does packaging, etc. | 16:49 |
| jcastro | for example our first jam we did in a bar and then didn't realize that under-21 people couldn't attend! | 16:49 |
| jcastro | now we have that in a wiki page so people don't repeat our mistake | 16:50 |
| dholbach | haha :) | 16:50 |
| dholbach | more questions? more feedback? more requests? | 16:51 |
| dholbach | ok my friends, thanks a lot for your questions | 16:55 |
| dholbach | let's all take a 5 minute break before cjwatson is up with "Doing merges right!" | 16:55 |
| dholbach | Next up is the incredible and unstoppable Colin Watson, talking to us about "Doing merges right!" | 17:00 |
| * cjwatson fusses around a bit looking for his notes. :-) | 17:00 | |
| cjwatson | Hi, I'm Colin Watson, from the Ubuntu Foundations team, and this talk is called "Doing Merges Right". | 17:00 |
| cjwatson | You can ask questions at any time in #ubuntu-classroom-chat. Please write QUESTION at the beginning of your question so I can see it more easily. I'll stop at the end of a section and answer some questions. | 17:00 |
| cjwatson | This session is mostly for people who've never done merges before and want to know how to do it properly, though it will help if you know what a patch looks like. | 17:01 |
| cjwatson | If you already know how to do merges, though, consider listening in anyway. I'm going to describe up-to-date tools that not everyone will already be familiar with, and I'll cover a few common mistakes along the way. | 17:01 |
| cjwatson | Apologies to the desktop folks, but this is unashamedly a terminal-based session. It is not full of sexy slides and whizzy graphics, because I don't do those. Love The Terminal. The Terminal Is Your Friend. | 17:01 |
| cjwatson | To follow along with this talk, you should install the bzr, bzr-builddeb, and ubuntu-dev-tools packages. Make sure that bzr is version 2.0 or better; karmic's is fine. | 17:01 |
| cjwatson | WHY | 17:01 |
| cjwatson | Every time we open a new development release of Ubuntu, we need to merge our changes with those made in Debian. | 17:01 |
| cjwatson | Not everyone finds this exciting, but it's really important; it keeps our repository up to date with the rest of the free software world, and it gives us an opportunity to audit the changes we're carrying and discard ones that don't make sense any more. | 17:02 |
| cjwatson | Since we have a lot of packages and a relatively small number of developers, it's important that we have some common understanding of how to go about this. | 17:02 |
| cjwatson | Plus, half the time you end up merging something you haven't worked on for months if at all, so some basic habits come in handy ... | 17:02 |
| cjwatson | In this session, I'm going to run through an example merge using a couple of different methods. | 17:02 |
| cjwatson | WHEN | 17:02 |
| cjwatson | http://merges.ubuntu.com/ is your friend for finding things to do. The lists are divided into a section for packages that haven't been uploaded so far this release, and a section for packages that have been touched but still have a newer version in Debian. The first section is usually more important. | 17:02 |
| cjwatson | Each merge has a "last uploader" recorded alongside it. | 17:02 |
| cjwatson | The last person who uploaded the package is the default assignee for each merge. We sometimes call this the "touched it last" method - you touched it last, it's yours! | 17:03 |
| cjwatson | You must always contact the last uploader before doing a merge; please don't just run down the list and submit lots of merges for sponsorship. | 17:03 |
| cjwatson | This is a convention to try to avoid duplicate work. You may also often find that the assignee for a merge has a reason for not just going ahead and merging (perhaps some other package needs to be adjusted), so always talk to them first. | 17:03 |
| cjwatson | That said, people often welcome help with their merge workload as long as they're told about it, so don't be shy about asking. | 17:03 |
| cjwatson | We expect all packages to have been merged at least once by Debian Import Freeze (see https://wiki.ubuntu.com/ReleaseSchedule) and after that point we don't actively try to keep the merge lists short. | 17:03 |
| cjwatson | After Feature Freeze, you have to have a good reason for doing a merge, not just "it was on the list", so expect to be asked searching questions by your sponsor if they can't see why you bothered. :-) | 17:04 |
| cjwatson | HOW | 17:04 |
| cjwatson | I'm going to go through the same example in two different ways. Firstly, I'm going to use Bazaar, the revision control system used for (among other things) a good deal of Ubuntu development. Secondly, I'm going to do things the old-fashioned way. | 17:04 |
| cjwatson | Bazaar is really much better at dealing with merges, since that's one of the core functions of a revision control system! For the example here, it doesn't make so much difference, but it gets more obvious with more complicated packages. | 17:04 |
| cjwatson | The Bazaar workflow isn't so extensively covered in MOTU documentation, so I'm going to spend a bit more time on it. | 17:04 |
| cjwatson | However, we're still in the process of rolling this out for all packages, and it doesn't quite all work yet. Sometimes you need to fall back to the old-school method to get your work done. | 17:04 |
| cjwatson | Don't worry too much about having to do this; the automatic imports will catch up even if you do your work outside Bazaar. | 17:05 |
| cjwatson | But first: | 17:05 |
| cjwatson | THE TAO OF MERGING | 17:05 |
| cjwatson | 1. Keep Ubuntu changes as small as possible. If Debian fixes the same bug in a different way, use their fix and discard ours unless it's clearly better in some way. | 17:05 |
| cjwatson | 2. Send changes upstream or to Debian whenever possible. We don't really want to maintain big divergent patches in Ubuntu if we can avoid it; unless things are truly Ubuntu-specific, it's always better to pass them up so that the rest of the free software world can use our improvements too. You can use the 'submittodebian' script to help with this. | 17:05 |
| cjwatson | 3. Sync with Debian whenever possible. When following these directions, if you find that there are no Ubuntu changes left, then stop! Request that the package be synced instead: https://wiki.ubuntu.com/SyncRequestProcess | 17:05 |
| cjwatson | 4. Write good changelogs that explain what changes we're still carrying in Ubuntu. This makes it easy to look at a package and quickly see how the Ubuntu version differs. | 17:05 |
| cjwatson | Any questions before I dive into our first example? | 17:06 |
| cjwatson | Shout if I'm going too fast. | 17:06 |
| cjwatson | 17:06 <randomaction> QUESTION: what do background colours in MoM mean? | 17:07 |
| cjwatson | It's odd that this isn't documented anywhere ... | 17:07 |
| cjwatson | (at least as far as I can see) | 17:07 |
| cjwatson | Anyway, the colours refer to the Priority field of the package. | 17:07 |
| cjwatson | Red is required, sort of orange/yellow is important, light green is standard, mid-green is optional, dark green is extra. | 17:08 |
| cjwatson | This is just a heuristic to try to sort more important packages to the top of the list. | 17:08 |
| cjwatson | It doesn't refer to the importance of the *merge*, which can't be determined automatically, but it helps. | 17:08 |
| cjwatson | (I think I might have the precise colour/priority mapping a little wrong, but you get the idea!) | 17:09 |
| cjwatson | OK, I'll carry on. | 17:09 |
| cjwatson | BAZAAR | 17:09 |
| cjwatson | I'm going to use a simple merge that's currently outstanding as an example: ipmitool. | 17:09 |
| cjwatson | Look at https://merges.ubuntu.com/universe.html and (if it hasn't been merged yet by the time you read this) you'll see ipmitool there, with Chuck Short's name against it as the last uploader. | 17:09 |
| cjwatson | Normally, you can access current branches for Ubuntu and Debian packages at lp:ubuntu/SOURCE or lp:debian/SOURCE respectively, where SOURCE is the source package name. | 17:09 |
| cjwatson | (You can use 'bzr lp-open' to look at these in a web browser, too.) | 17:10 |
| cjwatson | These are special aliases that expand to five-level names; for instance, lp:ubuntu/ipmitool is currently lp:~ubuntu-branches/ubuntu/lucid/gnu-fdisk/lucid. This breaks down as (owner, distribution, release, source package, branch name). | 17:10 |
| cjwatson | err, s/gnu-fdisk/ipmitool/ obviously! Sorry, error in my prep ... | 17:11 |
| cjwatson | Debian branches currently expand to .../debian/squeeze/..., since for the Lucid development cycle we're merging from Debian testing, not unstable. This may change in future. To get branches corresponding to unstable, use lp:debian/sid/SOURCE. | 17:11 |
| cjwatson | In order that the transcript of this talk will continue to be useful for some time to come, I've taken copies of the current state of the Ubuntu and Debian branches for ipmitool, and pushed them to Launchpad, using my own Launchpad username as the owner. Anyone can push whatever they like this way, as long as it meets the Launchpad terms of use. | 17:11 |
| === l is now known as Guest68800 | ||
| cjwatson | lp:~cjwatson/ubuntu/lucid/ipmitool/udw | 17:11 |
| cjwatson | lp:~cjwatson/debian/squeeze/ipmitool/udw | 17:11 |
| cjwatson | BAZAAR: STARTING THE MERGE | 17:11 |
| cjwatson | Let's prepare to do our example merge. | 17:11 |
| cjwatson | Make a new directory and cd into it. Type 'bzr init-repo .', which should respond with a few lines starting "Shared repository with trees (format: 2a)". This is optional, but it means that any branches we create under here will share common storage for their revisions, which saves you both disk space and network bandwidth when you have several branches with shared history. | 17:11 |
| cjwatson | Fetch the branches we want to merge like this: | 17:12 |
| cjwatson | bzr get -r7 lp:~cjwatson/ubuntu/lucid/ipmitool/udw ubuntu | 17:12 |
| cjwatson | bzr get lp:~cjwatson/debian/squeeze/ipmitool/udw debian | 17:12 |
| cjwatson | (Normally, of course, this would be lp:ubuntu/ipmitool etc., and we pick a specific revision of the Ubuntu branch here because the next revision has the results of the merge.) | 17:12 |
| cjwatson | If you've used Bazaar before, you may be reaching for 'bzr merge' about now. bzr-builddeb provides a 'bzr merge-package' command, which we're going to use instead. I'll explain the difference later on. | 17:13 |
| cjwatson | Let's try merging the Debian changes into the Ubuntu branch. Bazaar will automatically start from wherever the two branches you're working on diverged; you don't need to tell it "merge from 1.8.11-1 up to 1.8.11-2". | 17:13 |
| cjwatson | <cjwatson@sarantium ~/src/ubuntu/ipmitool/udw/ubuntu>$ bzr merge-package ../debian | 17:13 |
| cjwatson | Text conflict in debian/changelog | 17:13 |
| cjwatson | Text conflict in debian/control | 17:13 |
| cjwatson | Text conflict in debian/patches/series | 17:13 |
| cjwatson | 3 conflicts encountered. | 17:13 |
| cjwatson | The merge resulted in 3 conflicts. Please resolve these and commit the changes with "bzr commit". | 17:13 |
| cjwatson | BAZAAR: CONFLICTS | 17:13 |
| cjwatson | That was easy, wasn't it? Shame about the conflicts. Type 'bzr status' and you'll see what this has left in your working tree. Incidentally, you can get detailed help on any Bazaar command using 'bzr help COMMAND'. | 17:14 |
| cjwatson | (I'll pause for a bit here to let anyone following along catch up with the commands) | 17:14 |
| cjwatson | 17:15 <SevenMachines> i'm getting 'bzr get lp:~cjwatson/debian/squeeze/ipmitool/udw debian | 17:15 |
| cjwatson | 17:15 <SevenMachines> sorry, bzr: ERROR: unknown command "merge-package" | 17:15 |
| cjwatson | SevenMachines: You need to install the 'bzr-builddeb' package. | 17:15 |
| cjwatson | 17:01 <@cjwatson> To follow along with this talk, you should install the bzr, bzr-builddeb, and ubuntu-dev-tools packages. Make sure that bzr is version 2.0 or better; karmic's is fine. | 17:15 |
| cjwatson | presentation syndrome here, just fixing up the branch :-) | 17:18 |
| cjwatson | I've fixed the broken example Ubuntu branch now, so try again if you had trouble earlier. | 17:20 |
| cjwatson | so, carrying on ... | 17:21 |
| cjwatson | Looking at 'bzr status', this has changed four files, only one of which was entirely without incident (the new file debian/patches/passwd_option). Let's look at each of the three conflicts. I normally do debian/changelog last, since it's the summary of everything else I've done. | 17:21 |
| cjwatson | In each file with a conflict, you'll see "<<<<<<<", "=======", and ">>>>>>>" markers. These are placed each time Bazaar finds a part of the file that's changed on both sides of the merge. | 17:21 |
| cjwatson | The section between "<<<<<<<" and "=======" shows that part of the file in the current branch, and the section between "=======" and ">>>>>>>" shows that part of the file in the branch you're merging from. | 17:21 |
| cjwatson | You're expected to remove the markers once you've resolved the conflict. | 17:21 |
| cjwatson | Edit debian/control and you'll see markers around Maintainer and Build-Depends lines. Bazaar has left some files around that you can use to figure out what the change from the base revision on each side was: | 17:21 |
| cjwatson | diff -u debian/control.BASE debian/control.THIS | 17:22 |
| cjwatson | diff -u debian/control.BASE debian/control.OTHER | 17:22 |
| cjwatson | So in fact just the Maintainer lines changed on the Ubuntu side, and only the Build-Depends line changed on the Debian side. We can safely take the XSBC-Original-Maintainer and Maintainer lines from our side and the Build-Depends line from the Ubuntu side. Bazaar errs on the side of caution when changes are very close together in a file like this. | 17:22 |
| cjwatson | Save this and move on to debian/patches/series. Here we have a real conflict: one line was added on the Ubuntu side, and a different line on the Debian side. | 17:23 |
| cjwatson | However, if we look at the patch files in debian/patches/ that these lines are referring to, we can see that they touch totally different files and do completely different things, so we should just keep both. | 17:23 |
| cjwatson | (In my sample merge, I did this by moving the Ubuntu patch below the Debian one, so that all the Ubuntu work was "on top" at the end. In this case it isn't really important though.) | 17:24 |
| cjwatson | Now for debian/changelog. This will get more automatic in future, but for now you usually just get a big conflict with the Ubuntu changelog on one side and the Debian changelog on the other. Edit this so that the changelog entries come in version order (1.8.11-1ubuntu1 is less than 1.8.11-2, so it should come further down the file. | 17:24 |
| cjwatson | Now that we've gone through all the files, we can tell Bazaar that we've done so. Type 'bzr resolve'. (You normally don't need to tell it which files you've resolved; it will look at all the conflicted files and see if any conflict markers are still there.) In this case it says "All conflicts resolved" and we can be happy; otherwise we might need to go back for another look. | 17:25 |
| cjwatson | CHANGELOG | 17:25 |
| cjwatson | We're done now, right? Er, not quite. We still need to write a changelog entry for our merge. You can use 'dch -i' to create a skeleton one. | 17:25 |
| cjwatson | The style of this new changelog entry is important. It should describe all the important changes that are still present in Ubuntu after merging from Debian. | 17:25 |
| cjwatson | You don't need to list details that are intrinsic to it being an Ubuntu upload; don't clutter up changelogs with comments about Maintainer field changes, for instance. | 17:25 |
| cjwatson | You don't necessarily need to list any changes that you've dropped (if they've been merged into Debian, say) unless there's something particularly interesting about this. | 17:26 |
| cjwatson | Normally, you should try to focus on the *functional* changes ("Add widget creation ability") rather than painstakingly listing every file that's changed. The revision control system can tell people about the latter if they really care. | 17:26 |
| cjwatson | If the previous merge changelog entry matches whatever your personal standards of clarity are, then you can just copy and paste it. In this case (with due respect to the previous merger, who seemed to have a complicated job on his hands!), I chose to write my own: | 17:26 |
| cjwatson | ipmitool (1.8.11-2ubuntu1) UNRELEASED; urgency=low | 17:26 |
| cjwatson | * Merge from Debian testing. Remaining changes: | 17:26 |
| cjwatson | - Add Dell-specific commands. | 17:26 |
| cjwatson | -- Colin Watson <cjwatson@ubuntu.com> Fri, 29 Jan 2010 12:59:16 +0000 | 17:26 |
| cjwatson | (there were some more blank lines in there, lost in the process of pasting into IRC; fill them in mentally) | 17:27 |
| cjwatson | If there are any merge bugs in Launchpad, remember to close them here, using the "(LP: #NNNNNN)" syntax. | 17:27 |
| cjwatson | BAZAAR: FINISHING OFF | 17:27 |
| cjwatson | Have a look at your changes to make sure they're sensible. 'bzr status' shows a summary. 'bzr diff' shows the detailed patch you're about to commit, relative to what's currently in Ubuntu. You might also want to check the resulting Ubuntu changes relative to what's in Debian; for that, use 'bzr diff --old ../debian'. | 17:27 |
| cjwatson | Of course there's lots more stuff you can do here - you can use the full power of the revision control system. See online documentation for more. | 17:28 |
| cjwatson | All is well, so we can commit. The changelog says UNRELEASED, so this is marked as a work in progress, but we might as well record what we've done. | 17:28 |
| cjwatson | bzr commit -m 'merge from Debian 1.8.11-2' | 17:28 |
| cjwatson | Bazaar won't let you commit if there are still unresolved conflicts, which is a great protection against accidents. | 17:28 |
| cjwatson | Note that Bazaar doesn't commit merges automatically if they succeed; you still have the opportunity to look over them and make sure you're happy before actually changing your branch. | 17:29 |
| cjwatson | Let's now try building our package. Install the build-dependencies if you need to, and then type 'bzr bd' (short for builddeb). (This is like debuild, and in fact you can use debuild if you're careful and know what you're doing, but 'bzr bd' is a bit smarter about some details of Bazaar branches.) | 17:29 |
| cjwatson | Have a look at the top of the output. You'll notice that we never downloaded ipmitool_1.8.11.orig.tar.gz, but it finds it anyway because James Westby is a GENIUS. | 17:29 |
| cjwatson | Actually, the automatic package imports store enough data in each branch to be able to reconstruct the original source tarball. They use 'pristine-tar' for this, which is quite space-efficient. | 17:29 |
| cjwatson | 'bzr merge-package', and its relatives, 'bzr import-dsc' and 'bzr merge-upstream', take care of details like this, which is why it's worth using them rather than plain 'bzr merge'. | 17:30 |
| cjwatson | (They also make sure that there's something in the history representing each upstream release. If you want the details, look at 'bzr log -n0'.) | 17:30 |
| cjwatson | Now, as it happens, my package fails to build due to some horrible disagreement between autoconf and libtool. But dealing with autotools problems is a subject for another talk! | 17:30 |
| cjwatson | If you need to fix something like this, you can do so in a separate commit from the main merge; that way people can easily see the new changes you've made in the history. Make sure to document any new changes like this in debian/changelog. | 17:30 |
| cjwatson | Let's pretend it all worked, and I want to build a package for upload to Ubuntu (or perhaps to a PPA). | 17:31 |
| cjwatson | Run 'dch -r' to finalise the changelog, and 'bzr bd -S -- -v1.8.11-1ubuntu1' to build a source package. | 17:31 |
| cjwatson | The '-v1.8.11-1ubuntu1' bit says that you want all the changelog entries since 1.8.11-1ubuntu1 to end up in the .changes file. This means that people following along with the lucid-changes mailing list will get complete information about what changed, rather than just the relatively uninformative merge changelog entry. | 17:31 |
| cjwatson | If there's a new upstream version not already in Ubuntu, you'll need the -sa option as well to upload the .orig.tar.gz. | 17:32 |
| cjwatson | Assuming this works, you can look at the result in the parent directory. I'd recommend using 'debdiff' to compare your source package both to the last version in Ubuntu and to the version in Debian that you're merging. You don't need to be too obsessive about this, but a quick debdiff check has caught a lot of basic errors in the past. | 17:32 |
| cjwatson | 17:32 <sebner> cjwatson: [QUESTION]: What was the repo-init thing for at the beginning? | 17:32 |
| cjwatson | 'bzr init-repo .' sets up a shared repository. This means that, instead of having to store all the revisions for the Ubuntu branch and all the revisions for the Debian branch separately, you can store them in the common repository since actually several of them are just the exact same revisions. | 17:33 |
| cjwatson | It also means that you don't have to download those revisions twice, which saves a couple of megabytes. If your connection to the Internet is anything like mine, you might value this! | 17:34 |
| cjwatson | (carrying on) | 17:34 |
| cjwatson | If it all works, then 'debcommit -r' (this commits and tags), and upload! | 17:34 |
| cjwatson | You can push stuff to personal branches on Launchpad for review using 'bzr push lp:~YOURUSERNAME/ubuntu/lucid/SOURCE/BRANCH-NAME', and set up a merge proposal. | 17:34 |
| cjwatson | Using the merge proposal, you can generate debdiffs for bug reports in case your sponsor prefers that. | 17:35 |
| cjwatson | I have more links on this sort of stuff for further reading at the end, but any questions on the Bazaar section of this session before I carry on? | 17:36 |
| cjwatson | 17:36 <sebner> [QUESTION]: How can I haXX0r bzr to work like git when using bzr log. (It does the same horrible thing as svn, showing the first revisions instead of the recent ones and backscrolling is not always possible) | 17:36 |
| cjwatson | sebner: I'm not sure I understand your first problem, as bzr log shows the most recent revisions first | 17:36 |
| cjwatson | as does git log | 17:36 |
| cjwatson | but you can do 'bzr log --forward' if you want to change that | 17:37 |
| cjwatson | if you want a pager, | less :-) | 17:37 |
| cjwatson | It's often helpful to use 'bzr log -n0', which shows all merged revisions as well. | 17:37 |
| cjwatson | also 'bzr alias' is a generally useful thing to know about, for when you want to change a command's default options, or even add a new command that's more to your liking. | 17:42 |
| cjwatson | 17:42 <geser> I usually do "bzr ci" and "bzr mark-uploaded" instead of "debcommit". Should I switch to debcommit or doesn't it matter? | 17:42 |
| cjwatson | geser: debcommit saves you from having to type commit messages again in many cases. Aside from that, right now in practice they're equivalent. | 17:43 |
| cjwatson | If you work with lots of different revision control systems, debcommit acts as a partial abstraction layer, which I find moderately useful. But it's mainly avoiding having to type the same thing into the changelog and the commit log editor that I find useful. | 17:44 |
| cjwatson | 17:43 <sebner> [QUESTION]: How many % of the archive is already usable with such branches and actually being used currently | 17:45 |
| cjwatson | I don't have a percentage figure (maybe jml does, or james_w if he's watching?), but coverage has got to the point where if you're comfortable with the workflow it's worth trying. | 17:46 |
| cjwatson | http://package-import.ubuntu.com/failures/ has a log of current import failures. | 17:46 |
| cjwatson | (which is a bit longer than I remember, I wonder if there's some systemic problem right now) | 17:47 |
| jml | cjwatson, Last time james_w told me it was in the high 90s. | 17:47 |
| cjwatson | the main thing people are likely to run into is that it doesn't handle some cases of dpkg v3 source packages right now. I expect this will be fixed pretty soon. | 17:47 |
| cjwatson | Anyway, I'd better move on now as I have 13 minutes left. | 17:47 |
| cjwatson | OLD-SCHOOL | 17:47 |
| cjwatson | It turns out that it's kind of hard to import lots of versions of 15000-odd packages from two different distributions and get all the bits lined up properly. Who'd have thought? | 17:47 |
| cjwatson | Eventually this will settle down, but for now you'll still need to do things by hand every now and then. | 17:47 |
| cjwatson | merges.ubuntu.com can help with this. https://merges.ubuntu.com/i/ipmitool/REPORT shows the results of an automatic merge, and you'll see that there were a few conflicts, basically just what we went through above. | 17:48 |
| cjwatson | You can use 'grab-merge ipmitool' to get all the files. This gives you an 'ipmitool' directory with lots of files in it, including an 'ipmitool-1.8.11-2ubuntu1' subdirectory with the best that merge-o-matic could manage. | 17:48 |
| cjwatson | As before, you should go through and resolve the conflicts. There's no equivalent to 'bzr resolve', though also no protection against accidentally leaving conflicts around; you have to keep track of this yourself. | 17:48 |
| cjwatson | merge-o-matic has merged the changelogs for you, but you still need to write a changelog entry, as before. Make sure to change the changelog trailer line to your name and e-mail address rather than "Ubuntu Merge-o-Matic <mom@ubuntu.com>", or risk embarrassment! | 17:48 |
| cjwatson | Type '../merge-buildpackage' when you're done. This is like the 'bzr bd -S' bit above. Then unpack the source package somewhere and check that it builds. | 17:48 |
| cjwatson | Please use debdiff to make sure your changes against both Debian and Ubuntu make sense. This is much more important when you aren't using Bazaar, because you can't use 'bzr diff' to check each individual change you're making, and it's possible to leave conflicts in by mistake. | 17:49 |
| cjwatson | To get a sponsor to look at this, you'll need to generate debdiffs against both Debian and Ubuntu, attach them to a merge bug, and subscribe either ubuntu-main-sponsors or ubuntu-universe-sponsors as appropriate. There's more detail on this in the MOTU wiki area. | 17:49 |
| cjwatson | And remember: if you're actually going to ask for sponsorship for this merge, you should still check with the previous uploader, even though I used it as an example for this session! | 17:50 |
| cjwatson | FURTHER READING | 17:50 |
| cjwatson | For more details on merge proposals, see https://wiki.ubuntu.com/DistributedDevelopment/Documentation/Merging, and https://wiki.ubuntu.com/DistributedDevelopment/Documentation for general docs on using Bazaar for Ubuntu packaging. | 17:50 |
| cjwatson | For general advice on merging (particularly "old-school"), see https://wiki.ubuntu.com/UbuntuDevelopment/Merging, which has more detail than I could fit into this session. | 17:50 |
| cjwatson | FUTURE | 17:50 |
| cjwatson | Ultimately, we want to get to the point where every upstream has a Bazaar import, every Debian package has a Bazaar branch matched up with that import, and every Ubuntu package has a Bazaar branch matched up with both of those. | 17:50 |
| cjwatson | The closer we get to this, the easier it will be to merge code between upstream, Debian, and Ubuntu. We have this for a few select packages already: I handle grub2 and openssh this way and it's lovely. | 17:50 |
| cjwatson | If you want to help get us there for more packages, please join https://lists.ubuntu.com/mailman/listinfo/ubuntu-distributed-devel. | 17:51 |
| cjwatson | Thank you for following along, and happy merging! | 17:51 |
| cjwatson | Any questions? | 17:51 |
| cjwatson | 17:51 <sebner> [QUESTION]: Are there any consequences when using bzr to work on a package, then another(?) developer merges some new versions/revisions by hand and then you decide to merge the newest version again with bzr? | 17:51 |
| cjwatson | The automatic import script is reasonably clever (although it could be improved). | 17:51 |
| cjwatson | If it tries to import and finds that there are more revisions beyond the last thing it did, it imports, but moves the "conflicting" revisions aside and files a bug on the 'udd' project about them. | 17:52 |
| cjwatson | The main thing that's lacking from this right now is good notification to the people responsible for those changes. | 17:52 |
| cjwatson | But at least nothing is lost. | 17:52 |
| cjwatson | If you make changes in bzr and *do* upload them, then as long as you tag your upload the automatic importer will take that as evidence that that's a good representation of that version. | 17:53 |
| cjwatson | So if I uploaded 1.8.11-2ubuntu1 from the ipmitool example above and pushed my changes to lp:ubuntu/ipmitool, and then somebody else uploaded 1.8.11-2ubuntu2 without using bzr, that would all work fine - the automatic import would work on top of my changes. | 17:54 |
| cjwatson | 17:51 <vocx> QUESTION: With all the tools that exist today. Is it significantly easier to merge and patch Debian packages? I have a feeling that aspiring maintainers may be overwhelmed by the multitude of "bzr action" commands, along with the traditional "dpkg" tools. | 17:54 |
| cjwatson | So, there's always a learning curve, and it's certainly a bit difficult in a transitional period when you have to learn two sets of tools. | 17:55 |
| cjwatson | However, hardly any serious upstream project works without revision control. | 17:55 |
| cjwatson | So I think it's quite justifiable to try to move Ubuntu to having consistent revision control - in fact I think the end result will be much less confusing. | 17:55 |
| cjwatson | My experience is that once you've acquired basic familiarity with the relevant revision control tools, merging and patches is much, much less time-consuming, and the error rate is much lower. | 17:56 |
| cjwatson | I also hope that ultimately it will be easier to build ways to encourage aspiring developers this way. When you get right down to it, having to send repeated versions of debdiffs around isn't all *that* great either ... | 17:57 |
| cjwatson | Thanks again. If you have other questions, feel free to ask me directly or on #ubuntu-devel or #ubuntu-motu. | 17:59 |
| cjwatson | I'll hand over now to Jonathan Lange, a gentleman and a scholar, who's going to be talking about launchpadlib. | 17:59 |
| jml | cjwatson, why thank you | 17:59 |
| jml | as before, please stop me if I go too fast, or hassle me if I go too slow | 18:00 |
| jml | You might know already, but pretty much no matter what kind of Ubuntu development you are doing, you'll have to interact with Launchpad | 18:00 |
| jml | Interacting with Launchpad is fun and all, but sometimes you'll want a computer to do the interaction for you | 18:00 |
| jml | Say you want to suck out some of the data and make a pretty graph | 18:00 |
| jml | or you want to build a site like http://qa.ubuntu.com/ | 18:00 |
| jml | This session should help you do that... | 18:00 |
| jml | Before we begin, you should install the 'python-launchpadlib' package now. | 18:00 |
| jml | Confirm that it works by running: | 18:01 |
| jml | $ python | 18:01 |
| jml | and then | 18:01 |
| jml | >>> import launchpadlib | 18:01 |
| jml | >>> launchpadlib.__version__ | 18:01 |
| jml | you should get output like: | 18:01 |
| jml | '1.5.1' | 18:01 |
| jml | if it's an earlier version, then you need to either update the package, upgrade to a newer version of Ubuntu, or find the PPA. | 18:01 |
| jml | A moment for those installing the package | 18:01 |
| jml | Moment over. | 18:01 |
| jml | This session is going to have four parts: theory, example, gotchas and future. | 18:01 |
| jml | = 1. Theory = | 18:02 |
| jml | The Launchpad developers implemented an API for three reasons: | 18:02 |
| jml | 1. To let you get at your data | 18:02 |
| jml | 2. To allow you to do project-specific policy stuff | 18:02 |
| jml | 3. To watch people do cool things we would never have thought of. | 18:02 |
| jml | (I'm a Launchpad developer, hence the 'we') | 18:02 |
| jml | We decided to implement the API using something called RESTful Architecture | 18:02 |
| jml | http://en.wikipedia.org/wiki/Representational_State_Transfer | 18:02 |
| jml | This means that Launchpad's APIs are actually language neutral | 18:03 |
| jml | in theory, you can write code to control Launchpad in any language you want | 18:03 |
| jml | without screenscraping! | 18:03 |
| jml | You do HTTP requests with operations like GET, POST, PUSH and DELETE and send and receive JSON content that's described by a WADL file | 18:03 |
| jml | WADL == http://en.wikipedia.org/wiki/Web_Application_Description_Language | 18:03 |
| jml | Which is all nice and wonderful but you don't really have to care about it, because there's also a Python library | 18:03 |
| jml | called "launchpadlib" | 18:03 |
| jml | (have you installed it yet?) | 18:04 |
| jml | If you don't care at all about Python, now's the time to stop following this session and go to https://help.launchpad.net/API instead | 18:04 |
| jml | Because I'm about to go into the example. | 18:04 |
| jml | = 2. Example = | 18:04 |
| jml | You can follow along from this example by getting the branch at lp:~jml/+junk/bugstats | 18:04 |
| jml | You do that by running 'bzr branch lp:~jml/+junk/bugstats' | 18:04 |
| jml | The way to follow along is to change into that branch, and then run 'bzr revert -r 1' | 18:04 |
| jml | and then when I say "Next revision", do the same thing with "-r 2" instead | 18:04 |
| jml | and so on | 18:04 |
| jml | bzr revert -r1 | 18:05 |
| jml | The branch contains a single file 'bugstats.py' | 18:05 |
| jml | If you look at that file in your favorite text editor, you'll see that it's very very simple | 18:05 |
| jml | It imports sys, defines a main function that does nothing, and then runs that main function. | 18:05 |
| jml | Go ahead and run it now | 18:05 |
| jml | $ python bugstats.py | 18:05 |
| jml | It should do absolutely nothing, fairly quickly | 18:05 |
| jml | (at the speed of PYTHON!) | 18:05 |
| jml | OK. Next revision | 18:05 |
| jml | bzr revert -r2 | 18:06 |
| jml | (sorry alt-tab fail) | 18:06 |
| jml | Have a look at this version of bugstats.py. You might need to reload it in your editor. | 18:06 |
| jml | This is an extremely simple launchpadlib script | 18:06 |
| jml | It logs in to launchpad and prints out the title of bug number one | 18:06 |
| jml | I'll break down exactly what the script does in a moment. | 18:06 |
| jml | But first, you should run the script. | 18:06 |
| jml | $ python bugstats.py | 18:06 |
| jml | When you run it, an authorization page will open up in your web browser | 18:06 |
| jml | This is very important, this is how your Launchpad web credentials get applied to the application | 18:07 |
| jml | It's done using oauth, which I don't understand. | 18:07 |
| jml | http://oauth.net/ and http://en.wikipedia.org/wiki/OAuth are probably good places to start though. | 18:07 |
| jml | For now, pick "Read non-private data" | 18:07 |
| jml | Then, switch back to your terminal and press enter. | 18:07 |
| jml | The script should then print the title of bug number one. | 18:07 |
| jml | Run the script again | 18:07 |
| jml | Notice that launchpadlib has cached your credentials | 18:07 |
| jml | launchpad = Launchpad.login_with(APP_NAME, SERVICE_ROOT, CACHE_DIR) | 18:08 |
| jml | ^^^ that's the line of code what did it | 18:08 |
| jml | if you look at the script... | 18:08 |
| jml | you'll see I've imported 'os' and 'sys', which are standard Python modules. | 18:08 |
| jml | I've also imported 'Launchpad' and 'STAGING_SERVICE_ROOT' from launchpadlib | 18:08 |
| jml | Then I've defined an application name, a caching directory and a service root | 18:08 |
| jml | APP_NAME = 'jml-bug-stats' | 18:08 |
| jml | CACHE_DIR = os.path.expanduser('~/.launchpadlib/cache') | 18:08 |
| jml | SERVICE_ROOT = STAGING_SERVICE_ROOT | 18:08 |
| jml | There's nothing special about the names of the variables, I've just made them ALL CAPS to hint that they are constants | 18:08 |
| jml | APP_NAME is the name of the application. launchpadlib & Launchpad use this as a marker for authorization. You authorize an application based on its *name*. | 18:08 |
| jml | Go to https://staging.launchpad.net/people/+me/+oauth-tokens and search for "jml-bug-stats" | 18:09 |
| jml | you'll only find it if you actually ran the script | 18:09 |
| jml | if you have run the script, you should find it there, saying when you authorized it and what you authorized it to do | 18:09 |
| jml | You could revoke the authorization if you wanted to, but you shouldn't do that just now | 18:09 |
| jml | Anyway, that 'jml-bug-stats' is the app name used in the script. | 18:09 |
| jml | SERVICE_ROOT is _which_ Launchpad.net instance to use | 18:10 |
| jml | does everyone know about staging.launchpad.net, edge.launchpad.net and so forth? | 18:10 |
| jml | ... | 18:10 |
| jml | that's a yes. | 18:10 |
| jml | Today, we're using staging. Testing against staging is a good idea, since it's impossible to mess up your live production data by messing with staging. | 18:10 |
| jml | CACHE_DIR is the place where launchpadlib stores cached credentials and cached json objects | 18:10 |
| jml | It's worth having a poke around in it sometime | 18:11 |
| jml | most apps use ~/.launchpadlib/cache | 18:11 |
| jml | for reasons that escape me | 18:11 |
| jml | If you run ls -l ~/.launchpadlib/cache/api.staging.launchpad.net/credentials/ you should see a file called 'jml-bug-stats' | 18:11 |
| jml | That file contains the cached credentials for this app | 18:11 |
| jml | If you delete it, you'll have to re-authorize the application | 18:11 |
| jml | moving on to the script proper. | 18:11 |
| jml | The main() function has us logging into Launchpad and printing a bug title | 18:11 |
| jml | login_with is a handy convenience function that logs in with an application name, but looks in the cached credentials first to see if you've already logged in. It returns an instance of 'Launchpad' | 18:11 |
| jml | 'launchpad' is an object with a bunch of pre-defined top-level collections, and is the starting point of any launchpadlib app. | 18:11 |
| jml | https://help.launchpad.net/API/launchpadlib#The%20top-level%20objects will list all of them for you | 18:12 |
| jml | Here we use 'bugs', get the bug with id '1' and print its title. | 18:12 |
| jml | $ python bugstats.py | 18:12 |
| jml | Microsoft has a majority market share | 18:12 |
| jml | Easy | 18:12 |
| jml | Now for something a bit more useful | 18:12 |
| jml | Next revision | 18:12 |
| jml | bzr revert -r3 | 18:12 |
| jml | $ python bugstats.py | 18:12 |
| jml | Please specify a project and a username. | 18:12 |
| jml | $ python bugstats.py ubuntu jml | 18:13 |
| jml | Jonathan Lange has had 2 bugs fixed on Ubuntu | 18:13 |
| jml | it shows the number of bugs someone has had fixed | 18:13 |
| jml | useful, huh? | 18:13 |
| jml | take a look at the code | 18:13 |
| jml | It's got a work-around for a bug in launchpadlib to get the length of a returned collection. | 18:13 |
| jml | I don't really understand why the bug isn't fixed yet, since it's got to bite almost everyone who writes a non-trivial launchpadlib application. | 18:13 |
| jml | Anyway, this script is more advanced than the last one. | 18:13 |
| jml | look at ... | 18:14 |
| jml | pillar = launchpad.projects[pillar_name] | 18:14 |
| jml | "Pillar" is Launchpad developer jargon for a distribution, project or project group. e.g. ubuntu, gnome-do or gnome | 18:14 |
| jml | you can try running the script with your username and fave project | 18:14 |
| jml | The script takes the pillar name from the command line, along with a bug reporter name, finds all of the 'fix released' bugs and then prints out a friendly message. | 18:14 |
| jml | reporter = launchpad.people[reporter_name] | 18:14 |
| jml | fixed_bugtasks = pillar.searchTasks( | 18:14 |
| jml | bug_reporter=reporter, status=['Fix Released']) | 18:14 |
| jml | If I run it with this: | 18:15 |
| jml | $ python bugstats.py launchpad-code jml | 18:15 |
| jml | I get: | 18:15 |
| jml | Jonathan Lange has had 164 bugs fixed on Launchpad Bazaar Integration | 18:15 |
| jml | To me, the most interesting thing about this script (apart from the cool data) is "how did I figure out to call 'searchTasks'?" | 18:15 |
| jml | Well, each of the Launchpad instances has generated API documentation at +apidoc | 18:15 |
| jml | e.g. https://staging.launchpad.net/+apidoc/ | 18:15 |
| jml | or e.g. https://edge.launchpad.net/+apidoc/ | 18:15 |
| jml | I happen to know a bit about the Launchpad object model, and knew I'd need to get a bunch of "bug tasks" | 18:15 |
| jml | A bug task is that row in the table that has an assignee, status, importance etc. A bug can have many bug tasks. | 18:16 |
| jml | I went to the API documentation page and had a look around for something that would return a list of bug tasks. | 18:16 |
| jml | I found "searchTasks", and then gave it a try and it worked | 18:16 |
| jml | The API documentation also told me that people have display_name attributes and that projects do too. | 18:16 |
| jml | Note that the API documentation is *not* launchpadlib documentation. It's generic REST API documentation. | 18:16 |
| jml | That means you often need to translate from the abstract REST docs into concrete Python. It's not that hard. | 18:16 |
| jml | Anyway the script is pretty simple | 18:16 |
| jml | <geser> QUESTION: does python-launchpadlib also clean up the cache of old unused files? | 18:16 |
| jml | geser, No, it does not. | 18:16 |
| jml | geser, as far as I'm aware, there's absolutely no functionality in launchpadlib for removing cache files under any circumstances | 18:17 |
| jml | The next revision is something a little bit more complex | 18:17 |
| jml | bzr revert -r4 | 18:17 |
| jml | This script does almost exactly the same thing, except it tells you how successful the reporter was as a percentage of bugs fixed over bugs filed | 18:17 |
| jml | The new code is the total_bugtasks line, which finds all bugs of any status | 18:18 |
| jml | fixed_bugtasks = pillar.searchTasks( | 18:18 |
| jml | bug_reporter=reporter, status=['Fix Released']) | 18:18 |
| jml | total_bugtasks = pillar.searchTasks( | 18:18 |
| jml | bug_reporter=reporter, | 18:18 |
| jml | status=[ | 18:18 |
| jml | "New", | 18:18 |
| jml | "Incomplete", | 18:18 |
| jml | "Invalid", | 18:18 |
| jml | "Won't Fix", | 18:18 |
| jml | "Confirmed", | 18:18 |
| jml | "Triaged", | 18:18 |
| jml | "In Progress", | 18:18 |
| jml | "Fix Committed", | 18:18 |
| jml | 'Fix Released']) | 18:18 |
| jml | zoiks! | 18:18 |
| jml | (I thought I had my newlines done differently) | 18:18 |
| jml | I had to specify each status manually, because 'searchTasks' defaults to only finding open bugs | 18:19 |
| jml | Then there's some code to calculate a percentage and print it out nicely | 18:19 |
| jml | percentage = 100.0 * length(fixed_bugtasks) / length(total_bugtasks) | 18:19 |
| jml | print "%s is %.2f%% successful on bugs in %s" % ( | 18:19 |
| jml | reporter.display_name, percentage, pillar.display_name) | 18:19 |
| jml | Let's try running it! | 18:19 |
| jml | $ python bugstats.py ubuntu jml | 18:19 |
| jml | Jonathan Lange is 22.22% successful on bugs in Ubuntu | 18:19 |
| jml | $ python bugstats.py launchpad-code jml | 18:19 |
| jml | Jonathan Lange is 56.16% successful on bugs in Launchpad Bazaar Integration | 18:19 |
| jml | Again, you can try it with yourself, or with a friend, or with cjwatson | 18:20 |
| jml | That's all I have for the example. | 18:20 |
| jml | = 3. Gotchas = | 18:20 |
| jml | There are quite a few things that can trip you up with the Launchpad API | 18:20 |
| jml | We've already bumped into a couple of them: it's got bugs. | 18:20 |
| jml | https://bugs.edge.launchpad.net/launchpadlib/ for more info on that | 18:20 |
| jml | The second is that even though it does have documentation, it's not always easy to apply to what you need. | 18:21 |
| jml | (wtf, "Colin Watson is 79.65% successful on bugs in Ubuntu"!) | 18:21 |
| jml | I'm going to pimp http://help.launchpad.net/API again, since there really is a fair chunk of good documentation there. | 18:21 |
| jml | https://help.launchpad.net/API/Examples is good, as is https://help.launchpad.net/API/launchpadlib | 18:21 |
| jml | However, the docs often aren't up to scratch. | 18:21 |
| jml | There is only one way for them to get better | 18:22 |
| jml | People like your good selves must ask questions, get answers, and then edit the help.launchpad.net wiki. | 18:22 |
| jml | Another gotcha is error messages. | 18:22 |
| jml | Because launchpadlib is a very, very thin wrapper over a generic RESTful client library, if ever you get an exception raised, it's going to be really unhelpful. | 18:22 |
| jml | https://help.launchpad.net/API/Examples#Get%20a%20useful%20error%20message%20from%20launchpadlib shows the work-around. | 18:22 |
| jml | I think I filed a bug about that. | 18:22 |
| jml | hmm. | 18:23 |
| jml | there's another gotcha, which is *performance* | 18:23 |
| jml | For a lot of the things you probably want to do, you'll end up writing code that looks like this: | 18:23 |
| jml | for thing in bunch_of_things: | 18:23 |
| jml | thing.do_something_on_launchpad() | 18:23 |
| jml | Code like this is really really slow | 18:23 |
| jml | It'll do an HTTPS request for each 'thing' | 18:23 |
| jml | We'd like to have some way of reducing that load, but we don't have any good answers yet | 18:23 |
| jml | Only two more gotchas left :) | 18:23 |
| jml | Not everything that's in Launchpad itself is exposed via the API. | 18:23 |
| jml | Sorry. | 18:23 |
| jml | It's generally either really, really easy to expose stuff over the API or almost impossible. | 18:23 |
| jml | If the thing you want falls into the first category, you can probably patch Launchpad yourself. | 18:24 |
| jml | In any case, *file a bug*. Everything that's not exposed over the API is a bug. | 18:24 |
| jml | Last gotcha: testing launchpadlib apps is hard | 18:24 |
| jml | jkakar has done some work on this recently | 18:24 |
| jml | but I haven't had a chance to look at it. | 18:24 |
| jml | That's it on the gotchas: bugs, docs, errors, potato programming, unexposed methods, testing | 18:24 |
| jml | = 4. Future = | 18:25 |
| jml | There are plenty of other examples of launchpadlib apps out there | 18:25 |
| jml | bughugger uses launchpadlib (https://launchpad.net/bughugger) | 18:25 |
| jml | quickly has some templates for using launchpadlib (https://launchpad.net/quickly) | 18:25 |
| jml | The code that puts branches in https://code.launchpad.net/ubuntu is done entirely with launchpadlib (we talked about those branches last session!) | 18:25 |
| jml | There's also a stack of projects that are part of a group called 'lpx' that you can find at https://launchpad.net/lpx | 18:25 |
| jml | and at http://help.launchpad.net/API/Uses there's even more stuff | 18:25 |
| jml | If ever you need help, ask on #launchpad or #launchpad-dev on freenode or send an email to launchpad-users@lists.launchpad.net | 18:25 |
| jml | That's it from me. | 18:26 |
| jml | Questions? | 18:26 |
| jml | Cool ideas for launchpadlib programs? | 18:26 |
| jml | General complaints about Launchpad that I can respond soothingly to? | 18:26 |
| jml | By "complaint", I of course meant "suggestion for improvement" | 18:27 |
| jml | <vocx> QUESTION: so, Python plays a HUGE part in Ubuntu, no? Other scripting won't feel left out? | 18:27 |
| jml | A lot of Ubuntu stuff is written in Python. | 18:27 |
| jml | As I mentioned earlier, you don't *have* to use Python to control Launchpad, but it makes it a lot easier. | 18:28 |
| jml | <jönöbacon> QUESTION: Can you make more examples for python-snippets happen? | 18:28 |
| jml | jonobacon, I can point you at https://help.launchpad.net/API/Examples again, I guess :) | 18:29 |
| jml | <kamalmostafa> QUESTION: Is there a way to access two JSON values in one "call"... I.e. launchpad.people[reporter_name].latitude gets the latitude -- is there a way to also get the longitude at the same time? | 18:29 |
| jml | kamalmostafa, I'm fairly sure that if you get the object first, then subsequent attribute access doesn't do a separate webapp call. | 18:30 |
| jml | so... | 18:30 |
| jml | reporter = launchpad.people[reporter_name] | 18:30 |
| jml | reporter.latitude, reporter.longtitude | 18:30 |
| jml | <vocx> QUESTION: is there a need to worry about python 3.0 breaking something? | 18:30 |
| jml | vocx, wrt launchpadlib or wrt Ubuntu more generally? | 18:30 |
| jml | vocx, launchpadlib is a Python 2 application. It hasn't been tested with Python 3. | 18:31 |
| jml | vocx, I'm not the launchpadlib maintainer, but I tend to think of python3 as a completely different language. if a program works on both 2 and 3 it's a happy accident | 18:32 |
| jml | vocx, no plans to port to 3 yet. | 18:33 |
| jml | vocx, I guess we'd start doing that when Ubuntu stops providing Python 2 | 18:33 |
| jml | are there any other questions? | 18:34 |
| jml | QUESTION: Can we convince jonobacon to use metal umlauts in his name so my friends can go back to calling me Jono? | 18:34 |
| === jml is now known as joneaux | ||
| === joneaux is now known as jml | ||
| jml | Last chance for questions folks | 18:37 |
| jml | remember you can get the code for the example by 'bzr branch lp:~jml/+junk/bugstats' | 18:37 |
| jml | and that the folk on #launchpad-dev are helpful and friendly and want more people doing stuff with launchpadlib | 18:37 |
| jml | ok folks | 18:38 |
| jml | don't mind the dead time on Radio Ubuntu Dev Week, Kubuntu Junior Jobs will be up in 20 minutes | 18:39 |
| jml | have a great weekend | 18:39 |
| maco | Hiya | 19:03 |
| maco | Ready for the Papercuts session? | 19:04 |
| maco | I'm going to guess that anyone wanting to sit on in this session is here by now, so... | 19:05 |
| maco | Hi, I'm Mackenzie, and I'm a MOTU. I was asked this morning to cover this session for Celeste since she has school, so I'm going off of her notes | 19:06 |
| maco | This session is on KDE Junior Jobs and *buntu Papercuts (or paperkuts <g>) | 19:06 |
| maco | So first off, if you don't know yet, Kubuntu is a version of Ubuntu that uses KDE for its desktop environment instead of GNOME | 19:07 |
| maco | Most development happens upstream at KDE (kde.org) and then the Kubuntu developers package up each upstream release | 19:08 |
| maco | We try to stick closely to upstream, as giving back to the KDE is a good thing, and having to maintain a bunch of patches is a bad thing | 19:09 |
| maco | (or at least a not-fun thing) | 19:09 |
| maco | One thing we do because users seem to like it and it helps KDE get extra testing is package up pre-releases for KDE | 19:10 |
| maco | For example, KDE SC 4.4 RC2 is currently in one of the (many) Kubuntu PPAs | 19:10 |
| maco | (more info can always be found at kubuntu.org) | 19:10 |
| maco | A number of the Kubuntu developers also write patches which then get submitted upstream to KDE | 19:11 |
| maco | and then those new features end up in whatever the next final release is for KDE | 19:11 |
| maco | If you're interested in getting started contributing to Kubuntu and KDE, then bug fixes, small patches, papercuts, small improvements on existing features, and other junior jobs are all a GREAT way to get started | 19:12 |
| maco | For Karmic, the Hundred Papercuts project was started in Ubuntu, and Kubuntu had a share of them | 19:14 |
| maco | (see: https://launchpad.net/hundredpapercuts ) | 19:14 |
| maco | The goal is to fix 100 papercuts per release, or in the case of Kubuntu (as it's a smaller team (can I add, "and KDE's already awesomer than GNOME"?)), 10 papercuts | 19:15 |
| maco | A papercut is defined as "a trivially fixable usability bug that the average user would encounter in default installation of Ubuntu or Kubuntu Desktop Edition" | 19:16 |
| maco | Ok well, that's the surface goal | 19:17 |
| maco | The REAL goal is to fix all those little annoyances that pile up until the desktop becomes unusable or at least a pain in the rear | 19:18 |
| maco | For example, a task may require that you click 10 times among 3 different windows, and well, if you only have to do that once every few months, that's not a big deal | 19:19 |
| maco | but if you have to do that 50 times per day... | 19:19 |
| maco | It gets old fast ;) | 19:19 |
| maco | Anyway, Kubuntu tries to deal with 10 paperkuts per release, and we try to get the patches for these paperkuts accepted in upstream KDE as well | 19:19 |
| maco | Because, again, maintaining patches is much less fun than writing them | 19:20 |
| maco | Some of the past Kubuntu paperkuts were cases where text was a little too technical for an end-user and had to be changed, or in the Kickoff menu the user's icon was showing next to the search box instead of the user's name | 19:21 |
| maco | Here are a list of Kubuntu's papercuts from Karmic & Lucid: https://bugs.launchpad.net/hundredpapercuts/+bugs?field.searchtext=kubuntu&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field | 19:22 |
| maco | .status%3Alist=FIXRELEASED&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_supervisor=&field.bug_commenter=&field.subscriber=&field.omit_dupes.used=&field.omit_dupes=on&field.has_patch.used=&field.has_cve.used=&field.affects_me.used=&field.tag=&field.tags_combinator=ANY | 19:22 |
| maco | (and apparently not all of the kubuntu ones were tagged...) | 19:23 |
| maco | I suspect Celeste had links already arranged for seeing past and present Kubuntu papercuts, but I don't have them, so... | 19:25 |
| maco | here's some more that say kde but dont say kubuntu: https://bugs.launchpad.net/hundredpapercuts/+bugs?field.searchtext=kde&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field | 19:26 |
| maco | .status%3Alist=FIXRELEASED&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_supervisor=&field.bug_commenter=&field.subscriber=&field.omit_dupes.used=&field.omit_dupes=on&field.has_patch.used=&field.has_cve.used=&field.affects_me.used=&field.tag=&field.tags_combinator=ANY | 19:26 |
| maco | Anyway, moving on... | 19:26 |
| maco | If you think you've found a papercut, submit it as a bug to Launchpad | 19:26 |
| maco | Mark it as bug both against the "hundredpapercuts" project and the specific package in Kubuntu if you know it | 19:27 |
| maco | If you would like to fix one of those papercuts you see up there but you need some help ^ join #kubuntu-devel and | 19:28 |
| maco | er... and we'll help you out | 19:28 |
| maco | OK, so that's papercuts. I mentioned something called "Junior Jobs" at the start too | 19:29 |
| maco | Maybe you've seen bugs tagged "bitesize" in Launchpad, or you might know about Daniel Holbach's Harvest project to show low-hanging fruit for new developers | 19:30 |
| maco | Junior Jobs are KDE's version of that same idea | 19:30 |
| maco | And I'm being told in -chat that I should point out that "job" is like "volunteer job" in case you're thinking you can get money for it. Because you won't. | 19:30 |
| maco | QUESTION: link for the harvest project? (i am google challenged at the moment) | 19:32 |
| maco | Harvest can be found at http://daniel.holba.ch/harvest | 19:32 |
| maco | There are plans to make it prettier | 19:32 |
| maco | By the way, my computer is currently telling me that opening the "sourcepackage list" page on Harvest while having ~100 other tabs open in Firefox is an idea to which it is opposed. You've been warned. | 19:34 |
| maco | OK, so...Junior Jobs. The KDE techbase (one of the KDE wikis) page for Junior Jobs is here: http://techbase.kde.org/Contribute/Junior_Jobs | 19:35 |
| maco | As you can see, KDE separates its Junior Jobs by sub-project, so you can see just KMail or just Kopete or whatever's available jobs | 19:36 |
| maco | Or you can see all of the KDE Junior Jobs here: https://bugs.kde.org/buglist.cgi?keywords=junior-jobs&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&cmdtype=doit | 19:37 |
| maco | Earlier this question came up and I held it til reaching the Junior Jobs part... | 19:38 |
| maco | QUESTION: some bugs marked as Junior Jobs don't sound easy, like implementing a new KIO slave, how come they are marked as Junior Job? | 19:38 |
| maco | Now, I don't have a lot of involvement with upstream KDE like Celeste does, so she'd have a better answer than I do, but I wouldn't be surprised if there was a bit of mentorship in there somewhere | 19:39 |
| maco | I also don't think Junior Jobs are supposed to be aimed at "I've never written code before" but rather "I can code...kind of...but I've never worked on such a big project before" | 19:40 |
| maco | And a lot of the Papercuts are quite a bit more advanced than "I've never written code before" too, so... keep that in mind | 19:40 |
| maco | Nightrose: do you have a better answer to "some bugs marked as Junior Jobs don't sound easy, like implementing a new KIO slave, how come they are marked as Junior Job?" | 19:42 |
| maco | Oh yeah, let me introduce Nightrose, aka Lydia | 19:43 |
| maco | She's the Amarok lead, I believe | 19:43 |
| Nightrose | jep :) | 19:44 |
| Nightrose | so people said some of the junior jobs are hard | 19:44 |
| Nightrose | there are two reasons for this: | 19:44 |
| Nightrose | 1) the person who marked it as a junior job didn't know the program very well and didn't know how hard it actually is | 19:45 |
| Nightrose | 2) the person who marked it is having a hard time judging how hard a particular task is | 19:45 |
| Nightrose | neither is really nice but it happens | 19:45 |
| Nightrose | we try to keep them at junior level as wel as we can of course :) | 19:45 |
| maco | Nightrose: can they be un-marked as JJs? | 19:46 |
| Nightrose | yes of course - tell me or other people in #kde-bugs | 19:46 |
| maco | Great! Thank you :) | 19:46 |
| Nightrose | you're welcome :) | 19:46 |
| maco | Feel free to thwap me if I say anything wrong/stupid between now and 15 minutes :) | 19:47 |
| maco | <Quintasan> maco, Nightrose: is anyone in particular responsible for reporting PaperKuts to KDE? I mean something like Bug #510219, marked as tiriaged but not linked to any of KDE bugs, it's safe for me to report that upstream and link it to LP? | 19:48 |
| maco | Go right ahead :) | 19:48 |
| maco | QUESTION: If I want to patch KDE progressbar animation smoothness, where would I start? The oxygen theme? (I've patched it locally before :) ) | 19:49 |
| maco | Neither Nightrose nor I know know to answer this one, so I guess now's the time to note that on this IRC server there is #kde-devel, and they would likely have an answer | 19:49 |
| maco | And Mamarok is pointing out that Nightrose is release manager, not lead. oops! | 19:51 |
| maco | Anyway, time to start wrapping up | 19:51 |
| maco | So the benefits to participating in Junior Jobs or Papercuts are that you get a chance to test out your skills in a *real* project, not those silly things the teachers make you write at school | 19:52 |
| maco | You get a chance to gain some rapport with the rest of the project so that as your skillset grows, they know and trust you | 19:52 |
| maco | You end up witha more usable desktop, which is always great | 19:52 |
| maco | programming, why hasn't it caught with enough strength? Do women like C++? | 19:53 |
| maco | bah | 19:53 |
| maco | QUESTION: if Qt (C++) is supposedly hundreds of times better than GTK+ (C) for GUI programming, why hasn't it caught with enough strength? Do women like C++? | 19:53 |
| maco | Honestly, the reason I know how to do C/GTK+ programming and not C++/Qt is that my school taught C, not C++ | 19:54 |
| maco | and I haven't gotten around to learning C++ | 19:54 |
| maco | In either case, PyGTK and PyQt (and PyKDE) exist, so even so, learning C/C++ is not a requirement to participation | 19:55 |
| maco | And I'm just not going to answer that women...C++ part. People use what they like, regardless of gender. | 19:55 |
| Nightrose | there are also bindings for scripting languages | 19:55 |
| Nightrose | so you can do cool stuff in ruby or java script | 19:55 |
| maco | This does bring up the point that on top of learning just Python or C++, there are libraries you need to learn to work in the KDE world, so Junior Jobs / Papercuts give you a chance to build on top of your base programming knowledge if you just know Python or C++ | 19:56 |
| maco | Any other quick questions in the 3 minutes we have left? | 19:57 |
| maco | QUESTION: Does everybody know that KDE+Kubuntu will rule the world? | 19:58 |
| maco | Yes, but whether they're willing to admit it is another thing entirely | 19:58 |
| maco | <Mamarok> also, Qt/C++ allows to do shorter code, the Qt framework is very powerful <-- good point. you'll find you can do a lot of painful things very easily as you get into KDE programming | 19:59 |
| maco | OK, time's up here. Hope everyone's all excited to get cracking on these bugs | 20:00 |
| maco | Bye! | 20:00 |
| Nightrose | thanks everyone for coming and maco for rocking :) | 20:00 |
| maco | Thanks for your help, Lydia | 20:01 |
| persia | OK. That means it's time for Interpreting Stacktraces :) | 20:02 |
| persia | I get distracted with -classroom and -classroom-chat, and really prefer interactive discussions, so I encourage anyone to ask questions in this channel at any time (I prefer to be interrupted) | 20:03 |
| persia | As an example, we're going to look at a crash in dot. Anyone wanting to follow closely would benefit from getting a local copy of the cairo and graphviz sources (because we'll be looking at them later) | 20:05 |
| persia | Either karmic or lucid sources should be OK. | 20:05 |
| persia | A stacktrace is the overview of the current function call stack for a program, and is extremely useful when determining why a crash happens. | 20:07 |
| persia | Thanks pleia2 :) | 20:07 |
| pleia2 | welcome :) | 20:07 |
| persia | When a program executes, the various function calls and instructions in the program are run, typically in some sort of order, which we can see from the source code. | 20:08 |
| persia | In most programs, there is a main() function, which controls the entire program. In executing it, otehr functions will be called to do various things. | 20:08 |
| persia | These functions will then call other functions, and so on, in a nested fashion. | 20:09 |
| persia | Each function called is placed on the top of the stack when it is called, and when it, in turn, calls another function, that function is placed on the top of the stack. | 20:10 |
| persia | As a result, at any point, the function on the top of the stack is the currently executing code, and the rest of the stack contains all the nested functions needed to unwind to get back to the operating system starting the program. | 20:11 |
| persia | An example stacktrace (and the one we'll use for debugging today) is http://launchpadlibrarian.net/37459596/Stacktrace.txt | 20:12 |
| persia | The stack is commonly described in terms of frames. Frame #0 is the currently executing function, Frame #1 the function that called that function, etc. In our example, Frame #8 is main() which was called by the operating system when starting the program (sometimes you can see this, depending on the stacktrace) | 20:13 |
| persia | At each Frame, the stacktrace contains the address of the function, the name of the function, the values of (most of) the arguments to the function, and often some avtive variable values in the vicinity. | 20:14 |
| Pendulum | < vishalrao_vbox> QUESTION: who/how generates a textual stacktrace from, say, a core dump? | 20:14 |
| persia | Well, there's lots of ways to do that :) The two most common ways to generate stacktraces as used for crashes in Ubuntu are with gdb and apport. | 20:15 |
| Pendulum | < michae28> QUESTION : what is vicinity ? | 20:15 |
| persia | "vicinity" is a state of being near in space or relationship. Things in the same vicinity are close to each other. | 20:16 |
| persia | Could also be nearby, surrounding, etc. | 20:16 |
| Pendulum | < vishalrao_vbox> QUESTION: can you briefly mention the commandlines for gdb and apport to generate the stacktraces? | 20:16 |
| persia | The process of generating a stacktrace with gdb is described at https://wiki.ubuntu.com/Backtrace | 20:17 |
| persia | For apport, there are three ways to do it, all dependent on the apport-retracer infrastructure. | 20:17 |
| persia | The first being to enable apport crash tracking, which automatically uploads the crash information to launchpad and files a bug when a crash happens. | 20:18 |
| persia | The second being to use apport-bug to file a bug with a previously produced .crash report | 20:18 |
| persia | The third being to use apport-retrace manually to collect information. | 20:19 |
| persia | If the details in the crash report are uploaded to launchpad, the apport-retracers will typically use them to generate textual stacktraces with debugging information. | 20:19 |
| persia | So, by using the stacktrace, we can see exactly what the program is doing at the time the stacktrace is taken. | 20:20 |
| persia | In our example, the program started, did some stuff which resulted in a call to gvRenderJobs(), which did something and called emit_graph(), which did something and called emit_background(), and so forth. | 20:21 |
| Pendulum | < vishalrao_vbox> QUESTION: if i try to generate a backtrace in gdb and it just shows numbers instead of symbols, how do i get it to show me the function names etc? | 20:21 |
| persia | To determine the parts that aren't described, we'll need to review the source code along with the stacktrace. | 20:22 |
| persia | For most programs, the debugging symbols have been stripped out into ddebs. These contain mappings between the function call addresses and the code. | 20:23 |
| persia | I can never find the right wiki page that explains how to install them, but it should be navigable from the links in the /topic of #ubuntu-bugs (or maybe someone else can dig it up and share) | 20:23 |
| persia | The example stacktrace I've shown comes from Bug #503503 | 20:24 |
| ubottu | Bug 503503 on http://launchpad.net/bugs/503503 is private | 20:24 |
| * persia fixes that | 20:24 | |
| persia | Right. Now bug #503503 should be public | 20:24 |
| ubottu | Launchpad bug 503503 in graphviz "dot crashed with SIGSEGV in cairo_set_dash()" [Medium,New] https://launchpad.net/bugs/503503 | 20:24 |
| persia | This bug was filed using apport, which I find much easier to search. If you're looking for other crash bugs, using the apport-crash tag is great way to get a list from which to work. | 20:25 |
| persia | The title of the bug tells us that there was a segmentation fault when calling the function cairo_set_dash(). These tend to be my favorite sort of crash bugs, because they are usually fairly easy to track down. | 20:27 |
| persia | http://en.wikipedia.org/wiki/SIGSEGV has a little more information about segmentation faults. | 20:27 |
| persia | Other than the title, the rest of the bug report is mostly uninteresting at this point. Depending on what we find, we may want to investigate using some of the other provided information (for instance, if we're crashing in translations, the fact that it's reported with de_LU.UTF8 is important) | 20:28 |
| persia | Well, there's one other interesting thing, actually, which is the package against which the bug is filed. This tells us which package we need to get source from in order to interpret the stacktrace. | 20:29 |
| persia | Also, reviewing the stacktrace, one might notice that the source file identified at frame #0 is /build/buildd/cairot-1.8.8/src/cairo.c rather than some local file. This notation is used to identify code from other packages. In this case, it's the cairo package (version 1.8.8), so that's why we wanted to download graphviz and cairo sources earlier (and I'll now assume that you've all done that) | 20:31 |
| === vishalrao_web is now known as vishalrao | ||
| persia | The first step when reviewing a stacktrace is to find somewhere interesting to start. One could always just look at frame #0 where the crash is actually happening, but this is rarely enough context to actually understand the bug. | 20:32 |
| persia | Alternately, one could start from the very bottom of the stack, but usually the first several calls are far too general, and it's easy to get frustrated or distracted reading that much source. | 20:33 |
| persia | It's never possible to know the right point in advance, but I usually pick something near the middle, preferably something that has a lot of vairable references defined. | 20:33 |
| persia | So in this case, frame #4 looks like a sensible place to start. From here, we'll read up to frame #0 to understand things. If the problem has already happened at frame #4, we may end up backing up (perhaps even to frame #0) to fix it, but at least we'll understand the targets to search when we look at the more general code. | 20:35 |
| persia | So, frame #4 is the gvrender_box call at line 851 of gvrender.c . Since there are no funny pathnames, this is in the graphviz source. | 20:36 |
| persia | I usually use `find . -name ${SOURCE} -print` from within the source directory to quickly identify the right file. | 20:36 |
| persia | Running `find . -name gvrender.c -print` shows ./lib/gvc/gvrender.c | 20:37 |
| persia | Please open that file in your favorite text editor | 20:37 |
| persia | At line 851 is a gvrender_polygon call, which is giving us trouble. Unless we've previously worked with this source, we probably don't know what this means, although the name may help us guess. | 20:38 |
| persia | So, it's best to scroll up to the beginning of the function that is calling gvrender_polygon() and read down again to line 851 to get some understanding. | 20:38 |
| persia | The top of the function is at line 840, and the entire function consists of setting the values of variable A (type pointf) based on values from variable B (type boxf). | 20:39 |
| persia | So in our call, we are just passing through the job pointer and the filled boolean, along with A and a number. | 20:40 |
| persia | Next, we look at frame #3, which is at line 812 of gvrender.c | 20:41 |
| persia | Again, we scroll up to the beginning of the function (at line 805), and read down. We know the inputs from the last function we read, so we can understand this a bit better. | 20:42 |
| persia | The function assigns an engine from the job, apparently succeeds (because of the if), apparently confirms the job has the GVRENDER_DOES_TRANSFORM flag set, and calls gvre_polygon using all the same values that were passed in previously. | 20:44 |
| persia | Moving to frame #2, we need to switch source files, to gvrender_pango.c | 20:44 |
| persia | THis is in ./plugin/pango\gvrender_pango.c | 20:44 |
| persia | Line 277 is in the cairogen_polygon function, which isn't quite the name that was called, but seems to take the same inputs we had previously. | 20:45 |
| persia | Between the start of the function and the call in the stacktrace, we see only that *obj, *cr, and i are defined (and *obj and *cr are initialised). | 20:46 |
| persia | We then call cairogran_set_penstyle with job and cr. Looking at the stacktrace, we can see that job is optimised out (but had been0x90945d0 in previous calls, so is probably the same) | 20:47 |
| persia | And cr is 0x0. | 20:47 |
| persia | since we know that cr is a pointer to a variable of type cairo_t, it shouldn't have a value of 0 (this is a null pointer). | 20:48 |
| persia | Which probably means that cairo_t *cr = (cairo_t *) job->context; failed. | 20:48 |
| persia | Moving to frame #1 is just moving to line 235 | 20:49 |
| persia | Here the function sets *obj again, and based on the value of *obj, selects a way to call cairo_set_dash(). In our case, it's the neither a dashed nor a dotted pen. | 20:50 |
| persia | But we pass dashed anyway :) | 20:50 |
| persia | For frame #0, we need to look at the cairo source code. This is in src/cairo.c (it's easier to tell with foreign source files) | 20:51 |
| persia | Looking at line 1033, and up to the start of the funtion, we can see that we're right at the beginning. | 20:51 |
| persia | Before doing anything else real, the code checks the status of cr. Since cr is a 0, the expected structure isn't present, so cr->status isn't valid, which causes the segfault. | 20:52 |
| persia | This is another thing to fix,because it's best for a library not to crash, even when the wrong data is passed. | 20:52 |
| Pendulum | < vocx> QUESTION: are stack traces really important nowdays? I feel like much development is done now in programming languages such as Python and C#/Mono, which use their own runtime, and so don't "crash" in the traditional way, and don't produce a stacktrace. | 20:53 |
| persia | So, after reading the stacktrace, we've identified two things to fix, either one of which would solve the bug. | 20:53 |
| persia | 1) test to make sure one has a valid value for cr in cairogen_polygon(), and 2) put an exception handler around the status check in cairo_set_dash() | 20:54 |
| Pendulum | < vocx> QUESTION: in your experience, where are the crashes most often found? The program itself, or the supporting libraries, like Gtk+, Qt, glib, cairo, poppler, etc.? | 20:54 |
| persia | Python produces something called a Traceback, which is essentially the same as a stacktrace, except presented in the opposite order (frame #0 is at the bottom). | 20:55 |
| Pendulum | < michae28> QUESTION : <value optimized out> does it mean that the real value cannot be displayed ? | 20:55 |
| persia | But the vast majority of system libraries end up being written in compiled languages, so if there is a crash in a library, a stacktrace is likely interesting (even if the call comes from python or mono bindings) | 20:55 |
| persia | I've yet to find a crash that couldn't be worked around by type checking, value checking, return code checking, or exception handling in client code. | 20:56 |
| persia | But many crashes *also* expose an issue with the underlying library (as we saw in this case, where both graphviz and cairo could use code improvements). | 20:57 |
| persia | When the value is opimised out, it usually means that the value isn't relevant in some way, or is redundant. In our example case, the value for job was passed a couple times, and then dropped. We could still look at older uses, but it no longer mattered. | 20:58 |
| Pendulum | < SevenMachines> [QUESTION] Often you see argument memory addresses cropping up in stacktraces, is there a good way to get more information in the stacktrace on the actual object on the heap that was passed? | 20:58 |
| persia | Not really. I tend to try to assemble a mental model of the object by passing through several frames. | 20:58 |
| persia | If the intial guess for starting was wrong, one sometimes needs to go back further to understand the data being passed. | 20:59 |
| persia | But most of the time, crashes seem to be from a miscommunication between the library author and the application author, where one is expecting something different than what the other is providing. | 20:59 |
| persia | And these tend to be obvious (like the one we examined, where the code never checked to see if a valid value was being passed to cairo, and when cairo couldn't complete the request, didn't trap the return) | 21:00 |
| persia | Whether SIGSEGV is the best thing for cairo to return is a different question, but that it wasn't checked doesn't help. | 21:01 |
| persia | cairo should probably have trapped the SIGSEGV, and returned someting with _cairo_set_error, and graphviz should have checked the value of _cairo_set_error (as well as having checked to make sure cr was a valid pointer before using it) | 21:02 |
| persia | Are there more questions? I seem to be out of time, but there's no following sessions, so I'm happy to keep answering questions for a bit. | 21:03 |
| Pendulum | < michae28> QUESTION : to have all this stack informations, the program have been compiled with -g option ? | 21:04 |
| persia | I actually don't know. I've gotten stacktraces just using `gcc foo.c -o foo`, but this may have been from some default options that were set. | 21:05 |
| persia | Well, thanks everyone for attending. Feel free to catch me in #ubuntu-bugs if you're digging through any traces and have questions in the future. | 21:07 |
| persia | And thanks a lot for participating in Ubuntu Developer Week. I hope some of you will be presenting next time :) | 21:08 |
| persia | Pendulum: Thanks especially for forwarding questions. | 21:08 |
| Pendulum | persia: you're welcome :) Thank you for presenting :) | 21:09 |
| === luis__lopez is now known as luis_lopez | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
