| === dave_m is now known as dmart | ||
| === neverendingo__ is now known as neverendingo | ||
| === mthalmei is now known as MichiSoft | ||
| parkie | hey | 13:45 |
|---|---|---|
| parkie | quit | 13:51 |
| bittin` | hello | 14:11 |
| l403 | hello | 14:12 |
| MaNU_ | What are the preparations required for today's session : Fixing small bugs in Ubuntu | 14:24 |
| === openweek7 is now known as dlightle | ||
| === qwebirc39867 is now known as iEternity | ||
| qwebirc47066 | date-u | 15:53 |
| shiki- | qwebirc47066, that goes into your terminal.:) | 15:59 |
| mhall119|work | and you need a space: date -u | 16:01 |
| shiki- | anyway.. I missed yesterday's "classes".. so.. yeah.. I wonder about today.. :) | 16:01 |
| dlightle | shiki-: the IRC logs are posted if you want to read through them, not sure if you knew that | 16:02 |
| shiki- | ah yeah I'll read them on the weekend.. for now, I only have time to keep up with the today's lessons/classes | 16:03 |
| shiki- | ty for the hint anyway | 16:04 |
| === Amaranth_ is now known as Amaranth | ||
| === marvin_ is now known as Guest4654 | ||
| dholbach | Ubuntu Developer Week starting in 18 minutes | 16:42 |
| c_korn | hooray | 16:43 |
| devin122 | 10 min .. waiting patiently | 16:51 |
| Kamusin | nice | 16:52 |
| ideamonk | awesome :) | 16:52 |
| arvind_khadri | how do i remove the join and quit parts? | 16:54 |
| arvind_khadri | am using xchat | 16:54 |
| frandieguez | and I pidgin, I have the same problem | 16:55 |
| syedam | right click on the channel | 16:55 |
| syedam | and in extra alerts set hide join | 16:55 |
| arvind_khadri | syedam, got it :) thanks | 16:55 |
| syedam | sorry in settings | 16:55 |
| czambran | __ | 16:59 |
| dholbach | WELCOME TO ANOTHER GREAT DAY OF UBUNTU DEVELOPER WEEK! | 17:00 |
| doctormo | Thanks dholbach | 17:00 |
| bittin` | thx :) | 17:00 |
| dholbach | Who do we have here for "Fixing small bugs in Ubuntu"? :-) | 17:00 |
| dutchie | o/ | 17:00 |
| shiki- | :) | 17:00 |
| arvind_khadri | me | 17:00 |
| norax | me | 17:00 |
| jef_ | me :) | 17:00 |
| frandieguez__ | me! | 17:00 |
| sum-it | i am in | 17:00 |
| syedam | o/ | 17:01 |
| Jonnie_Simpson | me :D | 17:01 |
| czambran | Hi | 17:01 |
| James147 | :) | 17:01 |
| Gnome64 | ayay! | 17:01 |
| bptk421 | hi! | 17:01 |
| devin122 | me | 17:01 |
| c_korn | me | 17:01 |
| wers | :) | 17:01 |
| mozuku | me | 17:01 |
| ScottTesterman_ | :) | 17:01 |
| krams | me | 17:01 |
| ideamonk | :) | 17:01 |
| HobbleAlong | hi all | 17:01 |
| slord54 | *lurks* | 17:01 |
| raji | me 2 | 17:01 |
| AntoineLeclair | me | 17:01 |
| Geep_ | me | 17:01 |
| francesco_m | o/ | 17:01 |
| openweek1_ | howdy | 17:01 |
| ubuntufreak | me | 17:01 |
| sum-it | dholbach: QUESTION: do we need karmic for that? | 17:01 |
| abourgeois | me | 17:01 |
| fromme | hi | 17:01 |
| dholbach | sum-it: I'll answer that in a sec | 17:01 |
| ulysses | +m ? | 17:01 |
| awalton | 'lo | 17:01 |
| Copernicus1234 | +1 | 17:02 |
| Andre_Gondim | me | 17:02 |
| RainCT | :) | 17:02 |
| * freelancer317 waving | 17:02 | |
| dholbach | OK dear friends of Ubuntu Development, let's get the session started! | 17:02 |
| dholbach | for those of you who are new to Ubuntu Developer Week - this channel is muted, so please head over to #ubuntu-classroom-chat and ask questions there | 17:02 |
| dholbach | please make sure you prefix them with QUESTION: | 17:03 |
| dholbach | If have problems following in English or want to ask a question in your local language, fine people in these channels might help you out: | 17:03 |
| dholbach | * Catalan: #ubuntu-classroom-chat-ca | 17:03 |
| dholbach | * Danish: #ubuntu-nordic-dev | 17:03 |
| dholbach | * Finnish: #ubuntu-fi-devel | 17:03 |
| dholbach | * German: #ubuntu-classroom-chat-de | 17:03 |
| dholbach | * Spanish: #ubuntu-classroom-chat-es | 17:03 |
| dholbach | * French: #u-classroom | 17:03 |
| dholbach | To answer sum-it's question: no you don't necessarily need karmic for this session, but as I said yesterday, please have a look into https://wiki.ubuntu.com/UbuntuDevelopment/UsingDevelopmentReleases | 17:04 |
| dholbach | if you're interested in developing Ubuntu it's important that you run the devel release in some form | 17:04 |
| dholbach | some sane form like a virtual machine, etc. | 17:04 |
| dholbach | alright my friends: quick preparations for those of you who weren't here yesterday | 17:04 |
| dholbach | sudo apt-get install --no-install-recommends bzr ubuntu-dev-tools devscripts dpkg-dev cdbs | 17:05 |
| dholbach | also please enable "Sources" in System -> Administration -> Software Sources -> Ubuntu Software | 17:05 |
| dholbach | also please add something like this to your ~/.bashrc file: | 17:05 |
| dholbach | export DEBFULLNAME='Daniel Holbach' | 17:05 |
| dholbach | export DEBEMAIL='daniel.holbach@ubuntu.com' | 17:05 |
| dholbach | save it and run | 17:05 |
| dholbach | source ~/.bashrc | 17:06 |
| dholbach | please don't use MY NAME | 17:06 |
| dholbach | thanks :-) | 17:06 |
| dholbach | we covered this in yesterday's session so please either read the logs or ask somebody in #ubuntu-classroom-chat to help you | 17:06 |
| dholbach | ok... we want to fix simple bugs in Ubuntu | 17:06 |
| dholbach | I took the liberty of choosing a few that I think we should get done :-) | 17:07 |
| dholbach | please all take a look at https://bugs.launchpad.net/ubuntu/+source/edubuntu-addon-meta/+bug/404608 | 17:07 |
| dholbach | it talks about a small typo in a package description | 17:07 |
| dholbach | so to get the source code, please run: | 17:08 |
| dholbach | apt-get source edubuntu-addon-meta | 17:08 |
| dholbach | now please | 17:08 |
| dholbach | cd edubuntu-addon-meta-0.12 | 17:08 |
| dholbach | as explained yesterday you can find things like the package description in debian/control, so please open that file in your favourite editor | 17:08 |
| dholbach | ah and there the bug is... "form" in the last line | 17:09 |
| dholbach | please change it to "from" | 17:09 |
| dholbach | save the file | 17:09 |
| dholbach | ok... just a bit of background about debian/control: | 17:09 |
| dholbach | the first stanza is always about the source package (refer to yesterday's log or to https://wiki.ubuntu.com/PackagingGuide) | 17:10 |
| dholbach | the stanzas afterwards (just one in this case) describe the binary packages (the resulting .deb files) | 17:10 |
| dholbach | alright, as far as we can see this should fix the bug | 17:10 |
| dholbach | one thing I mentioned yesterday too is documentation | 17:10 |
| dholbach | we need to document the change we just did in debian/changelog | 17:11 |
| dholbach | luckily there's a nice tool in the devscripts package called dch that makes the job of editing the changelog a lot easier | 17:11 |
| dholbach | so if you set up ~/.bashrc with DEBEMAIL and DEBFULLNAME and ran source ~/.bashrc please now run: | 17:11 |
| dholbach | dch -i | 17:11 |
| dholbach | this will create a new changelog entry and increment the version number | 17:12 |
| dholbach | if you look at the entry closely, you'll see that it always starts with the source package name | 17:12 |
| dholbach | then there's the version number which I'll get back to in a sec | 17:12 |
| dholbach | next is the release in which we want to fix it in | 17:12 |
| dholbach | the urgency is a debian-ism which we can ignore | 17:13 |
| dholbach | next we have the entry which we still have to write, then our name, email and timestamp | 17:13 |
| dholbach | ok, I talked about version numbers a bit yesterday | 17:14 |
| dholbach | normally we'd add "ubuntu1" to indicate that we took a Debian package and modified it | 17:14 |
| dholbach | in this case it's an Ubuntu only package | 17:14 |
| dholbach | so instead of 12ubuntu1, we'll use 13 | 17:14 |
| dholbach | now I'll put something like this as the actual changelog entry | 17:15 |
| dholbach | * debian/control: replaced "form" with "from". (LP: #404608) | 17:15 |
| dholbach | it's important to use a form like this (or similar)... what did I do? | 17:15 |
| dholbach | - I described which file I touched | 17:16 |
| dholbach | - I described what I did | 17:16 |
| dholbach | - I mentioned the bug number in a special format | 17:16 |
| dholbach | it's important that you provide as much information as possible | 17:16 |
| dholbach | the bug report usually has all that information and enables others to revisit the bug and better understand why you did your changes | 17:16 |
| dholbach | also did I use (LP: #404608) because only in the (LP: #xxxxxxxxxx) format the bug will be automatically closed on upload | 17:17 |
| dholbach | alright... now save the file | 17:17 |
| dholbach | now please run debuild -S | 17:17 |
| dholbach | this will rebuild the source package (the .tar.gz and .dsc file) | 17:18 |
| dholbach | it might ask you for your gpg key, but it's not necessary to sign it | 17:18 |
| dholbach | now if you | 17:18 |
| dholbach | cd .. | 17:18 |
| dholbach | you should see edubuntu-addon-meta_0.12.dsc edubuntu-addon-meta_0.12.tar.gz edubuntu-addon-meta_0.13.dsc edubuntu-addon-meta_0.13.tar.gz | 17:18 |
| dholbach | (and a few other files) | 17:18 |
| dholbach | which means we successfully rebuilt the source package with our changes | 17:19 |
| dholbach | <arvind_khadri> QUESTION : why shouldnt we use debian/rules here to build the .deb again? | 17:19 |
| dholbach | arvind_khadri: for this exercise we don't need to build the .deb and you might want to take a look at pbuilder for building packages (https://wiki.ubuntu.com/PbuilderHowto) | 17:19 |
| dholbach | generally you're right though... if you want to fix a bug, you definitely need to test it too | 17:20 |
| dholbach | now it'd be a bit much for this session | 17:20 |
| dholbach | <bananeweizen> QUESTION: how do we know it's an Ubunty only package | 17:20 |
| dholbach | bananeweizen: edubuntu should be a hint in this case, generally it's a bit harder to tell | 17:20 |
| dholbach | https://packages.debian.org/src:<packagename> should find the debian package if available | 17:21 |
| dholbach | <noiz777> QUESTION: i did the debuild -S but it failed: clearsign failed: secret key not available, what do i do there? | 17:21 |
| dholbach | noiz777: normally you'd sign a source package so you can upload it to a PPA for example | 17:21 |
| dholbach | for this exercise it's not necessary | 17:21 |
| dholbach | alright | 17:21 |
| dholbach | now please run | 17:21 |
| dholbach | debdiff edubuntu-addon-meta_0.12.dsc edubuntu-addon-meta_0.13.dsc | 17:22 |
| dholbach | this will show you the diff between the old and the new version | 17:22 |
| dholbach | if you run | 17:22 |
| dholbach | debdiff edubuntu-addon-meta_0.12.dsc edubuntu-addon-meta_0.13.dsc > edubuntu-addon-meta.fix | 17:22 |
| dholbach | it will give you a nice file to attach to the bug report | 17:22 |
| dholbach | (remember: test it first :-)) | 17:22 |
| dholbach | I'll talk a bit about sponsoring / patch review later on | 17:22 |
| dholbach | so you know how to get your good fixes reviewed and included | 17:23 |
| dholbach | first bug fixed :-) | 17:23 |
| dholbach | ok... up next is https://bugs.launchpad.net/ubuntu/+source/qutim/+bug/346528 | 17:23 |
| dholbach | please run: | 17:23 |
| dholbach | apt-get source qutim | 17:23 |
| dholbach | when I prepared the session the bug was still under app-install-data-ubuntu - it took me a bit to realise that the app-install-data is retrieved from lots of .desktop files from various packages | 17:24 |
| dholbach | so if we fix it in qutim now, it should get fixed in the app-install-data too (for "Add/Remove...") | 17:25 |
| dholbach | <EagleScreen> QUESTION: I obtained this: gpg: Signature made Thu Mar 19 18:05:37 2009 CET using DSA key ID 92742B33 | 17:25 |
| dholbach | gpg: Can't check signature: public key not found | 17:25 |
| dholbach | what does it mean? | 17:25 |
| dholbach | EagleScreen: it means that the package was originally signed by somebody whose public key you don't have in your keyring - that's safe to ignore | 17:25 |
| dholbach | alright | 17:25 |
| dholbach | cd qutim-0.1 | 17:26 |
| dholbach | grep -ri massanger * | 17:26 |
| dholbach | this will search for "massanger" in all files, ignoring if it's upper case or lower case | 17:26 |
| dholbach | seems like we have two files to fix | 17:26 |
| dholbach | debian/qutim.desktop:GenericName=Instant Massanger | 17:26 |
| dholbach | debian/qutim.1:qutIM \- Qt Instant Massanger | 17:26 |
| dholbach | one of them is the .desktop file (that creates the menu entry) which is mentioned in the bug report | 17:27 |
| dholbach | and also there's the manpage | 17:27 |
| dholbach | now please run: | 17:28 |
| dholbach | sed -i 's/Massanger/Messenger/g' debian/qutim.desktop | 17:28 |
| dholbach | sed -i 's/Massanger/Messenger/g' debian/qutim.1 | 17:28 |
| dholbach | this will replace Massanger with Messenger in both files | 17:28 |
| dholbach | ok | 17:29 |
| dholbach | now let's document our changes | 17:29 |
| dholbach | please run | 17:29 |
| dholbach | dch -i | 17:29 |
| dholbach | 0.1-0ubuntu2 should be fine | 17:30 |
| dholbach | and as a description of what we did, I chose | 17:30 |
| dholbach | * debian/qutim.desktop, debian/qutim.1: replaced "Massanger" with | 17:30 |
| dholbach | "Messenger" (LP: #346528) | 17:30 |
| dholbach | (it's good practise to wrap lines at 80 characters) | 17:30 |
| dholbach | bug number two fixed too :-) | 17:31 |
| dholbach | <roxan> QUESTION: isn't typo fixed a better discription in this case ? | 17:31 |
| dholbach | roxan: sure, you can do that too - I usually prefer to explicitly mention what was wrong and how I fixed it :) | 17:31 |
| dholbach | alright... let's talk a bit about sponsoring and patch review | 17:32 |
| dholbach | once you have a nice patch, you obviously attach it to the bug report so people can check it out | 17:33 |
| dholbach | but for it to be included in Ubuntu (if you can't upload source packages yourself yet), you need to subscribe the reviewers team | 17:33 |
| dholbach | https://wiki.ubuntu.com/SponsorshipProcess explains the process in detail | 17:33 |
| dholbach | essentially you subscribe ubuntu-main-sponsors for packages in main or restricted | 17:34 |
| dholbach | and subscribe ubuntu-universe-sponsors for packages in universe or multiverse | 17:34 |
| dholbach | they'll give you feedback and upload the patch once it's ok | 17:34 |
| dholbach | <EagleScreen> QUESTION: I have a build problem: http://pastebin.ca/1550481 | 17:35 |
| dholbach | <rugby471> EagleScreen: try installing qmake & qmake-dev | 17:35 |
| dholbach | ^ for those of you wondering why debuild -S doesn't work in the case of qutim | 17:35 |
| dholbach | thanks rugby471 | 17:35 |
| dholbach | ok... let's move on to our third bug :) | 17:36 |
| dholbach | some questions first | 17:36 |
| dholbach | <rugby471> QUESTION: sometimes debdiffs take quite a while to be sponsored, what are the ways that I can make sure it get's uploaded as fast as it can? | 17:36 |
| dholbach | rugby471: make sure the patch is tip top tested and documented very very well | 17:36 |
| dholbach | so reviewers don't have to go into a very long feedback loop | 17:36 |
| dholbach | usually you can ask in #ubuntu-devel or #ubuntu-motu too to get some help | 17:36 |
| dholbach | <trothigar> QUESTION: can you find out using apt which repository a package comes from? | 17:37 |
| dholbach | apt-cache showsrc qutim | grep ^Dir | 17:37 |
| dholbach | <AntoineLeclair> QUESTION: When I want to submit a fix, what do I submit? The debdiff file only? | 17:37 |
| dholbach | AntoineLeclair: yes in cases where you just submit a simple fix, yes | 17:38 |
| dholbach | again https://wiki.ubuntu.com/SponsorshipProcess has more info on the topic | 17:38 |
| dholbach | ok, our third bug is: | 17:38 |
| dholbach | https://bugs.launchpad.net/ubuntu/+source/quickly/+bug/422212 | 17:38 |
| dholbach | for those of you who attended yesterday's session, you will know what it is about | 17:38 |
| dholbach | also you'll see that Markus Korn filed the bug after the session because he was interested in it :-) | 17:39 |
| dholbach | in most cases you can safely use the "apt-get source" command to obtain the source | 17:40 |
| dholbach | as I know that quickly is maintained in Launchpad's Code hosting (be sure to visit the session later this week!), we'll get the latest trunk of quickly and see if it needs fixing there | 17:40 |
| dholbach | please run | 17:41 |
| dholbach | bzr branch lp:quickly | 17:41 |
| dholbach | this will get the latest trunk of quickly | 17:41 |
| dholbach | this might take a little bit as it gets the whole history of quickly and its development | 17:41 |
| dholbach | once it's done, please | 17:41 |
| dholbach | cd quickly | 17:41 |
| dholbach | grep -ri arborting * | 17:41 |
| dholbach | which... again ... will search in all directories and check for upper and lower case alike | 17:42 |
| dholbach | I get the following output: | 17:42 |
| dholbach | bin/quickly: print _("Arborting.") | 17:42 |
| dholbach | bin/quickly: print _("Arborting.") | 17:42 |
| dholbach | quickly/tools.py: print _("Arborting.") | 17:42 |
| dholbach | so three occurences we need to fix | 17:42 |
| dholbach | these commands should get you there: | 17:43 |
| dholbach | sed -i 's/Arborting/Aborting/g' bin/quickly | 17:43 |
| dholbach | sed -i 's/Arborting/Aborting/g' quickly/tools.py | 17:43 |
| dholbach | I don't have enough time to go into the use of the sed command, but "-i" means "replace in place" (so modify the file) | 17:43 |
| dholbach | and the last "/g" means "fix all occurrences | 17:44 |
| dholbach | <Johan1> QUESTION : What is a trunk exactly ? | 17:44 |
| dholbach | Johan1: trunk is the development focus... usually projects use various branches in which small features or bugs are developed indepently of each other | 17:44 |
| dholbach | once they're deemed ready, they're merged into 'trunk' | 17:45 |
| dholbach | ... which gets released every now and then when it seems to make sense | 17:45 |
| dholbach | software projects often work a bit differently because of how they are set up | 17:45 |
| dholbach | in this case we want trunk | 17:45 |
| dholbach | alright | 17:46 |
| dholbach | once that's done, you can run | 17:46 |
| dholbach | bzr diff | 17:46 |
| dholbach | to have another look at your changes again | 17:46 |
| dholbach | instead of documenting in debian/changelog, I'll show you something else this time | 17:46 |
| dholbach | please run bzr commit --fixes lp:422212 -m "changed 'Arborting.' to 'Aborting.'" | 17:47 |
| dholbach | this will commit the change (locally) and add the log message "changed 'Arborting.' to 'Aborting.'" to the revision history | 17:48 |
| dholbach | you will be able to see it this way: bzr log | head | 17:48 |
| dholbach | the other great thing is that when you push the change to launchpad, it will automatically link your branch to the bug report | 17:48 |
| dholbach | http://bazaar-vcs.org/Documentation has more information on the topic | 17:49 |
| dholbach | let's try to see if we manage to fix another bug: | 17:49 |
| dholbach | https://bugs.launchpad.net/ubuntu/+source/qemulator/+bug/348172 | 17:49 |
| dholbach | another nice typo | 17:50 |
| dholbach | this time in the qemulator package | 17:50 |
| dholbach | apt-get source qemulator | 17:50 |
| dholbach | <EagleScreen> QUESTION: bzr log | head shows wrongly my e-mail, any thing similar to DEBEMAIL var? | 17:50 |
| dholbach | EagleScreen: you're right... try bzr whoami | 17:50 |
| dholbach | the qemulator case is a bit different | 17:51 |
| dholbach | cd qemulator-0.5/; grep -ri snpashot * | 17:51 |
| dholbach | will show you a few occurences | 17:52 |
| dholbach | some in binary files (which we can't fix) - they are translation files | 17:52 |
| dholbach | but before we go ahead and change Qemulator.pot and usr/local/lib/qemulator/qemulator.glade I need to explain something | 17:53 |
| dholbach | in our qutim and edubuntu-addon-something case we edited files in the debian/ directory | 17:53 |
| dholbach | these are files supplied by us (or other Ubuntu/Debian maintainers) | 17:53 |
| dholbach | in the case of quickly we fixed it in trunk (so where the upstream developers work) | 17:54 |
| dholbach | yesterday I talked a bit about how we make use of a .orig.tar.gz (that contains the source code that we get from the upstream software authors and which we don't change) | 17:54 |
| dholbach | we merely add the "packaging" in the debian/ directory | 17:54 |
| dholbach | so some maintainers decide to store patches that fix something in the source code of upstreams in debian/patches/ | 17:55 |
| dholbach | if you | 17:55 |
| dholbach | ls debian/patches | 17:55 |
| dholbach | you'll find there's already a patch | 17:55 |
| dholbach | https://wiki.ubuntu.com/PackagingGuide/PatchSystems goes into more detail about all the "patch systems" that maintainers use | 17:56 |
| dholbach | we have 4 minutes left, so let's make it quick | 17:56 |
| dholbach | run | 17:56 |
| dholbach | what-patch | 17:56 |
| dholbach | (from the ubuntu-dev-tools package) | 17:56 |
| dholbach | and it'll tell you that the patch system is CDBS | 17:56 |
| dholbach | so please run | 17:56 |
| dholbach | cdbs-edit-patch fix-snapshot-typo | 17:56 |
| dholbach | this will fire up a "sub shell" in which we fix the issue now | 17:57 |
| dholbach | now please run | 17:57 |
| dholbach | sed -i 's/snpashot/snapshot/g' Qemulator.pot usr/local/lib/qemulator/qemulator.glade | 17:57 |
| dholbach | this should fix the issue in both places | 17:57 |
| dholbach | now please hit Ctrl-D (or type 'exit') | 17:57 |
| dholbach | if you now look at ls debian/patches/ | 17:58 |
| dholbach | you'll see our patch file | 17:58 |
| dholbach | now we'd just do dch -i again to document the change we just did | 17:58 |
| dholbach | we'd say something like | 17:58 |
| dholbach | * debian/patches/fix-snapshot-typo.patch: replace 'snpashot' with 'snapshot'. (LP: #348172) | 17:59 |
| dholbach | done :-) | 17:59 |
| dholbach | <plumstead21> QUESTION: What's the best way of finding these 'easy' bugs that newbies stand a chance of being able to fix? | 17:59 |
| dholbach | plumstead21: great one! | 17:59 |
| dholbach | a few people wrote harvest which is supposed to find those low-hanging fruit | 17:59 |
| dholbach | its current home is http://daniel.holba.ch/harvest | 17:59 |
| dholbach | it's not very pretty, so some of us are working on https://wiki.ubuntu.com/Harvest/NewUI | 17:59 |
| dholbach | if you're a web developer PLEASE help out! | 17:59 |
| dholbach | dholbach at ubuntu dot com | 18:00 |
| dholbach | thanks a lot everybody! | 18:00 |
| syedam | thanks dholbach :) | 18:00 |
| dholbach | and please check out #ubuntu-motu for more help | 18:00 |
| ogasawara | thanks dholbach! | 18:00 |
| c_korn | thanks. great talk | 18:00 |
| dholbach | https://wiki.ubuntu.com/MOTU/GettingStarted too | 18:00 |
| jef_ | thanks :) | 18:00 |
| dinxter | thanks! | 18:00 |
| bittin` | t | 18:00 |
| lopo1 | thanks! | 18:00 |
| bittin` | thx dholbach | 18:00 |
| roxan | Thank You | 18:01 |
| ^arky^ | thanks dholbach , another great session | 18:01 |
| antisa | well done | 18:01 |
| Gnome64 | Next session : Kernel Triaging and Debugging | 18:01 |
| dholbach | next up is Leann Ogasawara who will help us dive into Kernel Triaging and Debugging! | 18:01 |
| frandieguez | great class!!! | 18:01 |
| ogasawara | Hi Everyone! Welcome :) | 18:01 |
| * iulian waves | 18:01 | |
| syedam | hii | 18:01 |
| bittin` | Hello | 18:01 |
| ogasawara | My name is Leann Ogasawara and I help manage the Ubuntu Kernel Team's incoming and existing bugs against the kernel. | 18:01 |
| frandieguez | hi ogasawara | 18:01 |
| ogasawara | Having to deal with such a large volume of bugs is always a huge challenge for us. | 18:02 |
| dholbach | note: chat and questions please in #ubuntu-classroom-chat | 18:02 |
| ogasawara | I thought this session would be a good opportunity to share some best practices I've learned along the way for triaging kernel bugs as well as share some information for helping debug issues. | 18:02 |
| ogasawara | We're always looking for more involvement from the community to help triage kernel bugs so hopefully after today's session some of you might be interested in helping out. | 18:02 |
| ogasawara | Let's first start with what the role of a kernel bug triager is. | 18:02 |
| ogasawara | The goal for any kernel bug triager is to get a bug into a state such that a developer can immediately being working on a fix. | 18:03 |
| ogasawara | Remember, as a triager we are often the first point of contact for a bug reporter. | 18:03 |
| ogasawara | It's important that we help move a bug into a good working state as well as help educate the bug reporter to submit better bug reports in the future. | 18:03 |
| ogasawara | So how does that happen? | 18:03 |
| ogasawara | First, we help make sure Ubuntu kernel bugs are assigned to the Ubuntu linux kernel package. | 18:03 |
| ogasawara | http://bugs.launchpad.net/ubuntu/+source/linux | 18:03 |
| ogasawara | For example, if someone is experiencing a kernel oops or panic, then that's obviously a kernel bug. | 18:04 |
| ogasawara | If a bug reporter did not correctly file the bug against the linux kernel package, help reassign the bug and kindly remind them to report future kernel bugs against the linux kernel package. | 18:04 |
| ogasawara | Failing to do so may result in the bug getting overlooked. | 18:04 |
| ogasawara | It may be helpful to also point them at https://wiki.ubuntu.com/Bugs/FindRightPackage . | 18:04 |
| ogasawara | Next, we want to make sure a bug is really not a duplicate of another bug. | 18:04 |
| ogasawara | This is where we as kernel triagers need to be careful. | 18:05 |
| ogasawara | Kernel bugs are usually hardware specific. | 18:05 |
| ogasawara | Just because someone may be experiencing the same symptom of another bug reporter doesn't necessarily mean they have the same bug. | 18:05 |
| ogasawara | When in doubt, don't mark it as a duplicate and ask for a second opinion. | 18:05 |
| ogasawara | Additionally, if you see someone comment on a bug and they don't have the same hardware, ask them to open a new bug report and explain why. | 18:05 |
| ogasawara | This really helps avoid bugs becoming wildly out of control and impossible for a developer to follow, let alone fix. | 18:06 |
| ogasawara | Next, help make sure the title of the bug as well as the bug description is informative. | 18:06 |
| ogasawara | "Sounds is broken" or "Suspend fails" is not informative. | 18:06 |
| ogasawara | Like I mentioned above, kernel bugs are usually hardware specific. It's always best to mention the affected hardware in the title as well as the bug description. | 18:06 |
| ogasawara | This will again help avoid bugs becoming a mess. | 18:07 |
| ogasawara | If you find you have the same hardware as a bug being reported, try to reproduce the bug yourself. | 18:07 |
| ogasawara | t's not unheard of for hardware to become faulty. | 18:07 |
| ogasawara | Being able to help confirm this is or is not the result of hardware going bad is important. | 18:07 |
| ogasawara | Also, be sure to document the steps to reproduce if possible. | 18:07 |
| ogasawara | Now I know the next part is sometimes a bit controversial, but it's also best if the issue has been confirmed against the latest kernel available. | 18:08 |
| ogasawara | I realize this is a touchy subject for some individuals and some reporters often object to always being asked to "test the latest". | 18:08 |
| ogasawara | However, when you are dealing with the kernel, keep in mind there are literally thousands of commits between each release. | 18:08 |
| ogasawara | Then consider that each commit touches more than just one line of code and you've now hit insanity trying to isolate one fix (if it even exists) for a single bug. | 18:08 |
| ogasawara | Finally, one of the most important aspects of traiging kernel bugs is making sure the appropriate log information is attached. | 18:09 |
| ogasawara | For the kernel this means dmesg output, lspci, kernel version info, etc. | 18:09 |
| ogasawara | I recommend that instead of asking bug reporters to attach these files individually, have them run apport-collect. | 18:09 |
| ogasawara | apport-collect will automatically gather and attach general linux kernel debug information for a bug. | 18:09 |
| ogasawara | For example, if we wanted kernel debug info attached to pretend bug 987654, the apport-collect command would look like: | 18:10 |
| ogasawara | apport-collect -p linux 987654 | 18:10 |
| ogasawara | The nice part about this is we're only asking the bug reporter to run a single command. | 18:10 |
| ogasawara | There's less room for error having a bug reporter run one command versus having a bug reporter run multiple commands. | 18:10 |
| ogasawara | Even better than reporting a bug and running apport-collect on it, is to use ubuntu-bug to report the bug in the first place. | 18:11 |
| ogasawara | This will automatically file the bug against the linux kernel package and again automatically gather and attach kernel debug info to the new bug. | 18:11 |
| ogasawara | The command looks like the following: | 18:11 |
| ogasawara | ubuntu-bug -p linux | 18:11 |
| ogasawara | In the process of attempting to triage a bug, if you've asked a bug reporter to provide more information, be sure to set the bug's status to Incomplete. | 18:11 |
| ogasawara | Also be sure to subscribe yourself to a bug so that you are automatically notified when they have responded with the requested information. | 18:12 |
| ogasawara | Once the bug looks ready for a developer to begin working on it, set the status of the bug to Triaged and make sure the Importance is set. | 18:12 |
| ogasawara | Note that being able to set a bug to Triaged and also to set the Importance requires that you be a member of the Ubuntu Bug Control team in Launchpad. | 18:12 |
| ogasawara | To learn how to join the ubuntu-bugcontrol team, refer to https://launchpad.net/~ubuntu-bugcontrol | 18:13 |
| ogasawara | I'd also like to bring up one last thing to keep in mind when triaging kernel bugs . . . and that's forwarding the bug upstream. | 18:13 |
| ogasawara | Before a bug can be forwarded upstream, it needs to be confirmed to exist when running the upstream kernel. | 18:13 |
| ogasawara | The Ubuntu kernel team has started building vanilla mainline kernel builds for users to test with. | 18:13 |
| ogasawara | See https://wiki.ubuntu.com/KernelTeam/MainlineBuilds | 18:13 |
| ogasawara | If a bug exists with the upstream kernel, the bug should be forwarded upstream so that the upstream kernel developers are also aware of the issue. | 18:14 |
| ogasawara | Additionally, it may be discovered that the bug is fixed upstream and we should pull in the fix back into the Ubuntu kernel. | 18:14 |
| ogasawara | If a bug has already been reported to the upstream kernel bugzilla, http://bugzilla.kernel.org/ , we should make sure we set up an upstream bug watch from the Launchpad bug report to the upstream bug report. | 18:14 |
| ogasawara | See https://wiki.ubuntu.com/Bugs/Watches for more information on how to set an upstream bug watch. | 18:14 |
| ogasawara | That should pretty much cover the basics of kernel bug triaging. | 18:15 |
| ogasawara | As always, feel free to take a look at https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies for more information. | 18:15 |
| ogasawara | Now that we've touched on the basics, I'd like to take this opportunity to quickly mention Kernel Bug Days. | 18:15 |
| ogasawara | For the past couple months the Ubuntu Kernel Team has been organizing kernel bug days every 2 weeks. | 18:15 |
| ogasawara | https://wiki.ubuntu.com/KernelTeam/BugDay | 18:15 |
| ogasawara | The goal of each bug day is to triage and fix kernel bugs. | 18:16 |
| ogasawara | Typically each bug day also has a general focus, for example suspend/resume bugs. | 18:16 |
| ogasawara | The entire Ubuntu Kernel Team dedicates their day to focus on the bug day but we would also appreciate any community involvement! | 18:16 |
| ogasawara | Each kernel bug day always has a specific Community section which contains a list of suggested bugs for community members to work on. | 18:16 |
| ogasawara | And, it just so happens that we're holding a kernel bug day today! | 18:17 |
| ogasawara | https://wiki.ubuntu.com/KernelTeam/BugDay/20090901 | 18:17 |
| ogasawara | So, if anyone would be interested in helping out, this is a great starting point to begin triaging kernel bugs. | 18:17 |
| ogasawara | I'd be more than willing to help anyone get started, just ping me in #ubuntu-kernel on FreeNode after this session. | 18:17 |
| ogasawara | Ok, I think now is a good time to stop and take some questions before we move on. | 18:17 |
| ogasawara | nareshov> question: i have a wireless card that requires firmware i think (broadcom), how can i test mainline kernels? | 18:18 |
| ogasawara | nareshov: unfortunately that is one corner case where you won't be able to use the mainline kernels | 18:18 |
| ogasawara | nareshov: however, if you know which firmware is required, open a bug and let us know | 18:18 |
| ogasawara | ok moving on | 18:19 |
| ogasawara | For the debugging part of this session, I'd like to focus on some common types of issues I run into on a regular basis and how I go about debugging them. | 18:19 |
| ogasawara | The first type of issue I see reported rather frequently are update/install issues. | 18:19 |
| ogasawara | The majority of these bugs are usually reported via apport and will come in with general debug information already attached. | 18:20 |
| ogasawara | The important debug file to look at will be the DpkgTerminalLog file. | 18:20 |
| ogasawara | Usually, somewhere near the end of the DpkgTerminalLog file will hopefully be some error messages about what failed during the update/install. | 18:20 |
| ogasawara | For example, lets take a look at https://bugs.edge.launchpad.net/ubuntu/+source/linux/+bug/398036 | 18:21 |
| ogasawara | falstaff|h> question: i've bug in karmic kernel, which is also in latest mainline kernel. Should I report the bug to the ubuntu kernel bug tracker or upstream? Or both? | 18:21 |
| ogasawara | falstaff|h: both would be great | 18:21 |
| ogasawara | falstaff|h: we can then link the launchpad bug report to the upstream one | 18:21 |
| ogasawara | ScottTesterman_> QUESTION: Today I noticed a bug report where the initial reporter had marked the bug "Confirmed," but there were no other reports to verify the bug. What's the best way of dealing with this if I cannot confirm the bug report? | 18:21 |
| ogasawara | ScottTesterman_: it's best to kindly remind bug reporters to no "Confirm" their own bugs | 18:22 |
| ogasawara | ScottTesterman_: unfortunately if you are unable to confirm yourself, the next best step is to make sure they have all the appropriate debug information attached | 18:22 |
| ogasawara | ok, back to bug 398036 | 18:23 |
| ogasawara | After examining the DpkgTerminalLog.txt file I noticed the following errors: | 18:23 |
| ogasawara | Merging changes into the new version | 18:23 |
| ogasawara | Conflicts found! Please edit `/var/run/grub/menu.lst' and sort them out manually. | 18:23 |
| ogasawara | The file `/var/run/grub/menu.lst.ucf-new' has a record of the failed merge of the configuration file. | 18:23 |
| ogasawara | User postinst hook script [/sbin/update-grub] exited with value 3 | 18:23 |
| ogasawara | As you can see, this is really an issue with grub which caused the kernel to fail to install. | 18:24 |
| ogasawara | Having seen this error message before I knew this was a duplicate of bug 269539. | 18:24 |
| ogasawara | Notice that prior to marking the bug as a duplicate I pasted the error message into a comment in the bug report. | 18:24 |
| ogasawara | Recall that I mentioned earlier that part of our job as a triager is to educate the bug reporter. | 18:25 |
| ogasawara | Pasting this information into the bug informs the bug reporter which debug file I looked at to find the error, what the error was, and why I marked it as a duplicate. | 18:25 |
| ogasawara | Now I know some may be thinking, "How am I supposed to know which bug this would be a duplicate of?!?!". | 18:26 |
| ogasawara | Well, having triaged many of these types of bugs myself, I took the liberty to write up a DebuggingUpdateErrors wiki for the kernel: | 18:26 |
| ogasawara | https://wiki.ubuntu.com/KernelTeam/DebuggingUpdateErrors | 18:26 |
| ogasawara | That wiki documents all the common kernel update/install bugs I've come across, what the error message looks like, and what the master bug is. | 18:26 |
| ogasawara | Sometimes you won't even want to mark the bug as a duplicate as it could be Invalid. | 18:26 |
| ogasawara | For example, if someone ran out of disk space while trying to upgrade, that's not a kernel bug. | 18:27 |
| ogasawara | This type of bug is usually indicated by a "gzip: stdout: No space left on device" line in the DpkgTemrinalLog file. | 18:27 |
| ogasawara | The kernel has no control over how much free disk space someone has :) So this is just an invalid bug and they need to free up space. | 18:27 |
| ogasawara | The next question some may have is "Why can't the errors be detected automatically when the bug is reported?". | 18:27 |
| ogasawara | The answer is it can! | 18:28 |
| ogasawara | For the majority of the bugs I've documented in that DebuggingUpdateErrors wiki, we've also written an ubuntu-bugpattern. | 18:28 |
| ogasawara | https://code.launchpad.net/~ubuntu-bugcontrol/apport/ubuntu-bugpatterns | 18:28 |
| ogasawara | If a bug is filed with apport and we've written a bugpattern to match it, the bug reporter will be notified that they do not need to file a new bug and they will be directed to the master bug instead. | 18:28 |
| ogasawara | If there is no master bug, as in the example of someone running out of disk space, they are directed to the DebuggingUpdateErrors wiki explaining the issue. | 18:29 |
| ogasawara | The ubuntu-bugpatterns are really helpful but some bugs still slip through. For example, if the error message were in Spanish instead of English, the bugpattern won't catch it. | 18:29 |
| ogasawara | Also, if someone didn't use apport to file the update/install bug, I ask them to take a look at https://wiki.ubuntu.com/DebuggingUpdateManager | 18:29 |
| ogasawara | The "Debugging Procedures" section outlines the debug files to gather and attach to the report. | 18:29 |
| ogasawara | Ok moving on to the next type of issue to debug . . . | 18:30 |
| ogasawara | The next common scenario I come across triaging kernel bugs is kernel regressions :( | 18:31 |
| ogasawara | First when any regression is reported, it's important that it gets tagged as a regression. | 18:31 |
| ogasawara | At the bottom or each bug report's description there should be a "Tags" line and a yellow pencil edit icon to add, remove, or update a bug's tag(s). | 18:31 |
| ogasawara | There are usually 4 different regression tags that kernel bugs will use: | 18:31 |
| ogasawara | 1) regression-potential - A bug discovered in the development release that was not present in the stable release. | 18:32 |
| ogasawara | For example, right now Karmic is known as the development release and Jaunty is the previous stable release. | 18:32 |
| ogasawara | If someone finds a regression while testing Karmic while we are still in the development phase, this would be tagged "regression-potential". | 18:32 |
| ogasawara | 2) regression-release - A regression in a new stable release. | 18:32 |
| ogasawara | For example, when Karmic officially releases and a regression is found, this would be tagged "regression-release". | 18:33 |
| ogasawara | regression-potential bugs could very well become regression-release bugs. | 18:33 |
| ogasawara | 3) regression-update - A regression introduced by an updated package in the stable release. | 18:33 |
| ogasawara | For example, if Jaunty released a new kernel update and if a regression were discovered, this would be tagged "regression-update" | 18:33 |
| ogasawara | 4) regression-proposed - A regression introduced by a package in -proposed | 18:33 |
| ogasawara | Prior to any updates being released, packages sit in what's called -proposed. See https://wiki.ubuntu.com/Testing/EnableProposed . If a regression is found in -proposed, this would be tagged "regression-proposed" | 18:34 |
| ogasawara | https://wiki.ubuntu.com/QATeam/RegressionTracking documents these tags in more detail. | 18:34 |
| ogasawara | Also see the regression tracker for the existing list of known regressions: | 18:34 |
| ogasawara | http://qa.ubuntu.com/reports/regression/regression_tracker.html | 18:34 |
| ogasawara | The regression tracker will automatically update with any bugs which get tagged as a regression. | 18:35 |
| ogasawara | The kernel team reviews regressions on a weekly basis so making sure they are tagged appropriately will make sure they get on the team's radar. | 18:35 |
| ogasawara | The next important part when dealing with regressions is to try to narrow down when the regression was introduced. | 18:35 |
| ogasawara | Recall that I mentioned earlier that there are thousands of commits between each kernel release. | 18:35 |
| ogasawara | If we just look at Jaunty which released with a 2.6.28 based kernel and Karmic which will release with a 2.6.31 based kernel, there's likely going to be over 37,000 commits. | 18:36 |
| ogasawara | Narrowing down those 37,000 commits is going to be key in helping the kernel team quickly find where the bad commit is. | 18:36 |
| ogasawara | What I like to do to help narrow down the regression (ie find the bad commit) is to do what we call a rough bisect. | 18:36 |
| ogasawara | A rough bisect basically involves selecting a working and non-working kernel and then systematically selecting a kernel in between the two to test. | 18:37 |
| ogasawara | Depending if the kernel in between works or not, we adjust the known working and non-working kernel and repeat the process until we can narrow down the working and non-working kernel as closely as possible. | 18:37 |
| ogasawara | This is where I like to use the mainline kernel builds since they're already built and ready to test. | 18:37 |
| ogasawara | So lets use a scenario that someone says they can suspend/resume just fine using Jaunty. | 18:38 |
| ogasawara | They just recently updated to Karmic and are running a 2.6.31-8.28 kernel and their system is hanging on resume. | 18:38 |
| ogasawara | First, I like to confirm that the regression is not an Ubuntu specific regression. | 18:38 |
| ogasawara | To do this I ask the bug reporter to test the mainline kernel which the current Ubuntu kernel was based on. | 18:38 |
| ogasawara | I don't expect anyone know know off the top of their head which mainline kernel an Ubuntu kernel was based on. | 18:39 |
| ogasawara | This is why I'd encourage you to use http://kernel.ubuntu.com/~kernel-ppa/info/kernel-version-map.html | 18:39 |
| ogasawara | This maps every Ubuntu kernel version to the mainline kernel version they were based on. | 18:39 |
| ogasawara | Examining the kernel version map, we see that 2.6.31-8.28 was based on mainline 2.6.31-rc7 | 18:39 |
| ogasawara | As a result I'd point the bug reporter to the 2.6.31-rc7 mainline kernel build and ask them to test. | 18:40 |
| ogasawara | http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.31-rc7 | 18:40 |
| ogasawara | If they confirm the bug remains, it's a good assumption this was a regression introduced by an upstream change. | 18:40 |
| ogasawara | Assuming this is the case, I want to get confirmation that this is still working with the upstream kernel Jaunty was based on. | 18:40 |
| ogasawara | Again looking at the kernel version map I see that Jaunty released with a 2.6.28-11.42 Ubuntu kernel which was based on the 2.6.28.9 upstream kernel. | 18:41 |
| ogasawara | So I of course ask them to test http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.28.9/ | 18:41 |
| ogasawara | Then we will have established that 2.6.28.9 is our working kernel and 2.6.31-rc7 is our non-working kernel. | 18:41 |
| ogasawara | Now we have to pick a kernel in between, so lets pick 2.6.30 | 18:42 |
| ogasawara | http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.30/ | 18:42 |
| ogasawara | We ask the reporter to test this and just for example sake, lets say the bug reporter tests 2.6.30 and notes that it is suspending/resuming just fine. | 18:42 |
| ogasawara | So now lets pick a kernel between 2.6.30 and 2.6.31-rc7, say 2.6.31-rc3 | 18:42 |
| ogasawara | http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.31-rc3/ | 18:43 |
| ogasawara | And again for example sake, lets say the reporter again says 2.6.31-rc3 works. | 18:43 |
| ogasawara | Now lets ask them to test 2.6.31-rc5, http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.31-rc5/ | 18:43 |
| ogasawara | And agian for example sake, lets say 2.6.31-rc5 still works after they test. | 18:44 |
| ogasawara | Now we finally ask them to test 2.6.31-rc6, http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.31-rc6/ | 18:44 |
| ogasawara | And lets suppose they come back this time and report 2.6.31-rc6 fails to resume. | 18:44 |
| ogasawara | So now we've narrowed down the regression between 2.6.31-rc5 and 2.6.31-rc6. | 18:44 |
| ogasawara | This is definitely a big help, however there are still 578 commits between -rc5 and -rc6. | 18:45 |
| ogasawara | Examining the timestamps for each of those builds, you'll notice that 2.6.31-rc5 was built on 01-Aug-2009 and 2.6.31-rc6 was build on 14-Aug-2009. | 18:45 |
| ogasawara | The nice thing is the Ubuntu kernel team also provides mainline daily kernel builds. | 18:46 |
| ogasawara | http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/ | 18:46 |
| ogasawara | So now we can apply the same rough bisect idea but use the daily kernel builds. | 18:46 |
| ogasawara | We can then hopefully narrow down between which exact two dates the regression was introduced. | 18:46 |
| ogasawara | So just for example sake and to speed things along, lets say we narrowed down the regression between daily kernel build 10-Aug-2009 and 11-Aug-2009. | 18:47 |
| ogasawara | Looking at the 10-Aug-2009 build log, http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/2009-08-10/BUILD.LOG | 18:47 |
| ogasawara | I see this was built based on commit f4b9a988685da6386d7f9a72df3098bcc3270526 - "Merge branch 'for-linus' of git://git.infradead.org/ubi-2.6" | 18:47 |
| ogasawara | Likewise looking at the 11-Aug-2009 build log, http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/2009-08-11/BUILD.LOG | 18:47 |
| ogasawara | I see this was built based on commit 85dfd81dc57e8183a277ddd7a56aa65c96f3f487 "pty: fix data loss when stopped (^S/^Q)" | 18:48 |
| ogasawara | There are now only 45 commits between these two builds. | 18:48 |
| ogasawara | Pointing a developer to examine 45 commits is much more manageable than asking them to examining 37,000+ commits or even 578 commits. | 18:48 |
| ogasawara | From that point a developer should be able to post additional test kernels to narrow down the exact offending commit which is causing the regression. | 18:49 |
| ogasawara | Unfortunately I have not yet documented this rough bisect process in a wiki yet. | 18:49 |
| ogasawara | If anyone is feeling ambitious, feel free to document it and I'll be more than happy to proof read :) | 18:49 |
| ogasawara | So I wanted to leave the remaining time to field any questions. | 18:50 |
| ogasawara | I apologize if I didn't get to something you were hoping I would. | 18:50 |
| ogasawara | I would however point you to our KnowledgeBase that contains lots of good additional debug information. | 18:50 |
| ogasawara | https://wiki.ubuntu.com/KernelTeam/KnowledgeBase | 18:50 |
| ogasawara | syedam> QUESTION : instead of using a bisect approach cant we see what files were touched in a paticular commit and check that commit and use this with the bisect method | 18:50 |
| ogasawara | syedam: indeed each commit outlines the changed set of files, and you could much more quickly do a bisect off of that | 18:51 |
| ogasawara | syedam: however that would require having some git knowledge and knowing how to build your own kernel | 18:52 |
| ogasawara | syedam> QUESTION: is there an easy way to build and package kernels | 18:52 |
| ogasawara | syedam: indeed there are. I'd take a look at https://help.ubuntu.com/community/Kernel/Compile | 18:53 |
| ogasawara | any other questions? if not we'll end just a few minutes early and I'll turn it over to didrocks. | 18:55 |
| ogasawara | bas89> QUESTION: If my system crashes, what are the main signals that the kernel was responsible for it? | 18:55 |
| ogasawara | bas89: the most noticeable indication will probably be a kernel panic | 18:55 |
| ogasawara | bas89: sometimes it'll get logged to dmesg, but sometimes you'll just see it flashed to your terminal and that's it | 18:56 |
| ogasawara | bas89: it's always best to capture as much of the panic as possible if you report it in a bug | 18:56 |
| ogasawara | Quarth> QUESTION : Are changes classifyed or someway seachchable by any classification method (like key words or subsystems, layers., sections..)? Real case: i have a problem with my lapton screen resolution, starting with 31-4 runs fine, starting with 31-8 is wrong. Should I use the bisect approach or there's a better way? | 18:56 |
| ogasawara | Quarth: if you are familiar with git you could list all the changes based on a subsystem, if not I'd suggest using the bisect approach | 18:57 |
| ogasawara | syedam> QUESTION: are the mainline / daily kernels built with debugging info | 18:57 |
| ogasawara | syedam: unfortunately I don't think so. I believe there is actually an existing bug report in Launchpad requesting this | 18:58 |
| ogasawara | falstaff|h> question: It also happend to me that ubuntu just freezed... Magic Keys worked, can i get the last output of dmesg after rebooting? syslog doesnt contained them.. | 18:58 |
| ogasawara | falstaff|h: try examining /var/log/kern.log.0 which should contain logs for a few boot cycles | 18:58 |
| ogasawara | Ok, I'll turn it over to didrocks who'd going to teach you how to update a package. | 19:00 |
| ogasawara | Thanks everyone! | 19:00 |
| didrocks | Thanks a lot ogasawara and congrats! | 19:00 |
| didrocks | I'll wait for a couple of minutes before beginning :) | 19:00 |
| didrocks | In the meanwhile, you can install a few packages: | 19:01 |
| didrocks | sudo apt-get install build-essential devscripts ubuntu-dev-tools debhelper diff patch quilt fakeroot lintian libtool gnome-common gnome-doc-utils gtk-doc-tools | 19:01 |
| didrocks | and ensure you have "deb-src http://archive.ubuntu.com/ubuntu/ jaunty main restricted" in /etc/apt/sources.list (and then executes sudo apt-get update) | 19:01 |
| didrocks | DING DONG, it's time to get started and fire some package updates! | 19:03 |
| didrocks | just to know a little about the audience, who is ready to update some packages? (answer on #ubuntu-clasroom-chat) :) | 19:04 |
| didrocks | waow, some people there! For those who don't know the basics/can't practice, there will be a lot of copy/paste in pastebin so that you can follow the lesson :) | 19:05 |
| didrocks | you can follow the session and do what I type in a jaunty ubuntu box | 19:06 |
| didrocks | just ensure you installed and changed what I said earlier ^ | 19:06 |
| didrocks | I will begin with a very generalist introduction so that people can follow what will be in this lesson | 19:06 |
| didrocks | <Quarth> didrock should I change jaunty for kamic? | 19:07 |
| didrocks | Quarth: no no, really, I adapted the lesson so that you can use jaunty :) | 19:07 |
| didrocks | so, just drop the "deb-src http://archive.ubuntu.com/ubuntu/ jaunty main restricted" line | 19:08 |
| didrocks | ok, some introduction: | 19:08 |
| didrocks | As most of user want to live on the edge about what best the Open Source community has to offer, we are going to see how to update a package to offer the very last release to all ubuntu users. | 19:08 |
| didrocks | First, be warned that once a release is out and for all supported releases (jaunty soon!), we never update a package to a new software version (appart from backports repository and ppa, when requested). | 19:08 |
| didrocks | We only cherrypick bug and security fixes from a new release to adapt it an older version. This is intended to have as little breakage as possible. | 19:09 |
| didrocks | So, that why OpenOffice 3 didn't do it into Intrepid. On the contrary, Jaunty that I hope all of you use, have it :) | 19:09 |
| didrocks | do you have any queston about what can be elected as an update, and what can't? | 19:10 |
| didrocks | apparently, everyone knows, so, I'm going on :) | 19:11 |
| didrocks | Today, we are going to update gnome-terminal. We will see quickly what are the different steps we have to handle generally to update packages, but the best is, of course, to practice! | 19:11 |
| didrocks | Even if I use bzr-buildpackage now to work on, we will not use it today. The Unstoppable James Westby will give an introduction on this this week | 19:12 |
| didrocks | Also, this lesson is not intended to teach you how to package. For this, see the corresponding courses in last developpers week session (https://wiki.ubuntu.com/UbuntuDeveloperWeek). Don't forget also the excellent packaginguide: https://wiki.ubuntu.com/PackagingGuide. | 19:12 |
| didrocks | ell, ready? Let's download the current version of gnome-terminal: | 19:12 |
| didrocks | mkdir gnome-terminal && cd gnome-terminal && apt-get source gnome-terminal | 19:12 |
| didrocks | This will download the last release present in jaunty, which is 2.26.0. | 19:13 |
| didrocks | tell me on -chat when you are ready :) | 19:13 |
| didrocks | ok, most of people seems to be ready. Let's get into the source package: cd gnome-terminal-2.26.0 | 19:14 |
| didrocks | To check if new release is available, if a debian/watch file is present, we just have to use: uscan --report --verbose. | 19:14 |
| didrocks | The output should be something like this: http://paste.ubuntu.com/262929/. You can see there that a new version is available and corresponds to 2.27.91 | 19:15 |
| didrocks | <EagleScreen> QUESTION: in which package is scan command? | 19:15 |
| didrocks | EagleScreen: it's not scan, but _uscan_ | 19:16 |
| didrocks | this one will be pull as a dep of devscripts (see before for compulsory package) | 19:16 |
| didrocks | (remember, you have to download this: sudo apt-get install build-essential devscripts ubuntu-dev-tools debhelper diff patch quilt fakeroot lintian libtool gnome-common gnome-doc-utils gtk-doc-tools) | 19:17 |
| didrocks | (and yes, that's a bunch :) | 19:17 |
| didrocks | so, 2.27.91 is out. Excited by this new version? \o/ Let's get the new upstream code using uscan! This command just download the new archive, and extract its contents, whith the debian/ubuntu changes applied! | 19:18 |
| didrocks | so, just executes uscan this time, with no option :) | 19:18 |
| didrocks | The output of the command is telling us that we have to do a "cd ../gnome-terminal-2.27.91" to get into the new package, let's do it | 19:19 |
| didrocks | you should get something similar to that: http://paste.ubuntu.com/262930/ | 19:19 |
| didrocks | is everything finished? :) | 19:20 |
| didrocks | <bas89> wow that is easy going! | 19:20 |
| didrocks | false friend, we have still a lot to do :) | 19:20 |
| didrocks | Easy, isn't it? Well, when the debian diff doesn't apply because of inline patch, it's getting more difficult, but most of packages are in good shape and every debian difference from vanilla version are | 19:21 |
| didrocks | in a seperate debian/ folder | 19:22 |
| didrocks | So, now, the hard begins :) | 19:22 |
| didrocks | hard work* | 19:22 |
| didrocks | just to precise what's done when executing uscan: | 19:22 |
| didrocks | - it's downloading the vanilla tar file, | 19:23 |
| didrocks | then, trying to apply ../gnome-terminal-2.26.0-0ubuntu2.diff.gz diff file | 19:23 |
| didrocks | (this file contains the debian/ diff) | 19:23 |
| didrocks | it also adds a new entry in debian/changelog to update to last version | 19:23 |
| didrocks | <trothigar> QUESTION: What is the difference between inline patches and patches under debian/patches? | 19:23 |
| didrocks | trothigar: in short, inline patches is bad (in my opinion) :) | 19:24 |
| didrocks | so, inline patches add diff in diff.gz, but those diffs applied directly to files | 19:24 |
| didrocks | patches under debian/patches enables still to use uscan, even if the patch won't apply in the new sources | 19:25 |
| didrocks | (we will experience that later ^^) | 19:25 |
| didrocks | bas89> QUESTION: Does uscan get that new version from upstream? What happens if there is an ubuntu-customized version of gnome-terminal on our harddisk? | 19:25 |
| didrocks | uscan looks at debian/watch to know where to fetch the original tarball | 19:26 |
| didrocks | if you downloaded it manually, you can use the uupdate <path to original tarball> | 19:26 |
| didrocks | this command will do the same thing than uscan, without downloading it (run it in your source package as well) | 19:26 |
| didrocks | in fact uscan calls uupdate :) | 19:26 |
| didrocks | <dinxter> QUESTION: What methods does uscan work with, tarballs, bzipped, svn?... | 19:27 |
| didrocks | from what I know, it works with tarballs and bzipped (if we just get origin tarball target) | 19:27 |
| didrocks | Now begins the real packager work. We have to see what changed in upstream release reading the NEWS files (less NEWS and q to exit): http://paste.ubuntu.com/262931/ | 19:28 |
| didrocks | This is mostly a bugfix release. We will see later what has been fixed. Now, let's discover what changed in configure.{ac,in} file: diff -Nup ../gnome-terminal-2.26.0/configure.ac configure.ac | 19:28 |
| didrocks | You will get http://paste.ubuntu.com/262933/ | 19:28 |
| didrocks | Here are the most important changes for a packager from previous version to the last release | 19:29 |
| didrocks | What matters for us? gt_version_minor and gt_version_micro changed to tell that a new version is available. That's just tell that upstream did a good job. | 19:30 |
| didrocks | If it's not present, there is for librairies something like SHVER for library that you have to change in debian/rules. In every cases, it's good to give a look at debian/rules to see if the version number is | 19:30 |
| didrocks | present in this file (debian/rules) | 19:30 |
| didrocks | Ok for everyone? | 19:30 |
| didrocks | What is most important here is the GTK_REQUIRED and VTE_REQUIRED change. That means that we have to bump the dependencies version request of the package (it will request now 2.14.0 for gtk and 0.20.0 for the new version) | 19:31 |
| didrocks | You can edit it with your prefered tool (vim ROCKS \o/) and change libgtk2.0-dev (>= 2.13.6) and libvte-dev (>= 1:0.19.1) to libgtk2.0-dev (>= 2.14.0) and libvte-dev (>= 1:0.20.0). | 19:32 |
| didrocks | those changes have to take place in debian/control.in | 19:32 |
| didrocks | <trothigar> QUESTION: what's the difference between control and control.in? | 19:33 |
| didrocks | hehe, that was my next sentence :) | 19:33 |
| didrocks | So then, as there is a debian/control.in file, we have to generate a new debian/control file from it. This is proceed by executing: DEB_AUTO_UPDATE_DEBIAN_CONTROL=yes fakeroot debian/rules clean | 19:33 |
| didrocks | so, debian/control.in is used to generate debian/control | 19:34 |
| didrocks | it generally adds some automation to debian/control, like last uploaders update, @GNOME_TEAM@ in debian is replaced by the debian gnome team mailing list | 19:34 |
| didrocks | Finally, inform of your change! dch -a and add to the file so that it will look like: http://paste.ubuntu.com/151422/. You see that the so kind uscan command has automatically created "gnome-terminal | 19:35 |
| didrocks | oupss, wrong pastebin, correct one is:http://paste.ubuntu.com/262934 | 19:35 |
| didrocks | You see that the so kind uscan command has automatically created "gnome-terminal 2.27.91" | 19:35 |
| didrocks | tell me when it's done :) | 19:35 |
| didrocks | (the diff diff files between old and new configure.in files can also tell us from added/removed dependencies btw. There is no black packager magic there :D) | 19:36 |
| didrocks | Ok, now that build dependencies are ok, we have to see if ubuntu/debian patches still apply to the new version. The what-patch commands tells us that this package usescdbs. Let's try using cdbs-edit-patch debian/patches/99_autoreconf.patch (last patch of the list) | 19:37 |
| didrocks | <EagleScreen> why this http://pastebin.ca/1550600 ? | 19:38 |
| didrocks | this command is to update debian/control from debian/control.in | 19:39 |
| didrocks | the fakeroot is to simulate that we are root even if we aren't :) | 19:39 |
| didrocks | (most debian/rules commands must be launched as root) | 19:39 |
| didrocks | DEB_AUTO_UPDATE_DEBIAN_CONTROL=yes is used to desactivate some stuff done by "debian/rules clean" as we just want to update debian/control.in | 19:40 |
| didrocks | to make this successfully, you need to have gnome-pkg-tools | 19:40 |
| didrocks | (that's why I listed it at the begining ^^) | 19:40 |
| didrocks | ok, so, what happened when we tried to apply the patch? | 19:41 |
| didrocks | We exited in error in the debian/patches/30_honour_point_pixel_sizes.patch | 19:41 |
| didrocks | That can mean two things: either upstream has integrated the patch (or we took previously the patch from upstream svn), or that the code has been slightely modified and we can't apply it easily. | 19:41 |
| didrocks | Looking at debian/changelog has to be the first thing to do: http://paste.ubuntu.com/151431/ | 19:42 |
| didrocks | <EagleScreen> QUESTION: how can I change the text editor that dch uses by default? | 19:42 |
| didrocks | EagleScreen: just change the environment EDITOR variable, or use update-alternatives | 19:42 |
| didrocks | In this case, we see a bug report LP: #345189 associated to the patch. Looking at it (https://bugs.launchpad.net/bugs/345189), we deduce that the change is present upstream, looking at the Fix Released | 19:43 |
| didrocks | so, the fix has been fixed upstream | 19:43 |
| didrocks | Consequently, we can safely remove the patch, "rm debian/patches/30_honour_point_pixel_sizes.patch" as it's no more useful | 19:43 |
| didrocks | If it wasn't the case and the cause was that upstream changed slightely its code, we had to cdbs ....patch and adapt it to make it apply | 19:44 |
| didrocks | (cdbs-edit-patch <file>.patch) | 19:44 |
| didrocks | I try to not use too many abbreviations, sorry :) | 19:44 |
| didrocks | <frandieguez__> QUESTION: so Fix Released at launchpad means the patch is applied to trunk? | 19:45 |
| didrocks | frandieguez: you have upstream task and package task on LP | 19:45 |
| didrocks | if upstream task is written as fixed release, they rolled a tarball with the fix | 19:45 |
| didrocks | if package task is set to fix released, that mean that at least, the fixed version is in the development distribution (presently, karmic) | 19:46 |
| didrocks | So, when something is fixed upstream, we can remove the patch | 19:46 |
| didrocks | That's why we have always to report our patch upstream (appart from specific ubuntu ones) :) | 19:47 |
| didrocks | it's good for them, less work for us, everyone wins \o/ | 19:47 |
| didrocks | Ok, bring this information to debian/changelog: dch -a and report the change to make it look like that: http://paste.ubuntu.com/262937/ | 19:47 |
| didrocks | so, if the patch was an inline patch, uupdate/uscan would have failed | 19:48 |
| didrocks | (as the patch will be directly in diff.gz) | 19:48 |
| didrocks | so, this will more clutter our work | 19:48 |
| didrocks | Ok! Let's go on with next patch: $ cdbs-edit-patch debian/patches/99_autoreconf.patch again. | 19:49 |
| didrocks | The last patch doesn't apply /o\ | 19:49 |
| didrocks | That's pretty normal: autotools/autoconf/autoreconf patch are different from others patches. They basically consist of generating configure scripts from configure.in, makefile.in on (yes, the famous ./configure is generated there!) | 19:49 |
| didrocks | We have to exit first, without updating the patch: "exit 1" | 19:49 |
| didrocks | (exit 0 update the patch, exit 1 update it) | 19:50 |
| didrocks | oupss | 19:50 |
| didrocks | (exit 0 updates the patch, exit 1 does not update it) | 19:50 |
| didrocks | that's better :) | 19:50 |
| didrocks | (for those interested: a good introduction about patch system is there: https://wiki.ubuntu.com/MeetingLogs/devweek0809/PackagePatches | 19:50 |
| didrocks | so, now, we will regenerate the patch from scratch | 19:51 |
| didrocks | what you can do, is to remove the patch: "rm debian/patches/99_autoreconf.patch" | 19:51 |
| didrocks | creating a blank one again: $ cdbs-edit-patch debian/patches/99_autoreconf.patch | 19:51 |
| didrocks | (it will again drop you in a subshell) | 19:52 |
| didrocks | then, let's regenerate new configure and makefiles: to generate the configure and makefiles: autoreconf | 19:52 |
| didrocks | so, this takes Makefile.in to create Makfile, configure.in to create configure, and so on. | 19:52 |
| didrocks | Once done (ignore the warnings), exit 0 to refresh the patch and document the change: dch -a to get http://paste.ubuntu.com/262940/ | 19:53 |
| didrocks | We have almost finished: every patches applies and build-dependencies are ok. Normally, you testbuild at this stage, but we won't do it as we are running out of time | 19:55 |
| didrocks | Once done, a good practice is to put changes from the upstream NEWS file in the changelog. So, get the changes following the given link in the NEWS file and report it to the changelog. | 19:56 |
| didrocks | once this extra bonus point is done, you will get this: http://paste.ubuntu.com/151454/ | 19:56 |
| didrocks | You can take a breath now! You have your new package updated! Think about testing it throughly and everything will be all right. | 19:57 |
| didrocks | Last thing to note is to check which bugs are fixed in current upstream release, find them on launchpad to be able to close them as well by referencing them in debian/changelog. | 19:57 |
| didrocks | This can sometimes take a lot of times. | 19:57 |
| didrocks | time* | 19:57 |
| didrocks | ok, that's all for this session, there are 3 minutes left for question, you can fire up quickly :) | 19:58 |
| didrocks | <bas89> ho do i get my updated version into the karmic repos? | 19:58 |
| didrocks | bas89: you should give a look to the sponsoring process until you have upload rights: https://wiki.ubuntu.com/SponsorshipProcess | 19:59 |
| didrocks | bas89: btw, this package is already updated. I picked it because it was interesting with a lot of issues :) | 19:59 |
| didrocks | <rugby471> QUESTION: if say you have a small patch, is it better to try and get it applied upstream and the update the package, or write a debian patch? | 19:59 |
| didrocks | rugby471: upstream is always the best choice. If you really feel you need it into the last version of ubuntu, apply in both places | 20:00 |
| didrocks | <frandieguez__> QUESTION: if a LOCO Team make new translations to a package why this is don't commited to the repository in order to make that release more completed... ?? I ask when the release of ubuntu has do | 20:00 |
| didrocks | (this will be the last question) | 20:01 |
| didrocks | frandieguez: frankly, I don't know a lot about translations, but new versions are dropped in localized packages which are updated regularly, even when the new version is released | 20:01 |
| didrocks | thanks all for all your question, I'll still be in -chat for a couple of minutes, don't hesitate to ask remaining questions | 20:02 |
| didrocks | now, the stage is opened to leonardr | 20:02 |
| leonardr | thanks didrocks | 20:02 |
| didrocks | he will teach you about blackmagic on pythonlaunchpadlib :) | 20:03 |
| leonardr | My name is Leonard Richardson. I'm on the Launchpad Foundations team and I'm the co-author of the O'Reilly book "RESTful Web Services". | 20:03 |
| leonardr | I'm here to talk about the Launchpad web service API--how to use it and what advances have been made since the last UDW. | 20:03 |
| leonardr | I'll do an infodump for a few minutes and then take your questions for the rest of the hour. | 20:03 |
| leonardr | If you have questions during the infodump, just put them in #ubuntu-classroom-chat. | 20:04 |
| leonardr | I give this infodump at every UDW, so it may be familiar to you. I ask you to bear with me so I can get everyone up to speed. | 20:04 |
| leonardr | 1. Intro | 20:04 |
| leonardr | First thing to know is that we've got docs talking about the API here: https://help.launchpad.net/API | 20:04 |
| leonardr | Put simply, we've created an easy way for you to integrate Launchpad into your own applications. | 20:04 |
| leonardr | f you perform the same tasks on Launchpad over and over again, you can write a script that automates the tasks for you. | 20:04 |
| leonardr | You don't have to rely on fragile screen-scraping. | 20:04 |
| leonardr | If you're a developer of an IDE, testing framework, or some other program that has something to do with software development, you can integrate Launchpad into the program to streamline the development processes. | 20:05 |
| leonardr | If you run a website for a project hosted on Launchpad, you can get project data from Launchpad and publish it on your website. | 20:05 |
| leonardr | And so on. You can use the API to do most of the things you can do through the Launchpad web site. | 20:05 |
| === NoX is now known as Guest50806 | ||
| leonardr | 2. Tools | 20:05 |
| leonardr | The simplest way to integrate is to use launchpadlib, a Python library we've written. | 20:05 |
| leonardr | see https://help.launchpad.net/API/launchpadlib, ubuntu package python-launchpadlib) | 20:05 |
| leonardr | This gives you a Python-idiomatic interface to the Launchpad API, so you don't have to know anything about HTTP client programming: | 20:06 |
| leonardr | >>> launchpad.me.name | 20:06 |
| leonardr | u'leonardr' | 20:06 |
| leonardr | >>> launchpad.bugs[1].title | 20:06 |
| leonardr | u'Microsoft has a majority market share' | 20:06 |
| leonardr | But it's also easy to learn the API's HTTP-based protocol and write your own client in some other language. | 20:06 |
| leonardr | (see https://help.launchpad.net/API/Hacking) | 20:06 |
| leonardr | 3. Progress and Roadmap | 20:06 |
| leonardr | At the last UDW I said that the web service publishes information about people, bugs, code branches, archives, and the launchpad registry (the projects, milestones, etc.) | 20:06 |
| leonardr | Here's what's been published since then (AFAIK): | 20:06 |
| leonardr | * The translation import queue (more or less read-only) | 20:07 |
| leonardr | * Lots of distro stuff i don't understand, like package uploads and package sets | 20:07 |
| leonardr | * Merge proposals and code reviews | 20:07 |
| leonardr | * Project releases | 20:07 |
| leonardr | * The hardware database | 20:07 |
| leonardr | The future: | 20:07 |
| leonardr | Publication through the web service is still not a priority for translations (apart from the import queue, which is done), answers, or blueprints. | 20:07 |
| leonardr | Work on publishing new things on the LP web service has slowed down since the last UDW, since most of Launchpad is published now. | 20:07 |
| leonardr | Right now I'm working on making the underlying library (lazr.restful) an attractive option for anyone who wants to publish a web service. | 20:08 |
| leonardr | We are also working on improving the client. | 20:08 |
| leonardr | So, that's the infodump, i invite your questions. | 20:08 |
| leonardr | <rugby471> leonardr: I have used python-launchpadlib in memaker and I loved it's simplicity, however the one gripe I had with it was the very first authentication request the user has to do with launchpad, is there any plans/thoughts on how to make this a bit less clunky? | 20:08 |
| leonardr | rugby471: yes | 20:09 |
| leonardr | this work is tracked in bug 387297 | 20:09 |
| leonardr | basically, the workflow is the way it is because we wanted to exploit the fact that the user already trusts their web browser with their launchpad credentials, where (no offense) they don't trust your application | 20:10 |
| leonardr | because developers dislike this workflow to the extent that they're hacking around it, we've decided to create some alternate trusted clients | 20:11 |
| leonardr | instead of opening up the user's web browser, you'll be able to run a console trusted client app, a gtk client app, etc | 20:11 |
| leonardr | it'll be similar to the pinentry application | 20:11 |
| leonardr | this is kind of on the back burner while i work on lazr.restful, but it is in progress | 20:12 |
| leonardr | any other questions? | 20:12 |
| leonardr | <frandieguez__> QUESTION: are there libraries for other languages, like ruby? | 20:13 |
| leonardr | the only other library i know of is the one we wrote in javascript to use in launchpad ajax code | 20:13 |
| leonardr | if you know of a good wadl library for a language, it's not difficult to write a launchpadlib-like library on top of it | 20:14 |
| leonardr | just read in the wadl file that describes launchpad's capabilities | 20:14 |
| leonardr | however, wadl libraries are not very common | 20:14 |
| leonardr | in addition to taking questions, if anyone attending this chat has written a library that uses launchpadlib, or integrated the web service into an applicatino, i'd like to hear about it | 20:15 |
| leonardr | i'll collate what you say and put it in at the end so people can see what's been done already | 20:15 |
| leonardr | #ubuntu-classroom-chat is a little quiet, but i'll be here until bdmurray takes over in 40 minutes | 20:18 |
| leonardr | as long as we're here i'll talk a bit about the server side of things | 20:19 |
| leonardr | which is what i'm working on now | 20:19 |
| leonardr | i'm running a project called lazr.restful | 20:20 |
| leonardr | https://edge.launchpad.net/lazr.restful | 20:20 |
| leonardr | this used to be part of launchpad and was split out and made open source a few months before launchpad was | 20:20 |
| leonardr | it's a zope application that publishes data model objects through a web service | 20:21 |
| leonardr | for the past couple of months i've been working on making it appealing to people who don't use zope | 20:21 |
| leonardr | right now we have the zope application hidden behind a piece of wsgi middleware | 20:21 |
| leonardr | and i've been able to use it internally to publish django model objects | 20:22 |
| leonardr | so pretty soon it should be usable by the zope-phobic | 20:22 |
| leonardr | <tormod> QUESTION: are there some "hello world" examples on say, how to mass-process bugs etc | 20:22 |
| leonardr | there are several example scripts in launchpadlib/examples, and there's a big pile of them about to be added to that directory (i'm not sure from where) | 20:23 |
| leonardr | let me see if there's one on there dealing with bugs | 20:23 |
| leonardr | it's actually samples/, not examples/, and there is no bugs code there | 20:24 |
| leonardr | i can work up something live, right now | 20:24 |
| leonardr | so, i've got a python session going, and i've got the api reference in my web browser | 20:25 |
| leonardr | that's https://edge.launchpad.net/+apidoc/ | 20:25 |
| leonardr | two steps to operate on a bunch of bugs: 1. identify the bugs, 2. operate on them | 20:26 |
| leonardr | <tormod> if you need an example, change all xorg.0.logs to plain/text :) | 20:27 |
| leonardr | ok, i'll do what i can | 20:27 |
| leonardr | we start by navigating to the bugs | 20:27 |
| leonardr | >>> xorg = launchpad.projects['xorg'] | 20:27 |
| leonardr | i'm looking at https://edge.launchpad.net/+apidoc/#project for something that will help | 20:28 |
| leonardr | (i don't actually know the lp api very well, as i only wrote the framework) | 20:28 |
| leonardr | i think it might be smarter to go in through https://edge.launchpad.net/+apidoc/#bugs | 20:29 |
| leonardr | ok, an introspection method, xorg.lp_operations, shows an operation called 'searchTasks' | 20:31 |
| leonardr | that'll get us closer to the bugs | 20:31 |
| leonardr | i'll just get the new tasks: | 20:32 |
| leonardr | >>> tasks = xorg.searchTasks(status="New") | 20:32 |
| leonardr | now we iterate over the tasks, find any attachments, and hack the attachments as per tormod's example | 20:33 |
| leonardr | so let's take one task as an example; then the rest is just iteration | 20:34 |
| leonardr | we have the task. the bug attachments are on the bug | 20:35 |
| leonardr | 'bug' is a property of bug_task, so: bug = task.bug | 20:35 |
| leonardr | each bug has a collection of attachments, so we iterate over bug.attachments | 20:36 |
| leonardr | i'm just picking the first task, the first attachment, etc | 20:36 |
| leonardr | so the code i've run so far is: | 20:36 |
| leonardr | bugs = [x.bug for x in tasks] | 20:36 |
| leonardr | bug = bugs[0] | 20:36 |
| leonardr | attachment = [x for x in bug.attachments][0] | 20:37 |
| leonardr | tormod points out: | 20:37 |
| leonardr | <tormod> I can not find "mime type" under https://edge.launchpad.net/+apidoc/#bug_attachment | 20:37 |
| leonardr | that's because the bug_attachment object isn't the actual attachment | 20:38 |
| leonardr | it's an entry in the database containing information about the attachment | 20:38 |
| leonardr | the actual attachment is available as attachment.data | 20:38 |
| leonardr | if you get attachment.data in launchpadlib you'll have a HostedFile object | 20:38 |
| leonardr | i need to look up how these work... | 20:39 |
| leonardr | i'm looking in https://help.launchpad.net/API/launchpadlib | 20:39 |
| leonardr | https://help.launchpad.net/API/launchpadlib#Hosted%20files | 20:39 |
| leonardr | rugby471 is familiar with these objects since a mugshot works the same way | 20:40 |
| leonardr | we can get a filehandle on the object by opening it for read | 20:40 |
| leonardr | and the mime type is an attribute on the filehandle | 20:40 |
| leonardr | >>> handle = attachment.data.open() | 20:41 |
| leonardr | >>> handle | 20:41 |
| leonardr | <launchpadlib.resource.HostedFileBuffer instance at 0x9e2a72c> | 20:41 |
| leonardr | >>> handle.filename | 20:41 |
| leonardr | 'Xorg.0.log' | 20:41 |
| leonardr | >>> handle.content_type | 20:41 |
| leonardr | 'text/plain' | 20:41 |
| leonardr | so that content type is ok | 20:42 |
| leonardr | but let's say we wanted to change it | 20:42 |
| leonardr | because these files are stored in the launchpad librarian, you can't just change the content type and save, the way you can change a bug's description | 20:42 |
| leonardr | you need to create a new librarian file | 20:42 |
| leonardr | basically, open the filehandle for write, specifying the correct content type and filename. then write the content to the filehandle and close it | 20:42 |
| leonardr | something like this | 20:43 |
| leonardr | >>> old_name = handle.filename | 20:43 |
| leonardr | >>> old_content = handle.read() | 20:43 |
| leonardr | >>> write_handle = attachment.data.open("w", "text/plain", old_name) | 20:43 |
| leonardr | >>> write.handle.write(old_content) | 20:43 |
| leonardr | >>> write_handle.close() | 20:44 |
| leonardr | not the most convenient code (because librarian files are write-once), but it's possible | 20:44 |
| leonardr | so you'd do that for every bug in the xorg project that met your criteria | 20:44 |
| leonardr | unfortunately there's no way to directly specify your criteria "bugs that have an attachment called Xorg.0.log with a content type other than text/plain" | 20:45 |
| leonardr | you'll need to use some cruder criteria, such as new bugs, as i used | 20:45 |
| leonardr | ^arky^ requested the code i just put up on pastebin: http://pastebin.ubuntu.com/263382/ | 20:48 |
| leonardr | no guarantees that will work as is, but by using dir() you should be able to make it work | 20:48 |
| leonardr | and that's how i proceed in general when writing these scripts | 20:49 |
| leonardr | i don't know much about lp, but i use the introspection methods and the api doc to zoom in on where i want to be | 20:49 |
| leonardr | <tormod> do you have to delete the old file from the librarian then? or are they indexed by filename? | 20:49 |
| leonardr | you can delete attachment.data with attachment.data.delete(). i don't think it's necessary. the old attachment will just stop beign referenced and will eventualyl be garbage collected | 20:50 |
| leonardr | i believe they're indexed by a unique id that you never see. that's how there can be 10,000 different attachments all called Xorg.0.log | 20:51 |
| leonardr | hm, i've been talking in classroom-chat by mistake | 20:55 |
| leonardr | i'll paste in a few q & as | 20:56 |
| leonardr | <^arky^> QUESTION: Can we use python-launchpadlib (1.5.1-0ubuntu1) to try these code examples | 20:56 |
| leonardr | <leonardr> ^arky^: yes, but the first time you run one of these scripts you will need to give launchpadlib permission to access your launchpad account | 20:56 |
| leonardr | <tormod> what was dir() about? | 20:56 |
| leonardr | <leonardr> tormod: you can use dir() on any launchpadlib object to see its capabilities are | 20:56 |
| leonardr | <leonardr> so we also created a bunch of special introspection properties | 20:56 |
| leonardr | <leonardr> described here: https://help.launchpad.net/API/launchpadlib#Getting%20help | 20:56 |
| leonardr | <leonardr> a quick example of the introspection properties: | 20:56 |
| leonardr | <leonardr> >>> launchpad.people['leonardr'].lp_entries | 20:56 |
| leonardr | <leonardr> ['preferred_email_address', 'archive', 'team_owner', 'mugshot'] | 20:56 |
| leonardr | <leonardr> lp_entries shows you the individual objects associated with some other object | 20:56 |
| leonardr | <leonardr> in this case, an email address, an archive, another person (if leonardr were a team, it might have an owner), and a binary file (mugshot) | 20:56 |
| leonardr | it's almost time for bdmurray to take over, so any last minute questions | 20:57 |
| leonardr | i don't see bdmurray here (maybe he has some other alias) | 20:59 |
| bdmurray | leonardr: I'm here ;-) | 21:00 |
| leonardr | before i go, one example of an app that integrates into launchpad, from rugby471 | 21:00 |
| leonardr | I have integrated it into memaker for one click updating of the user's mugshot ( package memaker in karmic) | 21:00 |
| leonardr | bdmurray: ah, i was looking to give you op, but you already have it | 21:00 |
| leonardr | one more question before he takes over | 21:00 |
| leonardr | <thekorn> leonardr, QUESTION: is the ability to change the content of an attachment a feature or a bug? | 21:00 |
| leonardr | i don't know. you'd need to ask the bugs team. my guess is it's a feature | 21:01 |
| leonardr | in general, it's a feature that you can replace the contents of a hosted file (like a mugshot), but maybe the bugs team wants you to upload a new attachment rather than modifying one | 21:01 |
| leonardr | bdmurray, i yield the floor | 21:03 |
| bdmurray | leonardr: thanks! | 21:03 |
| bdmurray | Hi, my name is Brian Murray and I'm a member of the Ubuntu QA team. | 21:03 |
| bdmurray | I'm here today to talk about how you can get higher quality bug reports about packages that you care about. | 21:03 |
| bdmurray | You can do this by writing an apport hook for your particular package. | 21:04 |
| bdmurray | First off, what is apport? | 21:04 |
| bdmurray | Apport is a system which intercepts crashes right when they happen, in development releases of Ubuntu, and gathers useful information about the crash and the operating system environment. | 21:04 |
| bdmurray | Additionally, it is used as a mechanism to file non-crash bug reports about software so that we receive more detailed reports. | 21:05 |
| bdmurray | Let's look at a sample apport bug report - http://launchpad.net/bugs/416701. | 21:05 |
| bdmurray | The bzr package does not have an apport hook but some useful information is still collected. | 21:05 |
| bdmurray | We have the architecture, the release being used, the package version and the source package name. | 21:05 |
| bdmurray | Additionally, in the Dependencies.txt attachment we have information about the versions of packages upon which bzr depends. | 21:06 |
| bdmurray | Are there any questions about apport so far? | 21:06 |
| bdmurray | Okay then, carrying on. | 21:07 |
| bdmurray | So while all of that can be really useful an apport hook for a package allows us to gather specific information for that package. | 21:08 |
| bdmurray | For example, consider a bug report about usplash. | 21:08 |
| bdmurray | usplash has a dedicated configuration file, located at "/etc/usplash.conf", and this would be something quite helpful in debugging a usplash bug report but not very useful in other package bug reports. | 21:08 |
| bdmurray | Apport looks for package hooks in "/usr/share/apport/package-hooks/" for ones named after the package for which they will be used. | 21:08 |
| bdmurray | Looking in "/usr/share/apport/package-hooks/" lets take a look at the usplash hook - with the filename usplash.py. | 21:09 |
| bdmurray | The package hooks are written in python. | 21:09 |
| bdmurray | We can see that the usplash.py hook imports the apport.hookutils module - "from apport.hookutils import *". | 21:09 |
| bdmurray | hookutils is a collection of readymade and safe functions for many commonly used things. There are functions for attaching a file's contents, getting a command's output, grabbing hardware information and much more. | 21:10 |
| bdmurray | This package hook is using 'attach_file_if_exists' and 'attach_hardware'. | 21:10 |
| bdmurray | 'attach_file_if_exists' is pretty self explanatory but what does attach_hardware include? | 21:10 |
| bdmurray | Let's look at the hookutils module to find out. | 21:10 |
| bdmurray | You can use "python -c 'import apport.hookutils; help(apport.hookutils)'" or you can view it using codebrowse - http://bazaar.launchpad.net/~apport-hackers/apport/trunk/annotate/head%3A/apport/hookutils.py. | 21:11 |
| bdmurray | he 'attach_hardware' function starts at line 72. | 21:12 |
| bdmurray | As we can see it adds a wide variety of hardware information to a bug report which can be quite useful for some packages like the kernel and usplash! | 21:12 |
| bdmurray | Having the apport package hooks has reduced the amount of bug ping pong necessary to get information out of a bug reporter and having these convenience functions reduces the amount of work it takes to write a package hook. | 21:12 |
| bdmurray | In addition to 'attach_hardware' and 'attach_file_if_exists' other functions include: 'attach_conffiles', 'attach_dmesg', 'attach_alsa', 'command_output', 'recent_syslog', 'pci_devices' and 'attach_related_packages'. | 21:13 |
| bdmurray | In the event that you have a group of packages that would benefit from a shared convenience function, please file a bug about apport and include the function you'd like added. | 21:13 |
| bdmurray | Are there any questions so far? | 21:14 |
| bdmurray | < tormod> QUESTION: is there a way to present a question to the bug reporter? | 21:15 |
| bdmurray | tormod: yes, there actually is and this is a new feature of apport I'll be getting to shorty. | 21:15 |
| bdmurray | shortly even ;-) | 21:16 |
| bdmurray | Back to the usplash hook, we can see that the usplash configuartion file is added like so "attach_file_if_exists(report, '/etc/usplash.conf', 'UsplashConf')". | 21:17 |
| bdmurray | This means that a bug reported about usplash using apport should have an attachment named 'UsplashConf' and it will contain the reporter's usplash.conf file. | 21:17 |
| bdmurray | Looking at http://launchpad.net/bugs/393238 we can see this actually isn't the case. | 21:17 |
| bdmurray | Because usplash.conf is only 4 or 5 lines it ends up getting put into the bug description. However, most items end up getting added as attachments in Launchpad. | 21:18 |
| bdmurray | Regardless the information is still there and can help triaging the bug. | 21:18 |
| bdmurray | Now that we've covered what a hook can include lets's look at how to control when hooks are run. | 21:18 |
| bdmurray | In the totem hook, "source_totem.py", we can see there is a check to see if the hook was called due to a crash - "if report['ProblemType'] == 'Crash':". | 21:19 |
| bdmurray | If it is due to a crash the hook is not run. | 21:19 |
| bdmurray | If someone were reporting a bug, using ubuntu-bug or 'Help -> Report a Problem', the ProblemType would be Bug. | 21:19 |
| bdmurray | In this case the totem hook is only ran when the "ProblemType" is "Bug". | 21:20 |
| bdmurray | < thekorn> QUESTION: let's say I'm writing an apport hook which adds a log file, should this hook automatically purge sensitive data, what's best practice in such cases? | 21:20 |
| bdmurray | thekorn: let me find an example - I saw a hook that does some scrubbing recently | 21:21 |
| bdmurray | thekorn: there is one in mysql-dfsg-5.1 | 21:23 |
| bdmurray | http://pastebin.ubuntu.com/263396/ | 21:24 |
| bdmurray | you can see it is checking for the password in the configuration file and replaces it before attaching it to the report | 21:24 |
| bdmurray | I think this, scrubbing the data before uploading, is the best idea | 21:25 |
| bdmurray | Now to tormod's question ... | 21:26 |
| bdmurray | The totem hook is particularly interesting as it is utilizes interactive questions - which is a new feature in apport. | 21:26 |
| bdmurray | You can see how this works by executing "ubuntu-bug totem", but please don't actually report the bug! ;-) | 21:26 |
| bdmurray | The totem hook asks questions and runs tests to determine if the bug report is related to alsa, pulseaudio or codecs in gstreamer. | 21:26 |
| bdmurray | Well add of course totem itself. | 21:27 |
| bdmurray | The totem apport hook also sets the affected package appropriately. | 21:27 |
| bdmurray | The interactive questions can greatly reduce the amount of triaging work required and helps make the initial bug report much more complete. | 21:28 |
| bdmurray | More detailed information regarding how to use the hook user interface can be found in the python help for the available functions: "python -c 'import apport.ui; help(apport.ui.HookUI)'". | 21:28 |
| bdmurray | Are there any questions about the interactive questions or the convenience functions? | 21:29 |
| bdmurray | Now that we know a lot of what hooks can do, lets talk about how to write and test one. | 21:34 |
| bdmurray | After the last Ubuntu Developer Summit I compiled a list of packages that had recently received a fair number of bug reports and subsequently might benefit from an apport package hook. | 21:35 |
| bdmurray | That list can be found at https://wiki.ubuntu.com/QATeam/Specs/IncreaseApportCoverage. | 21:35 |
| bdmurray | Let's take rhythmbox from that list and write a simple apport hook for it. | 21:35 |
| bdmurray | I say simple because I'm not entirely certain what would be useful but thought these things might be. | 21:36 |
| bdmurray | < andresmujica> QUESTION: What's the purpose of the general-hooks available at /usr/share/apport/general-hooks ? | 21:36 |
| bdmurray | andresmujica: these are always run. For example if you look at the automatix.py one you can see ifi certain criteria are met the bug reports are not filed. | 21:38 |
| bdmurray | And the ubuntu.py checks to see if the package is an ubuntu one among other things. | 21:40 |
| bdmurray | The easiest way of writing and testing a package hook is to put one in "/usr/share/apport/package-hooks" named after the appropriate package. | 21:40 |
| bdmurray | I'll create one called "source_rhythmbox.py". | 21:40 |
| bdmurray | We'll import hookutils from apport and os, then in the add_info function we'll add the rhythmdb.xml file, if it exists, and gconf settings for rhythmbox. | 21:41 |
| bdmurray | The hook as written looks like http://pastebin.ubuntu.com/262850/. | 21:41 |
| bdmurray | Its really only 2 or so lines of code but now we'll have some potentially useful information in every rhythmbox crash or bug report. | 21:41 |
| bdmurray | After I've put this file in "/usr/share/apport/package-hooks" I can test it using "ubuntu-bug". | 21:42 |
| bdmurray | After running "ubuntu-bug rhythmbox", apport presents the user with a dialog asking them if they want to send the problem report. | 21:42 |
| bdmurray | In this dialog box we can see the complete contents of the report and if our collected information was attached. | 21:42 |
| bdmurray | I see a key named rhythmdb.xml which contains the same as the one on my system and a key named GconfRhythbox which contains my gconf settings for rhythmbox. | 21:43 |
| bdmurray | contains the same information that is ;-) | 21:43 |
| bdmurray | If you look at the compiz hook, source_compiz.py, you can see it includes a debugging portion so you can just execute it. However, I personally find this harder to parse. | 21:44 |
| bdmurray | That's really all there is to writing and testing an apport hook! | 21:44 |
| bdmurray | < andresmujica> QUESTION: Why source_rhythmbox.py and not just | 21:45 |
| bdmurray | rhythmbox.py ? | 21:45 |
| bdmurray | andresmujica: this way any binary package produced by rhythmbox would have the hook run for it. | 21:46 |
| bdmurray | This is less of an issue with rhythmbox but makes a lot of sense with other packages. | 21:47 |
| bdmurray | Like evolution for example | 21:48 |
| bdmurray | thekorn: the evolution hook is also a good example of data scrubbing | 21:48 |
| bdmurray | In the event that you write a hook for a package that you can not upload or need help getting sponsored, please report a bug about the package missing a hook. | 21:49 |
| bdmurray | Then add the hook as a patch (or a merge proposal) and subscribe me, brian-murray, and the appropriate sponsor's team to the bug report. | 21:49 |
| bdmurray | I'll work to ensure that it gets uploaded. | 21:49 |
| bdmurray | If you need any further help the apport code is very well documented and there are a few wiki pages about it - https://wiki.ubuntu.com/Apport and https://wiki.ubuntu.com/Apport/DeveloperHowTo. Additionally, you can find me in the ubuntu-bugs channel. | 21:49 |
| bdmurray | < sbarthelemy> QUESTION: in which package should the new apport-hook go, in apport or in rhythmbox? | 21:49 |
| bdmurray | apport hooks are distributed with the package that will use them | 21:50 |
| bdmurray | er, the package that will benefit from them | 21:50 |
| bdmurray | that's why I didn't have the mysql hook readily availabe as the package is not installed on my system | 21:50 |
| bdmurray | additionally, this make it easier for us as developers as we can just stick them in our packages and not have to muck with apport | 21:51 |
| bdmurray | < openweek4> QUESTION: will apport hooks work with 3rd party applications? | 21:53 |
| bdmurray | openweek4: I'm actually really not certain about that question. If you look at the ubuntuone-client hook - bugs get reported to Launchpad but about the ubuntuone project. So it might very well be possible. However, apport would have to be able to communicate with the appropriate bug tracking system. | 21:54 |
| bdmurray | < sbarthelemy> QUESTION: the apport hook is expected to be included in debian's own rhythmbox too? | 21:55 |
| bdmurray | sbarthelemy: most of the apport hooks are carried as a diff from what I have seen | 21:55 |
| bdmurray | Are there any more questions? | 21:56 |
| bdmurray | Alright well thanks for coming everyone and I hope this was informative. I look forward to seeing some more apport hooks! | 21:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!