[00:38] <dtchen> for those of you idling for the piuparts/vm session, i've placed some notes at https://wiki.ubuntu.com/Packaging/Training/Logs/2009-04-23
[00:39] <dtchen> it's a good idea to go ahead and configure pbuilder, piuparts, and your vm
[01:00] <dtchen> all right folks, it being 00:00 utc and such, let's roll on through. i hear there's a release being prepared. :-)
[01:01] <nhandler> :)
[01:01] <dtchen> again, this packaging session will introduce piuparts for package testing, and we'll mention how to use a VM to further test functionality
[01:02] <dtchen> there are some notes (and corrections!) at https://wiki.ubuntu.com/Packaging/Training/Logs/2009-04-23, so please take look and get your prerequisites all shiny
[01:02] <kklimonda> damn, I've managed to stay awake for this session.. any cookies for me? ;}
[01:02] <dtchen> cough, take a* look
[01:02] <dtchen> any questions before we dive in?
[01:03] <kklimonda> Why so late? ;)
[01:03] <dtchen> (no idea! the schedule rotation deems it so)
[01:03] <dtchen> ok, and with that resounding silence, let's get started.
[01:04] <kklimonda> dtchen: wait - how to configure piuparts?
[01:04] <kklimonda> do i even have to?
[01:04] <eythian> hopefully this VM gets installed before you're too far through.
[01:04] <kklimonda> :)
[01:04] <dtchen> piuparts(8) is a tool for providing a somewhat systematic method for testing whether a binary package can be installed and removed, and/or any of the intermediate upgrade steps
[01:05] <Pollywog> we are installing a vm?
[01:05] <dtchen> kklimonda: as long as you have piuparts installed, that will suffice
[01:05] <eythian> Pollywog: My desktop is intrepid
[01:05] <Pollywog> ic
[01:05] <dtchen> eythian: / Pollywog: you don't necessarily *need* a VM, but it's quite useful for actual usage testing
[01:06] <dtchen> (ok, back to topic)
[01:06] <dtchen> some (many?) people are familiar with building binary packages from source packages on their local systems
[01:07] <dtchen> pbuilder is but one such utility - granted, it's a very nice utility
[01:07] <dtchen> briefly, one would do something akin to: # pbuilder build foo.dsc
[01:08] <dtchen> if everything "goes well", one ends up with ../*.deb (and the other parts of the source package)
[01:08] <dtchen> people often then use dpkg -i (or gdebi or ...) to install ../*.deb
[01:09] <dtchen> however, pbuilder has one fairly glaring omission in the maintenance sense: it doesn't actually test-install/remove/etc. the generated binary packages
[01:09] <kklimonda> dtchen: is it possible to install packages in pbuilder chroot without too much work?
[01:09] <nhandler> Yes kklimonda
[01:09] <dtchen> kklimonda: absolutely, and i'll touch on that briefly, too, with something called pbuilder hooks
[01:10] <dtchen> since kklimonda provided such a nice segue...
[01:10] <dtchen> pbuilder has an extension mechanism known as hooks. take a look in /usr/share/doc/pbuilder/examples/
[01:11] <dtchen> now, piuparts and pbuilder hooks for executing an install/upgrade/removal test are nominally similar
[01:11] <dtchen> where piuparts really differs is the ability to test install/upgrade/removal for an entire archive
[01:11] <dtchen> that's a bit much for pbuilder (although it's certainly possible with some additional scripting)
[01:12] <dtchen> (so, next time you encounter liw online, give him a virtual beer)
[01:12] <dtchen> does anyone have any questions regarding what piuparts is used for?
[01:13] <kklimonda> no
[01:13] <dtchen> ok, now let's look at the functional testing of binary packages
[01:14] <dtchen> piuparts and pbuilder (with hooks) are useful for fairly basic testing, but they don't actually test whether your pbuilt/sbuilt package actually starts, throws pretty screen bling up, crackles/hisses/pops, etc.
[01:14] <dtchen> (err, oops, was that pulseaudio?)
[01:14] <kklimonda> :)
[01:15] <dtchen> to get a sense of whether your package works, you really should use a VM. at the very least, you can install in the pbuilder chroot and invoke the program(s).
[01:15] <dtchen> now that i've set the stage, let's look at the two munged source packages i've placed at http://kernel.ubuntu.com/~dtchen/package-training-2009.04.23/
[01:16] <dtchen> the first is oss-compat, and we'll use that to demonstrate piuparts
[01:16] <nhandler> Should we download the packages?
[01:16] <dtchen> yes, we should have the munged source packages downloaded
[01:17] <kklimonda> done
[01:17] <dtchen> for people who preferred `apt-get source oss-compat', i'll pastebin the debdiff
[01:17] <dtchen> (well, the relevant portion)
[01:18] <nhandler> For everyone else, you can use dget to download the source package
[01:18] <dtchen> http://paste.ubuntu.com/156227/
[01:19] <dtchen> so the basic idea here is that as of jaunty, module-init-tools requires files to have the ".conf" suffix/extension
[01:19] <dtchen> this modification is a fairly straightforward one: we look at the source package and modify debian/postinst
[01:20] <dtchen> (then adjust debian/control, add/modify the changelog entry, etc.)
[01:20] <dtchen> now, when that package is pbuilt using pbuilder [without hooks], the deb will be generated correctly
[01:21] <dtchen> this case is an excellent illustration of where the install pbuilder hook and piuparts shine
[01:21] <dtchen> because the actual error lies in the postinst, pbuilder [without hooks] won't throw any errors
[01:21] <dtchen> so let's actually use piuparts ;-)
[01:22] <dtchen> the syntax is fairly basic:
[01:22] <dtchen> sudo piuparts foo.deb
[01:23] <dtchen> on my system, because i use apt-cacher-ng, the syntax is bit more convoluted:
[01:23] <dtchen> sudo piuparts -m "http://localhost:3142/ubuntu" --skip-minimize -- foo.deb
[01:24] <kklimonda> dtchen: i get " E: Couldn't find package debfoster"
[01:25] <kklimonda> I've added --skip-minimize and it worked
[01:25] <dtchen> if you're using jaunty, you'll probably need the --skip-minimize
[01:25] <dtchen> now, piuparts will create a minimal chroot if you use the above command, but most of you likely have a pbuilder base.tgz (or jaunty.tgz or whatnot) already
[01:26] <dtchen> you can instruct piuparts to use the existing pbuilder base.tgz by passing -p
[01:26] <dtchen> (so, as piuparts runs...)
[01:26] <kklimonda> dtchen: -p ? not -b ?
[01:27] <dtchen> kklimonda: -b is more flexible
[01:27] <kklimonda> i see - it's a shortcut
[01:27] <kklimonda> :)
[01:27] <kklimonda> dtchen: but still when i use my pbuilder base.tgz I get "E: Couldn't find package debfoster" error
[01:28] <dtchen> kklimonda: are you using jaunty's piuparts?
[01:28] <kklimonda> dtchen: yes
[01:28] <kklimonda> 0.35ubuntu1
[01:28] <dtchen> let's troubleshoot that offline
[01:29] <Pollywog> so piuparts is different for different releases?
[01:29] <juanje1> kklimonda: adding --keep-sources-list option probably it works
[01:29] <kklimonda> dtchen: appending --skip-minimize (as adviced in bug #317721) helps
[01:29] <kklimonda> oh, no ubottu here :)
[01:29] <dtchen> Pollywog: ideally, no - however, there is a difference between the Debian and Ubuntu components and priorities
[01:30] <kklimonda> juanje1: yes - it also worked
[01:30] <dtchen> the Ubuntu modification in jaunty helps work around it, but it's imperfect)(
[01:30] <dtchen> (hmm, Home key fail)
[01:31] <asomething> dtchen: -b is used to specify a specific base tarball, right? I've got pbuilder set up for jaunty, intrepid, and sid
[01:31] <dtchen> asomething: correct
[01:31] <dtchen> now, piuparts's output is fairly verbose
[01:32] <dtchen> the parts we're interested in lie toward the end of the output
[01:32] <dtchen> namely, PASS/FAIL/ERROR
[01:33] <dtchen> note the:
[01:33] <dtchen> ERROR: Broken symlinks:
[01:33] <dtchen>   /etc/modprobe.d/oss-compat.conf -> /lib/oss-compat/linu
[01:33] <dtchen> (that's deftly illustrated at http://paste.ubuntu.com/156227/)
[01:34] <dtchen> the process of correcting the error as a maintainer is fairly intuitive
[01:34] <dtchen> we readd the omitted character, regenerate the source package, rebuild it, rerun piuparts on it. (rinse, lather, repeat)
[01:35] <dtchen> any questions thus far?
[01:36] <dtchen> ok, so we've quickly demonstrated how piuparts can assist in catching maintainer script errors
[01:37] <dtchen> now, as maintainers/developers, we're supposed to be testing all our changes
[01:37] <dtchen> due to whatever excuses, many of us don't ;-)
[01:37] <dtchen> functional testing, however, is quite important
[01:38] <dtchen> really, changes must be tested in a VM for both correctness and efficiency
[01:38] <dtchen> the second example munged source package in my kernel.ubuntu.com space is pavucontrol
[01:39] <dtchen> the change that actually makes jaunty's pavucontrol binary package usable is introduced in debian/patches/0002-Handle-noninitialised-data.patch
[01:40] <dtchen> several iterations of that patch were generated before i discovered that upstream's fix wasn't entirely applicable to our source package
[01:40] <dtchen> i.e., sometimes it's not as straightforward as "just applying upstream fixes"
[01:41] <dtchen> briefly, jaunty ships with an older pulseaudio source package with many, many backported alsa fixes
[01:41] <dtchen> these backported fixes differ from the current upstream source package, and therefore there are several assumptions in upstream's pavucontrol fix that aren't applicable to Ubuntu
[01:42] <kklimonda> mhm
[01:42] <kklimonda> so we should use "test before release" as our mantra ? :)
[01:44] <dtchen> for our purposes, however, we're only going to illustrate that using a VM to test changes is the most efficient manner of demonstrating that a bug was fixed
[01:44] <dtchen> (that's bug 359274)
[01:44] <dtchen> if one were to take my munged source package, it is functionally equivalent to jaunty's superceded 0.9.7-1ubuntu1
[01:44] <dtchen> that version crashes as soon as a user attempts to mute System Sounds in the Playback tab
[01:44] <dtchen> thus, testing in a VM (or a chroot or whatnot) is the simplest way to demonstrate the efficacy of a fix
[01:44] <dtchen> absolutely
[01:45] <dtchen> since many people don't have VMs configured, i won't insist that everyone duplicate the crash right this moment, but run through it on your own time, and remember that development really needs this constant testing
[01:46] <dtchen> anecdote: if i had consistently used piuparts on my alsa-* and pulseaudio* source uploads for jaunty, i probably would have eliminated a half-dozen uploads
[01:46] <dtchen> so, what do we take away from this session? using piuparts is good. testing in a vm after using piuparts is good.
[01:47] <dtchen> i'll open it up for general questions now.
[01:47] <Pollywog> are there some Ubuntu-specific tutorials?
[01:47] <Pollywog> for piuparts
[01:48] <dtchen> no, but feel free to pitch in :-)
[01:48] <dtchen> (this session was just an intro to piuparts)
[01:48] <kklimonda> dtchen: i have a minor problem with pbuilder but it's unrelated to this class.
[01:48] <dtchen> kklimonda: right, we can discuss that offline in -motu after this session
[01:48] <kklimonda> ok
[01:49] <dtchen> Pollywog: do you have specific questions on using piuparts in a particular context?
[01:49] <kklimonda> dtchen: thanks for piuparts introduction - i was going to read about it some time ago and then forgot how it's called :)
[01:49] <nhandler> dtchen: Is there a hook for pbuilder to have it run the deb through piuparts?
[01:50] <Pollywog> no, I am really confused because I have not used piuparts
[01:50] <dtchen> nhandler: no, but there's some duplication in /usr/share/doc/pbuilder/examples/B91dpkg-i
[01:50] <Pollywog> I have played with pbuilder a few times
[01:50] <Pollywog> but in Debian
[01:51] <dtchen> Pollywog: ok. very briefly, one uses pbuilder to generate a set of binary package(s) from a source package
[01:51] <dtchen> Pollywog: however, pbuilder by default does not actually test the installation/upgrade/removal of said generated binary package(s)
[01:52] <dtchen> Pollywog: it can be particularly troublesome if errors lie in the maintainer scripts
[01:52] <dtchen> (since we only encounter those at package install/upgrade/removal)
[01:54] <dtchen> Pollywog: try this: sudo piuparts --skip-minimize hello
[01:54] <dtchen> juanje1: i have not tried using jaunty's piuparts on hardy, so i don't know offhand if it's feasible
[01:54] <dtchen> juanje1: one thing to be aware of is the python2.6 transition
[01:55] <dtchen> Pollywog: err, i think i omitted an -a
[01:56] <Pollywog> I will need to get the sources for hello
[01:57] <Pollywog> oh something happened
[01:57] <dtchen> Pollywog: actually, you don't have to retrieve the sources
[01:57] <Pollywog> yes I see that now
[01:57] <dtchen> Pollywog: -a instructs piuparts to retrieve from the repository using apt-get
[01:57] <Pollywog> I added the -a
[01:58] <dtchen> juanje1: i generally filter the output in some other process
[01:58] <dtchen> juanje1: the verboseness of piuparts can be cumbersome
[01:59] <Pollywog> is there a way to adjust the verbosity?
[01:59] <dtchen> Pollywog: not quite
[01:59] <Pollywog> k
[02:00] <dtchen> any other questions before we wrap up?
[02:00] <Pollywog> none here
[02:00] <Pollywog> dtchen: thanks for the class
[02:00] <dtchen> no prob, thanks all!
[02:00] <juanje1> dtchen: yep, thanks :-)
[02:01] <asomething> dtchen: thanks!
[02:02] <dtchen> so, next week is dholbach with "Getting Started with Ubuntu Development"
[02:02] <dtchen> it will take place on 30 April 2009, 06:00 UTC
[02:02] <dtchen> hope to see you here!
[23:06] <HymnToLife> guess I'll do a Photoshop break in an hour
[23:06] <macman> this chan has always been here ?
[23:07] <HymnToLife> yes
[23:07] <HymnToLife> Thu 23 Apr 2009 22:07:27 UTC
[23:07] <HymnToLife> bodhi_zazen: surely you mean in TWO hours ;)
[23:08] <kklimonda> what is this class going to be about?
[23:08] <HymnToLife> /topic
[23:09] <kklimonda> SSH?
[23:09] <bodhi_zazen> ssh
[23:10] <bodhi_zazen> two hours, lol
[23:10] <bodhi_zazen> ah, yes :)
[23:10] <bodhi_zazen> sorry
[23:10] <kklimonda> what exactly? ssh basics - rsa keys etc. ?
[23:11] <bodhi_zazen> kklimonda: what ever people like
[23:12] <bodhi_zazen> I would like these sessions to be Q&A more then lectures
[23:12] <bodhi_zazen> and I have a shared ssh session set up so can demo commands in a shared session
[23:12] <kklimonda> mhm
[23:13] <bodhi_zazen> >:)
[23:13] <bodhi_zazen> http://paste.ubuntu.com/156798/
[23:13] <bodhi_zazen> you will not be able to log in as of yet though
[23:15] <HymnToLife> >O
[23:15] <HymnToLife> a lecture on SSH would be easy
[23:16] <HymnToLife> 1. Get "SSH: the definitive guite" from O'REilly
[23:16] <HymnToLife> 2. Read it
[23:16] <HymnToLife> 3. ???
[23:16] <HymnToLife> 4. PROFIT!
[23:18] <cprofitt> what?
[23:18] <HymnToLife> lol
[23:19] <HymnToLife> you, sir, just won an internets
[23:23] <HymnToLife> :<