=== txwikinger2 is now known as txwikinger
=== ajmitch_ is now known as ajmitch
[06:56] <maxpaguru> salve here i am
[07:06] <RockyRoad> !seen mvo
[07:06] <ubot2> I have no seen command
[07:13] <dholbach> <dholbach> hello everybody
[07:13] <dholbach>  I just tried calling mvo, to no avail... I hope he's OK still
[07:13] <dholbach>  shall we make this an impromptu Q&A session about Ubuntu Development and Packaging instead?
[07:13] <dholbach>  who's here for the session?
[07:14] <dholbach> don't be shy... raise your hands! :-))
[07:14] <juanje> me! :-)
[07:14]  * micahg is here :)
[07:14]  * juanje hugs dholbach ;-)
[07:15] <ara> o/
[07:15]  * dholbach hugs y'all back :)
[07:16] <mbudde> o/
[07:16] <dholbach> do you have any questions about ubuntu development, about packaging, maybe even a specific problem that you're dealing with at the moment... some general thoughts about how ubuntu development works? anything we could help out with?
[07:16] <maxpaguru> °/
[07:17]  * juanje thinking about....
[07:18] <micahg> I had a build fail on me
[07:18] <micahg> but I think it was because a patch was out of date
[07:18] <dholbach> micahg: do you have a build log somewhere?
[07:18] <micahg> http://launchpadlibrarian.net/27434400/buildlog_ubuntu-jaunty-i386.acidbase_1.4.3.1-0ubuntu1~ppa1_FAILEDTOBUILD.txt.gz
[07:18] <dholbach> let's see
[07:19] <dholbach> yes
[07:19] <dholbach> dpatch  apply-all
[07:19] <dholbach> applying patch 01_default_config to ./ ... ok.
[07:19] <dholbach> applying patch 02_update_external_links to ./ ... failed.
[07:19] <micahg> I didn't check the patches before uploading
[07:19] <dholbach> that's because a patch is out of date
[07:19] <micahg> ok
[07:19] <micahg> I'll look into it another time
[07:19] <dholbach> does everybody know what dpatch is and how it works?
[07:20] <maxpaguru> not so much ;)
[07:20] <dholbach> great :)
[07:20] <juanje> a bit
[07:21] <dholbach> so dpatch is one of the patch systems we use for packaging
[07:21] <dholbach> because we don't use distributed development with a clear history of changes for everything yet
[07:21] <dholbach> it sometimes makes sense to separate out patches
[07:21] <dholbach> so let's say you're packaging a huge piece of upstream software and the software authors do one release every year
[07:22] <dholbach> in that case you might come into a situation where you need to make changes their code or backport fixes
[07:23] <dholbach> it can get very hard to stay on top of what's happening and who changed what if you just apply all changes directly on the source
[07:23] <dholbach> that's why some package maintainers decide to put separate patches (one for every issue) into the debian/patches directory
[07:23] <dholbach> acidbase, as micahg mentioned above, makes use of dpatch
[07:24] <dholbach> 01_default_config seems to make some changes to the default configuration, and applies fine
[07:24] <dholbach> 02_update_external_links unfortunately doesn't
[07:24] <dholbach> the way that dpatch works is quite simple
[07:24] <dholbach> you basically just run:
[07:24] <dholbach>    dpatch-edit-patch 03-new-patch-that-fixes-bug-1234567
[07:25] <dholbach> and it will open a new shell environment, where you can make all kind of changes
[07:25] <dholbach> when you hit Ctrl-D to close the shell, it will create a new file called debian/patches/03-new-patch-that-fixes-bug-1234567.patch
[07:25] <dholbach> that includes the diff of what you just did
[07:25] <dholbach> then you add 03-new-patch-that-fixes-bug-1234567 to debian/patches/00list (for internal housekeeping) and it will automatically get applied during the build
[07:26] <dholbach> does that make sense?
[07:26] <mbudde> Yes it does :)
[07:26] <dholbach> https://wiki.ubuntu.com/PackagingGuide/PatchSystems#dpatch maybe makes it a bit clearer
[07:26] <RockyRoad> nice tool :)
[07:26] <dholbach> (and has a quick example)
[07:26] <maxpaguru> yes indeed!
[07:27] <dholbach> dpatch apply-all / dpatch unapply-all are useful commands too
[07:27] <dholbach> I hope one day we can get rid of all patch systems when we have full source imports in bzr though and keep track of all the history in there
[07:27] <mbudde> Is there a preferred patch system (dpatch, quilt, ...) or does that depend on the package?
[07:28] <dholbach> mbudde: I think it depends on what the package maintainer likes best
[07:28] <dholbach> quilt seems to be the new black right now, but I really never grew to like it :)
[07:29] <micahg> wow
[07:29] <micahg> dpatch-edit-patch is cool
[07:29] <micahg> it does it all for you :)
[07:29] <dholbach> as nice as patch systems are for housekeeping (backport fix to version A, delete patch file, when version A+1 gets out), a bzr merge <upstream branch> is much more obvious :-)
[07:30] <dholbach> I guess we should have James Westby here as a speaker again, he's the man when it comes to distributed development :)
[07:30] <dholbach> do we have any other questions?
[07:31] <mbudde> I have a problem with a package I'm updating.
[07:31] <micahg> is there a way to set a default key for signing with debuild?
[07:31] <dholbach> micahg: yes, hang on
[07:31] <mbudde> http://launchpadlibrarian.net/27276693/buildlog_ubuntu-jaunty-i386.purple-plugin-pack_2.5.1-0ubuntu1~ppa3~jaunty_FULLYBUILT.txt.gz
[07:31] <dholbach> something like this in your .bashrc should do the trick:
[07:31] <dholbach> export DEBFULLNAME='Daniel Holbach'
[07:31] <dholbach> export DEBEMAIL='daniel.holbach@ubuntu.com'
[07:32] <dholbach> (and make sure that email address is a key id on your gpg key)
[07:32] <micahg> dholbach: I have 2 keys for the one address
[07:32] <micahg> don't ask...
[07:32] <mbudde> I get a whole lot of dpkg-shlibdeps warnings, is there any way to stop that?
[07:32] <mbudde> Or is it a problem at all?
[07:32] <dholbach> micahg: I think you can specify the key id somewhere, but I'd need to look it up
[07:33] <micahg> ok
[07:33] <micahg> I know I can do it on the command line
[07:34] <dholbach> mbudde: that looks more like an upstream problem to me - it might make sense to have a chat with the upstream authors about it
[07:34] <mbudde> Ok :)
[07:34] <dholbach> mbudde: but those warnings are not critical at all
[07:35] <dholbach> there might be something wrong with the linking (sorry, I'm no expert there)
[07:35] <dholbach> do we have any more questions... maybe more general questions?
[07:36] <dholbach> mbudde, micahg: good work - I'm happy you're working on some packaging stuff already
[07:37] <mbudde> It's fun! :)
[07:37] <dholbach> is there anything you'd like me to talk about a bit?
[07:37] <mbudde> What about getting stuff back into debian?
[07:37] <maxpaguru> i'd like to know how to use bugs in LP
[07:38] <dholbach> mbudde: check out https://wiki.ubuntu.com/Debian/Bugs (there's the 'submittodebian' tool in ubuntu-dev-tools which makes forwarding patches really easy)
[07:39] <micahg> I just started :)
[07:39] <dholbach> mbudde: now is a really good time to forward stuff to debian: we're merging lots and lots of packages right now and if you see a delta that could as well go to debian, forward it
[07:39] <dholbach> maxpaguru: can you elaborate? what kind of bugs?
[07:40] <dholbach> or is there anything in particular that you're wondering about?
=== MaWaLe is now known as tunisian
=== tunisian is now known as MaWaLe
[07:41] <maxpaguru> how to handle bugs in general , from where to begin , i'm a very newby;)
[07:41] <dholbach> ok
[07:41] <dholbach> so let's say you found a simple bug that you want to work on and fix it
=== ziroday is now known as zeroday
[07:42] <dholbach> you'd go and download the current source package by running   apt-get source package-you-want-to-fix
[07:42] <dholbach> then you'd make the simple change in the source tree
[07:42] <dholbach> then you'd document it, by running      dch -i
[07:42] <dholbach> you'd add a new changelog entry, properly explain what you did to fix it
[07:43] <dholbach> and add    (LP: #1234567)   to the changelog entry
[07:43] <dholbach> with the bug number of the bug you're working on
[07:43] <dholbach> (ah... before that you'd probably assign the bug to yourself and set the status to 'in progress')
[07:44] <micahg> indeed
[07:44] <dholbach> then you'd run        debuild -S      to regenerate the source package
[07:44] <micahg> dholbach: why not make it a patch?
[07:44] <dholbach> micahg: I'll get to that in a bit
[07:44] <micahg> ok
[07:44] <micahg> :)
[07:44] <dholbach> then you'd test-build the package (you could use pbuilder for that: https://wiki.ubuntu.com/PbuilderHowto)
[07:44] <MaWaLe> sorry for interferring : where can i find the log of the classroom (i missed it :( )
[07:45] <dholbach> MaWaLe: https://irclogs.ubuntu.com - it gets updated every now and then
[07:45] <MaWaLe> thx dholbach :)
[07:45] <dholbach> also I'll put it up later at https://wiki.ubuntu.com/Packaging/Training/Logs
[07:45] <micahg> approx every hour
[07:45] <dholbach> then you'd test the resulting package, make sure the bug is really fixed
[07:46] <maxpaguru> dholbach: Thanks where the tutorial links to that stuff . are there?
[07:46] <dholbach> then you'd run
[07:46] <dholbach>    debdiff package-you-want-to-fix_old-version.dsc package-you-want-to-fix_new-version.dsc > package-you-want-to-fix.debdiff
[07:47] <dholbach> this would generate the full patch (in package-you-want-to-fix.debdiff), that you could attach to the bug report
[07:47] <dholbach> then you'd subscribe ubuntu-main-sponsors (if the package is in main/restricted) or ubuntu-universe-sponsors (if it's in universe/multiverse)
[07:47] <dholbach> and somebody will take care of reviewing it and uploading it for you
[07:48] <dholbach> the nice thing about "(LP: #1234567)" is that the bug will automatically gets closed, when the package is accepted
[07:48] <dholbach> the docs for that are available at: https://wiki.ubuntu.com/PackagingGuide/Recipes/Debdiff
[07:48] <dholbach> AndrewGe1: https://wiki.ubuntu.com/SponsorshipProcess
[07:49] <dholbach> sorry AndrewGe1 - automatic tab completion
[07:49] <dholbach> maxpaguru: did that make sense so far?
[07:49] <maxpaguru> dholbach: great it's much clearer!
[07:49] <dholbach> awesome!
[07:50] <dholbach> any more questions? :)
[07:50] <juanje> dholbach: so, the better way to fix a bug is with a debdiff? I missed the UDS' session about patches, debdiff, branches and I was wondering which one would be the better way. I usualy try to attach a bzr branch with the fixes so the maintainer can merge the changes
[07:50] <juanje> but I don't know if this is the better way
[07:51] <dholbach> juanje: if the package is in bzr already, the branch is probably the better option
[07:51] <juanje> dholbach: ok
[07:51] <maxpaguru> dholbach: I understand , it's clear i have to follow these steps one by one in detail. tnx so much
[07:52] <juanje> dholbach: and how the changelog should be changed in a debdiff?
[07:52] <dholbach> juanje: in that case, you could do something like:    bzr push --fixes lp:1234567 lp:~juanje/package-you-want-to-fix/my-fix
[07:52] <juanje> dholbach: yeah, make sense, thanks :-)
[07:52] <dholbach> juanje: if you ran   dch -i    before, you added an entry in the changelog, which will turn up in the debdiff file
[07:53] <maxpaguru> where to find bzr tutorial?
[07:53] <dholbach> hang on
[07:53] <juanje> dholbach: yes, but I mean if you write it like you were writting your onw package or as other people (the maintainer) will change it after?
[07:54] <dholbach> maxpaguru: http://doc.bazaar-vcs.org/bzr.dev/en/mini-tutorial/index.html
[07:54] <dholbach> for general bzr use
[07:55] <dholbach> and https://wiki.ubuntu.com/DistributedDevelopment/Documentation for use in the packaging / ubuntu development context
[07:55] <maxpaguru> ok ;)
[07:56] <dholbach> juanje: I'm not quite sure I understand - can you give me an example?
[07:56] <juanje> maxpaguru: this could be useful as well: https://wiki.ubuntu.com/BzrMaintainerHowto
[07:56] <juanje> dholbach: yeah, sorry, I didn't explain myself well....
[07:57] <juanje> dholbach: what I mean is that when you fix some package
[07:57] <juanje> and it's not yours
[07:57] <juanje> do you have to "propose" a changelog?
[07:57] <maxpaguru> juanje: thanks I'll follow all those links!
[07:58] <juanje> or write the changelog as the final one
[07:58] <juanje> ?
[07:58] <dholbach> we don't have the concept of "package maintainer" in Ubuntu
[07:58] <dholbach> so we all maintain all packages as a team
[07:58] <juanje> ummmmm
[07:58] <juanje> ok
[07:58] <juanje> :-)
[07:58] <dholbach> so it definitely helps if you add a changelog entry
[07:58] <juanje> dholbach: but, if you are not MOTU or so?
[07:59] <dholbach> juanje: then you still add a changelog entry and put up the full debdiff (or branch) for review through sponsorship
[07:59] <juanje> ok
[07:59] <juanje> understood :-)
[07:59] <dholbach> perfect
[07:59] <dholbach> it's important that you put a lot of effort into documenting your changes
[07:59] <dholbach> I usually write changelog entries like this:
[08:00] <maxpaguru> Bye everyone I'll see the log later :-)
[08:01] <dholbach>   * src/something.c, src/something.h: remove all occurrences of "frobnicator", changes taken from upstream SVN commit 1542, fixes the build again (LP: #638153)
[08:01] <dholbach> that way people will see: which files you changed, what you changed, why you changed it and that the upstream developers did the same thing, plus you close the current Ubuntu bug
[08:02] <dholbach> it's important we do that, because as I said above: we all maintain packages as a team
[08:02] <juanje> dholbach: I usually write the changes on the bzr log and when I make changelog entry in it, but I had some doubts about if then the maintainer had to rewrite it (also maybe changing the version or so). Actually my doubts were more about the version you put to the changelog entry
[08:02] <dholbach> so the next uploader of the package doesn't have to guess why you did a certain change
[08:02] <dholbach> ... or you won't have to guess your intentions half a year later :)
[08:02] <juanje> dholbach: agree :-)
[08:02] <dholbach> juanje: usually dch -i should do the right thing
[08:02] <juanje> it's very important :-)
[08:02] <juanje> ok
[08:03] <dholbach> juanje: if you add a new changelog entry and run debcommit afterwards, it will use that changelog entry for the bzr commit message :)
[08:03] <dholbach> ...two bird with one stone...
[08:04] <juanje> ummmm
[08:04] <juanje> I swa something about debcommit, but I haven't try yet
[08:04] <dholbach> juanje: you have a question?
[08:04] <dholbach> it's good stuff :)
[08:04] <juanje> I think, I'll try :-)
[08:05] <dholbach> rock on
[08:05] <dholbach> ok... thanks everybody! if you have any more questions, please feel free to join #ubuntu-motu and ask questions there
[08:05] <dholbach> we're happy to help you out!
[08:05] <juanje> dholbach: thanks for all!
[08:05] <juanje> :-)
[08:05] <dholbach> mvo's session will be rescheduled - I'll let you all know about the session in a bit
[08:06] <dholbach> thanks everyone and have a great day!
[08:06] <RockyRoad> thx dholbach :)
[08:06] <mbudde> dholbach, thank you!
[08:07] <dholbach> my pleasure :)
[08:07] <maxpaguru> thanks to dholbach and everyone! :-)
[08:09] <ara> mvo session will be today at 15:00UTC
[08:09] <ara> more details on the planet today :)
[08:09] <juanje> mbudde: do you have some where the code from the package you got the fail?
[08:09] <juanje> ara: thanks :-)
[08:10] <juanje> mbudde: I have some ideas about the warnings I like to confirm. I you don't mind
[08:11] <mbudde> juanje, great! :) You can get the package from my PPA: https://launchpad.net/~mbudde/+archive/ppa
[08:12] <juanje> mbudde: thanks :-) I tell you latter ;-)
[08:13] <mbudde> juanje, if I'm not online please do send me an email
[08:15] <juanje> mbudde: sure ;-)
[08:15] <ricecom> too late.. :(
[08:50] <ricecom> bye
=== zeroday is now known as ziroday
[09:39] <Metralha> This session was too early for me...:(
[10:35]  * MaWaLe is away: brb
[10:37]  * MaWaLe is back (gone 00:01:34)
=== pipedrea1 is now known as pipedream
[12:08] <ia> hello. could you tell me, please - the next classroom about packaging will be in 6:00 UTC pm? or it has been already at 6:00 am?
=== nhandler changed the topic of #ubuntu-classroom to: Ubuntu Classroom || https://wiki.ubuntu.com/Classroom || https://lists.ubuntu.com/mailman/listinfo/ubuntu-classroom || Upcoming: 4 June, 15:00 UTC: Make Your Package Upgrade Correctly; 11 June, 12:00 UTC: Java library packaging || Run 'date -u' in a terminal to find out the UTC time
[12:14] <nhandler> ia: The next packaging training session is at 15:00 UTC today due to a scheduling conflict
=== ejat is now known as e-jat
=== yofel_ is now known as yofel
[15:59] <mvo_> hello!
=== mvo_ is now known as mvo
[15:59] <mvo> welcome everyone - and sorry that I could not make it this morning
[16:00] <mvo> and a special thanks to dholbach for doing a Q&A
[16:00] <dholbach> mvo: no worries :)
[16:01]  * iulian m00s
[16:01] <mvo> I'm going to talk about how to make package upgrade cleanly
[16:01] <mvo> I will give a overview on the common problems I see when looking at upgrade failure logs
[16:02] <toabctl> hi
[16:02] <mvo> on the bright side: dpkg/apt tend to do pretty well - but if stuff goes wrong recoverying is not a great user experience
[16:02] <mvo> so we should try to make sure that stuff does not get wrong :)
[16:02] <keffie_jayx> o/
[16:02] <mvo> (and improve our tools to deal better with failures)
[16:03] <mvo> if you have any questions or special topics that you would like to talk about, just ask here (or in #ubuntu-classroom-chat)
[16:04] <Rinia> hello everybody
[16:04] <mvo> I prepared some notes, if there are no questions/comments I will just paste a bit into the channel
[16:04] <keffie_jayx> :D
[16:05] <mvo> = How to make packages upgrade cleanly =
[16:05] <mvo> Normal deb packages usually upgrade just fine, dpkg takes care of all the heavy lifting and shuffling of files. The most common source of problem on upgrades are:
[16:05] <mvo> • file overwrite problems (e.g. when a package was split)
[16:05] <mvo> • maintainer script problems
[16:05] <mvo> A general good advise is to keep the amount of code in the maintainer script small and clean. Shell is a language that can be tricky and a failure in the script will make the upgrade abort and gives the user a bad experience.
[16:07] <mvo> the most common source of overwrite problems are caused by package splits
[16:08] <dholbach> mvo: why are package split? does that happen very often? is ther an example? :-)
[16:08] <mvo> but they can also happen if two package just happen to install the same file into the same location. if that can not be avoided (e.g. by changeing the program to look into a different location) the package can not be installed in parallel and should conflict
[16:08] <mvo> for a package split we use a different technique
[16:08] <mvo> dholbach: thanks, I give a example now :)
[16:08] <mvo> == Package splits ==
[16:08] <mvo> If a package foo gets split into two packages like foo and foo-common there needs to be a "Replaces" line in the "foo-common" package. The reason is that on upgrade, the new package "foo-common" may get installed when the old "foo" is still installed. Because the foo-common has files that belong to the old foo dpkg will complain if the proper "Replaces" line is missing.
[16:08] <dholbach> woohoo
[16:09] <mvo> this is pretty common for projects that start out as e.g. update-manager and then later grow new frontends (like udpate-manager-text). then the package typically gets split into update-manager-common, update-manager-gtk, update-manager-text etc
[16:10] <mvo> Example:
[16:10] <mvo>  foo 1.0 is a big package
[16:10] <mvo>  foo 1.1 is split into foo and foo-common
[16:10] <mvo> foo-common now needs a line:
[16:10] <mvo> Replaces: foo (<< 1.1)
[16:10] <mvo> to ensure that if the new foo-common is unpacked before the foo package is upgraded everything still works as expected.
[16:10] <neurobuntu> how does (<< 1.1) differ from (< 1.1) ?
[16:13] <mvo> neurobuntu: the "<" is a older (and now deprecated way) of expressing the same. it used to mean earlier-or-equal
[16:13] <mvo> neurobuntu: but its a bit confusing, this is why "<<" and "<=" etc are now used
[16:13] <mvo> The most common case is something like the following for "foo":
[16:13] <mvo> Depends: foo-data (= ${source:Version})
[16:13] <mvo> This means that dpkg needs to unpack foo-data first before it unpacks the new foo. So without the Replaces line above it will not work.
[16:13] <neurobuntu> ok thanks
[16:14] <mvo> cheers
[16:15] <mvo> any questions on this particular bit (splitting/overwrite problems)?
[16:15] <mvo> I should note that we have a system that can check for file conflicts in the archive, but its currently in the need for some bugfixing
[16:16] <dholbach> what is a good way to test if my packages upgrade well? is  dpkg -i *.deb  good enough?
[16:17] <mvo> dholbach: yes, that is a good way. its a good way of testing before a upload to make sure the current version is installed
[16:17] <mvo> and then install the new package like this
[16:17] <mvo> the good news is that for the common case (package just got some new files/some old files got removed) dpkg will do everything
[16:17] <mvo> and we do not have to worry
[16:18] <mvo> testing is always good, its pretty anoying if a forgoten "fi" in a maintainer script causes problems later :)
[16:18] <mvo> which brings me to the next topic:
[16:18] <mvo> == Maintainer script problems ==
[16:19] <mvo> We have two classes of issues here
[16:19] <mvo> one are "just" bugs
[16:19] <mvo> a forgoten "fi"
[16:19] <mvo> the use of bash syntax (like "==")
[16:20] <mvo> instead of POSIX (which expects "=")
[16:20] <mvo> etc
[16:20] <mvo> (we use dash as /bin/sh so we need to be careful with that, some other distros do not have this problem)
[16:22] <mvo> the other class of problems is that the system is in a flux during a complex upgrade (like a upgrade from 6.06 -> 8.04 :)
[16:22] <dholbach> mvo: what are maintainer scripts? is there a good way to test them? :)
[16:22] <mvo> dholbach: good point, sorry for using jargon :)
[16:22] <dholbach> (I hope everybody else feels free to pester mvo with questions too and it's not just me. :-))
[16:23] <mvo> maintainer scripts are the scripts known as: "prerm, postrm, preinst, postinst"
[16:23] <mvo> they are run as part of the package installation
[16:23] <toabctl> mvo, are they makefiles? or just shellscripts?
[16:23] <mvo> they can contain any shell code and are used for stuff like "ldconfig"
[16:23] <mvo> toabctl: usually shell scripts, some contain perl too (notable stuff that uses debhelper)
[16:24] <neurobuntu> when upgrading a package, are the prerm and postrm run (almost like the old package is removed before the new is installed)?
[16:25] <mvo> neurobuntu: yes they are. I have a example for this in a bit, it can become relatively complex
[16:25] <mvo> a maintaner script may alter the system state in any way, this is why we do not support rollbacks or downgrades in a general way. because it may simply be not possible to undo (or know) what a maintainer script did
[16:25] <mvo> (but that is just a side note :)
[16:26] <dholbach> mvo: is there any way to test them or do I just re-install the package to see if it worked?
[16:26] <mvo> dholbach: they can be tested by running them with the approprate arguments (simulating the way they are called by dpkg). this can be very helpful, especially when run with "sh -x" to see line by line what is actually happening
[16:27] <mvo> I will give a example for this
[16:27] <mvo> Problems here are usually caused by the fact that the system is in a pretty unusual state during a upgrade. Some packages are half-installed, some are not available etc. Maintainer scripts should not make too many assumptions about the system because during a upgrade those may well be wrong. A script should gracefully whenever possible.
[16:27] <mvo> Its useful to know how maintainer scripts get called:
[16:27] <mvo> http://debid.vlsm.org/share/Debian-Doc/debian-policy/ch-maintainerscripts.html has all the details in Section "6.5".
[16:27] <mvo> The way apt works is that it splits the changes into dpkg runs that get called with "--unpack" and then "--configure". On a typical upgrade, this is:
[16:27] <mvo> Unpack:
[16:27] <mvo> •  old-prerm upgrade new-version
[16:27] <mvo> • new-preinst upgrade old-version
[16:27] <mvo> • files from the new package are now in place
[16:27] <mvo> • old-postrm upgrade new-version
[16:27] <mvo> Configure:
[16:27] <mvo> • postinst configure most-recently-configured-version
[16:28] <mvo> neurobuntu: I hope the table below anser the earlier question
[16:28] <neurobuntu> yes it does
[16:28] <mvo> so quite a bit of script that are run :)
[16:28] <mvo> and if something goes wrong, it gets even more complext, if e.g. the old-prerm fails, the new-prm is called
[16:28] <neurobuntu> what is the logic behing running the new preinst and copying the new files before running the old-postrm
[16:28] <mvo> this can be useful when recovering from a mistake in a maitainer script
[16:32] <mvo> neurobuntu: a good question. I currently do not have a good example, generally the postrm is there to do final cleanup
[16:33] <mvo> useful for e.g. update-menu or something similar that updates the debian menu and needs the new menu files to get reliable data
[16:34] <mvo> (not the best example in the world I guess :)
[16:34] <mvo> The time between unpack and configure can be pretty long (on release upgrades) because apt tries to put as much work into a single dpkg run as possible (to speed things up, dpkg invocation is slow).
[16:34] <Rail> FYI, detailed information about maintainer scripts can be found at http://www.debian.org/doc/debian-policy/ch-maintainerscripts.html
[16:36] <mvo> Rail: thanks! especially section 6.6 is interessting here
[16:36] <mvo> This can be tricky if critical functionality relies on the work that is done in configure. A example is python when the symlinks get removed in preinst and re-created in postinst. This means that the package python functionality is not available during a release upgrade (people can not play certain gnome-games ;) Its also common for daemons that stop in preinst and start again in postinst). Often this is unavoidable, but it should be kept
[16:36] <mvo>  in mind when thinking about the package maintainer scripts.
[16:37] <mvo> generally I would recommend writing small and simple maintainer scripts if at all possible, the less complexity the less likely are bugs :)
[16:38] <dholbach> mvo: is there a way to avoid maintainer scripts easily?
[16:38] <mvo> debhelper takes care of a lot of this
[16:38] <mvo> e.g. the ldconfig scripts are all auto generated
[16:39] <mvo> or the python magic
[16:39] <mvo> or init stuff
[16:39] <dholbach> or is there a way around having to write (really good) shell scripts? :)
[16:39] <mvo> and thats good, auto-generated code is nice because it needs to be fixed only in one place if a errorr is found
[16:40] <mvo> another new development is the increased use of triggers
[16:40] <neurobuntu> triggers?
[16:40] <mvo> triggers are a way to express a interessted in a certain directory for example
[16:41] <mvo> its a relatively new concept
[16:41] <neurobuntu> ok i'll be sure to read up on them
[16:41] <mvo> a package like e.g. "man-db" tells dpkg that its interessed in changes in /usr/share/man
[16:41] <mvo> and everytime a file is installed there, dpkg will notice that and mark the trigger as pending
[16:42] <mvo> then dpkg will run the pending triggers at the end
[16:42] <mvo> (or when needed)
[16:42] <mvo> so it runs the man-db.postinst with "triggered" as argument
[16:42] <mvo> and the man-db package then processes the new man-pages and builds a db from them
[16:43] <neurobuntu> so man-db says its interested in /usr/share/man I install package foo which installs /usr/shar/man/foo.man and then dpkg runs the man-db.postinst?
[16:43] <mvo> this is a very useful concept because it means that mandb can be run when everything else got installed (needs to be run only once)
[16:43] <mvo> neurobuntu: yes
[16:43] <neurobuntu> wow thats really slick!
[16:44] <mvo> neurobuntu: re "configure" (from #ubuntu-classroom-chat")
[16:45] <mvo> the configure here is something different, it just shares the same name with the autotools configure
[16:45] <mvo> dpkg roughtly splits a package into "unpacked" and "configured"
[16:45] <mvo> unpack means the files are in place
[16:45] <mvo> configured means that the package is actually usable
[16:45] <mvo> the switches are "dpkg --unpack $long_list_of_packages"
[16:46] <mvo> and dpkg --configure $long_list_of_packages"
[16:46] <mvo> in a way, this is a performance optimization to avoid having to call dpkg too often
[16:47] <mvo> each time dpkg is run it builds a (internal) database of all installed files and the relations
[16:47] <mvo> that can be time consuming
[16:47] <mvo> so the less times dpkg needs to be called, the better
[16:47] <neurobuntu> ok that makes sense...  forgive me if you answered this earlier but after the files are in place (dpkg --unpack),  preinst gets run?
[16:48] <neurobuntu> or are all the preinst scripts run before the unpack happens and then all the postinst scripts are run?
[16:49] <mvo> neurobuntu: the preinst script gets run before the new files are in place. this can cause bugs, we had a package that tried to start a daemon in preinst, but the daemon was not on disk yet, so the package was not installable (or rather, installed, but always gave a error message)
[16:49] <mvo> ia: thanks for your question - this is a tricky one
[16:50] <neurobuntu> ok thanks
[16:50] <neurobuntu> this is making sense
[16:50] <mvo> ia: the python problem we had were hard to reproduce because the system is in so much of a flux during a upgrade. its tricky to get this right. in my experience jaunty-final did pretty well and the cases of failure were rare
[16:50] <mvo> but its a good example why complexity in maintainer scripts is generally something that may not work so well
[16:51] <mvo> because its hard to predict the environment in which the script is run, that may depend on the installed packages on the system that is upgraded etc
[16:51] <mvo> very hard to reproduce
[16:52] <mvo> what we can  do now is upgrade-testing on real HW
[16:52] <mvo> there is a update-manager --sandbox switch
[16:52] <mvo> for jaunty->karmic upgrades
[16:53] <mvo> that will perform a upgrade into a writeable filesystem snapshot in /tmp - but it keeps the real system unchanged (or rather, restores it on the next reboot)
[16:53] <mvo> this is quite useful to test real world upgrades
[16:53] <mvo> chroots (e.g. pbuilder login) or other tests are useful too
[16:54] <mvo> we plan to have a auto-upgrade-tester package for karmic that can be used to test system upgrades of any given input packages, lets see how this goes :)
[16:54] <mvo> I guess the summary is: the best way for us (as a distro) to ensure clean upgrades is lots of testing (on a wide range of setups)
[16:55] <mvo> for the maintainer: write good, clean, small maintainer scripts :)
[16:56] <ia> mvo: ok, thanks :-)
[16:56] <mvo> (for the specific python problem we plan to change the way the python packages are done to invlove a little bit less magic :)
[16:56] <mvo> for karmic
[16:57] <neurobuntu> mvo thanks this has been really helpful
[16:58] <mvo> thanks for comming and listening!
[16:58] <mvo> and asking good questions of course :)
[17:00] <mvo> ia: the auto-upgrade-tester is a project that can perform a full upgrade in a virtual environment
[17:00] <mvo> it can use chroot, kvm and ec2 (the later is still a bit experimental)
[17:00] <mvo> kvm is really the most useful one
[17:01] <mvo> you give it a base image that can be any kvm image (e..g build via ubuntu-vm-builder) and then it will perform a release upgrade from the installed version of ubuntu to the next using update-manager in a non-interactive session
[17:01] <mvo> that takes ~1h on my realatively slow system
[17:02] <mvo> after that I can see if the given image upgrades cleanly or if there are issues
[17:02] <mvo> its very useful as a QA tool
[17:02] <mvo> its currently only available via bzr, but we want to package it
[17:02] <mvo> so that everyone can test and play with it
[17:03] <mvo> the base-image is not touched, so when a failure happen a fix can be tested as well with it
[17:04] <mvo> e.g. when python-foo does not upgrade cleanly (but only in a complex dist-upgrade), a fix can be uploaded to a test PPA and the upgrade can be pointed to that PPA to test if the fix is really good
[17:04] <mvo> does that answer the question? or should I expand on a particular are more?
[17:04] <mvo> area
[17:05] <ia> mvo: yep, thanks for answer :-)
[17:05] <mvo> :)
[17:10] <mvo> if there are no further question I say thanks for comming and have a good day
[17:10] <Rail> mvo: thank you
[17:10] <Rail> and others for questions
[17:27] <Rail>  /close
=== MaWaLe is now known as MaWaLe_
=== MaWaLe_ is now known as MaWaLe__
=== MaWaLe__ is now known as MaWaLe
=== MaWaLe_ is now known as MaWaLe
=== k00011 is now known as k0001
=== bazhang_ is now known as bazhang