| bazhang | @schedule | 05:20 |
|---|---|---|
| ubottu | bazhang: Schedule for Etc/UTC: 28 Jul 14:00: Maintainer scripts | 31 Jul 14:00: Mentoring Reception | 31 Jul 16:00: Ubuntu Mobile | 02 Aug 13:00: Xubuntu Community | 03 Aug 18:00: Mozilla Team | 05 Aug 22:00: Community Council | 05:20 |
| bazhang | dmwaters, could you specify a bit more? what modules do you need to load | 05:22 |
| === dosnubbi1 is now known as dosnubbie | ||
| === croppa_ is now known as croppa | ||
| goshawk | hi | 14:54 |
| DRebellion | allo | 14:54 |
| goshawk | hi norsetto | 14:54 |
| goshawk | :) | 14:54 |
| norsetto | heya :-) | 14:54 |
| * DRebellion draws a giant stickman on the whiteboard | 14:55 | |
| DRebellion | :P | 14:55 |
| norsetto | Hi everybody, thanks for coming and welcome to this lecture! | 15:01 |
| norsetto | Who do we have here? | 15:01 |
| DRebellion | me | 15:01 |
| coolbhavi | coolbhavi, me | 15:01 |
| goshawk | me | 15:01 |
| === pleia2 changed the topic of #ubuntu-classroom to: Ubuntu Open Week is over, thanks for participating! | Information and Logs: https://wiki.ubuntu.com/UbuntuOpenWeek | Current Session: Monday 28th July at 14:00 UTC: MOTU School Session: Maintainer Scripts by Cesare Tirabassi | ||
| norsetto | hi DRebellion! | 15:01 |
| norsetto | coolbhavi, goshawk: nice to see you here | 15:01 |
| goshawk | norsetto: are you cesare tirabassi? | 15:02 |
| norsetto | nobody else? Too timid to talk? | 15:02 |
| tuxmaniac | Hi everybody | 15:02 |
| norsetto | goshawk: the very one | 15:02 |
| norsetto | hi tuxmaniac | 15:02 |
| tarvid | nah just too many of us | 15:02 |
| Xk2c | hi | 15:02 |
| RainCT | heh | 15:02 |
| ssweeny | if everyone sounded off the session would be over | 15:02 |
| NielsE | hi | 15:02 |
| Dabian | I am here too .. trying to catch up. | 15:02 |
| Dabian | The topic change confused me. | 15:03 |
| norsetto | ok, I guess we can start, make yourself confortable, take a can of your preferred drink ;-) | 15:03 |
| coolbhavi | norsetto, I ve seen you working at LP answer tracker Interested to attend this session | 15:03 |
| coolbhavi | very interested | 15:03 |
| norsetto | Dabian: what topic change? | 15:03 |
| * huats too :) | 15:04 | |
| norsetto | Dabian: look good to me :-) | 15:04 |
| pleia2 | Dabian: just updated from "Next" to "Current" :) | 15:04 |
| Dabian | The topic of the channel changed ... I'm not used to this client. | 15:04 |
| norsetto | We will talk in general about the basics of the debian package management and cover maintainer scripts in some detail | 15:04 |
| goshawk | norsetto: i'm italian too :) (your name is italian) | 15:04 |
| norsetto | We will not tackle very complex subjects, I tried to make this very informative but at the same time accessible to all newcomers, even those with little debian/ubuntu experience (sorry if this will bore everybody else) | 15:04 |
| norsetto | goshawk: yes, nobody is perfect ;-) | 15:05 |
| norsetto | If there is something you don't understand, stop me and ask, ok? | 15:05 |
| coolbhavi | ok | 15:05 |
| Dabian | OK | 15:05 |
| NielsE | ok :-) | 15:05 |
| norsetto | Anybody know what a debian binary package is (I mean, the .deb file)? | 15:05 |
| goshawk | ok | 15:05 |
| goshawk | i know | 15:05 |
| DRebellion | norsetto, a file tree essentially | 15:05 |
| NielsE | yes | 15:06 |
| norsetto | yep, its just an archive file | 15:06 |
| norsetto | You can download a deb, for instance ucf, hardy version: wget http://archive.ubuntu.com/ubuntu/pool/main/u/ucf/ucf_3.005_all.deb | 15:06 |
| norsetto | wget http://archive.ubuntu.com/ubuntu/pool/main/u/ucf/ucf_3.005_all.deb | 15:06 |
| norsetto | ok, everybody got it? We will use this quite often during this lecture. | 15:07 |
| NielsE | got it | 15:07 |
| Xk2c | yep | 15:07 |
| DRebellion | yar | 15:07 |
| norsetto | Now check it with: ar -t ucf_3.005_all.deb | 15:07 |
| Dabian | Well .. I don't know the structure of it, but I guess its a compressed directory structure of some kind. | 15:07 |
| norsetto | What do you see? | 15:07 |
| bhavi_ | norsetto, question: can we update a package maintained by some one else? | 15:07 |
| norsetto | bhavi_: yes, of course you can, but is always better to first ask | 15:08 |
| Xk2c | debian-binary control.tar.gz data.tar.gz | 15:08 |
| norsetto | debian-binary is simply an ascii file containing the version of the debian format (now 2.0) | 15:08 |
| Dabian | I got it now .. about to check it. | 15:08 |
| norsetto | data.tar.gz contains the files to be installed | 15:09 |
| Kopfgeldjaeger2 | Three files, one containing the control file and sometimes {pre,post}{rm,inst} and the other one containing the application itself (binary files, man pages, scripts) | 15:09 |
| norsetto | control.tar.gz contains the control files, we will see later in details what these are | 15:09 |
| norsetto | Kopfgeldjaeger2: yes | 15:09 |
| norsetto | That's about it, simple isn't? | 15:10 |
| NielsE | seems easy | 15:10 |
| DRebellion | norsetto, why does debian use the 'ar' format instead of 'tar' | 15:10 |
| DRebellion | ? | 15:10 |
| Dabian | I am trying to catch up. | 15:10 |
| norsetto | DRebellion: good question, I guess thats because ar is simpler | 15:11 |
| DRebellion | norsetto, then, why don't we all use ar instead of tar? | 15:11 |
| norsetto | DRebellion: because ar is not universal | 15:11 |
| norsetto | DRebellion: ar is just GNU, tar is unix and linux | 15:12 |
| DRebellion | ok | 15:12 |
| norsetto | I guess everybody know what dpkg is? | 15:12 |
| norsetto | OK, dpkg is Debian main package management program (some people would disagree with the word management though) | 15:12 |
| bhavi_ | yes | 15:13 |
| DRebellion | just a note, wikipedia says that tar has weak integrity checking and supports unicode, ar does not. | 15:13 |
| norsetto | DRebellion: yes, its simpler ;-) | 15:13 |
| norsetto | It can be used to install, unpack, configure, remove or purge packages | 15:13 |
| norsetto | It can also be used to extract or list the content of a package | 15:14 |
| norsetto | dpkg will not attempt to satisfy dependencies, for that you have to use a more advanced tool, like apt (which uses dpkg as a backend) | 15:14 |
| norsetto | If you want to be a good maintainer, you should learn how dpkg works when it installs/upgrades or removes packages | 15:15 |
| norsetto | please let me know if you think I'm going too fast | 15:15 |
| MikeMc | ok | 15:15 |
| Xk2c | is ok | 15:15 |
| NielsE | not for me (yet) | 15:15 |
| DRebellion | fine | 15:15 |
| norsetto | What would you do if, for whatever reasons, dpkg is corrupted on your system and doesn't work anymore? | 15:15 |
| Xk2c | boot knoppix | 15:16 |
| coolbhavi | reconfigure dpkg | 15:16 |
| DRebellion | norsetto, download the dpkg debs, extract the file structure, copy it to your root (/). | 15:16 |
| Kopfgeldjaeger2 | sudo apt-get -f install (in some cases) | 15:16 |
| norsetto | well, think about what we just learned about the structure of a deb package | 15:16 |
| norsetto | DRebellion: exactly | 15:17 |
| DRebellion | :) | 15:17 |
| norsetto | Kopfgeldjaeger2: that will not work, since dpkg will not work | 15:17 |
| coolbhavi | dpkg --configure -a? | 15:17 |
| Kopfgeldjaeger2 | norsetto: yay, i just thought we had unresolved dependencies or so | 15:17 |
| norsetto | coolbhavi: also that would fail, dpkg doesn't work | 15:17 |
| coolbhavi | OK! | 15:18 |
| Kopfgeldjaeger2 | but as that's not the case, jsut ignore it | 15:18 |
| norsetto | For example, lets see what dpkg will do when requested to remove a package | 15:18 |
| norsetto | You can check man dpkg for some basics, there are more complex and exhaustive information in the Debian policy, chapter 6 | 15:18 |
| norsetto | Here is a link if you want a reference during this lecture: http://www.debian.org/doc/debian-policy/ch-maintainerscripts.html | 15:19 |
| norsetto | When dpkg is requested to remove a package, first, it will run the package's prerm script | 15:19 |
| norsetto | The prerm script is one of the maintainer scripts, we will see it later in details | 15:19 |
| norsetto | Then it will remove the files that were installed for that package, minus the configuration files | 15:19 |
| norsetto | We will see later what the configuration files are | 15:19 |
| norsetto | Then it will run the postrm script | 15:20 |
| norsetto | As you guessed it, another maintainer script | 15:20 |
| norsetto | Is that sequence clear? | 15:20 |
| coolbhavi | yes | 15:20 |
| DRebellion | got it | 15:20 |
| Xk2c | prerm then postrm | 15:20 |
| norsetto | For purging it is the same but as a last step it will also remove the configuration files | 15:20 |
| norsetto | Another more complex example is for an upgrade | 15:21 |
| norsetto | First of all it will extract the control files of the new package (the control.tar.gz) | 15:21 |
| coolbhavi | OK | 15:21 |
| norsetto | It will then execute the prerm script of the old package | 15:21 |
| norsetto | After which it will run the preinst script from the new package | 15:21 |
| norsetto | Again, preinst is a maintainer script | 15:22 |
| norsetto | Afterwards, it will remove the old files and unpack the new files | 15:22 |
| norsetto | It will then execute the postrm script of the old package | 15:22 |
| norsetto | Then it will unpack the configuration files | 15:22 |
| norsetto | And finally run the new postinst script | 15:22 |
| norsetto | And finally the last maintainer script :-) | 15:22 |
| norsetto | You don't need to memorise all these steps, a man dpkg will refresh your memory when you need it | 15:23 |
| norsetto | As the name indicates, prerm is roughly called before a removal, postinst, after an installation and so on | 15:23 |
| Dabian | If dpkg is corrupted .. I couldn't use apt-get, right? | 15:23 |
| norsetto | All clear so far? If so, lets see what these scripts in general are used for | 15:24 |
| norsetto | Dabian: absolutely | 15:24 |
| norsetto | As a general rule of thumb, many preinst stop services for packages which are being upgraded | 15:24 |
| norsetto | Many postinst start or restart services once a new package has been installed or upgraded | 15:24 |
| norsetto | Sometime you may use postinst to finish a configuration, asking as well user input. This should preferably be done via debconf | 15:25 |
| norsetto | We won't cover debconf in this lecture, but you should read about it; very good info can be found here: http://www.fifi.org/doc/debconf-doc/tutorial.html | 15:25 |
| norsetto | prerm are typically used to stop services for a package before removing the package | 15:26 |
| norsetto | postrm typically modifies links or other files associated with the package, and/or removes files created by the package | 15:26 |
| norsetto | When you want/need to add a maintainer script in your package, as for every packaging file you add it in the debian directory | 15:27 |
| DRebellion | norsetto, unfortunately, I have to leave. | 15:27 |
| norsetto | For instance the prerm script for the binary <foo> will be called <foo>.prerm and so on, if there is only one binary package you may omit the <foo> | 15:27 |
| norsetto | DRebellion: ok, it was good to have you here, even if for short | 15:27 |
| DRebellion | ;) | 15:27 |
| coolbhavi | norsetto, me too | 15:27 |
| geser | another good link to see how maintainer scripts interact is http://women.debian.org/wiki/English/MaintainerScripts | 15:28 |
| DRebellion | I will read the logs later. | 15:28 |
| norsetto | geser: yes, we will come to that later :-) | 15:28 |
| norsetto | All these will be installed in the binary package by dpkg-deb | 15:28 |
| norsetto | OK, before we cover those in detail, lets see what are the configuration files. Anyone has any idea? | 15:28 |
| Dabian | Uhm | 15:29 |
| Xk2c | everything in /etc | 15:29 |
| Dabian | Which configuration files? | 15:29 |
| Dabian | Oh .. the ones that purge removes | 15:29 |
| norsetto | Dabian: those I was talking about before, when I mentioned purging for instance | 15:29 |
| Dabian | Right | 15:29 |
| norsetto | Configuration files are all files which affects the operation of a program, or provides site- or host-specific information, or otherwise customises the behaviour of a program | 15:30 |
| Dabian | Well, those are the ones that tells the programs in the package how to adapt to the current system, and how to work. | 15:30 |
| norsetto | Some examples which should be in your system are: | 15:30 |
| norsetto | Dabian: exactly | 15:30 |
| norsetto | /etc/apt/sources.list | 15:30 |
| norsetto | you all know about that | 15:30 |
| norsetto | /etc/X11/xorg.conf (this tends to disappear but I still want it!) | 15:30 |
| norsetto | /etc/network/interfaces | 15:31 |
| norsetto | Now, the policy dictates that any configuration files created or used by your package must reside in /etc | 15:31 |
| norsetto | which is the case for all these examples | 15:31 |
| norsetto | For any installed <package>, you will find a list of its configuration files in /var/lib/dpkg/info/<package>.conffiles | 15:31 |
| norsetto | For instance: | 15:32 |
| norsetto | /var/lib/dpkg/info/x11-common.conffiles | 15:32 |
| huats | you say "now" it was not the case with previous policy (before 3.8.0) ? | 15:32 |
| norsetto | /var/lib/dpkg/info/udev.conffiles | 15:32 |
| Dabian | ahhh | 15:32 |
| norsetto | No, I meant it as an interlocution ;-) | 15:32 |
| huats | ok :) | 15:32 |
| Dabian | Hence the "," | 15:33 |
| Dabian | huats: However, I think debconf is /kinda/ new ... (I don't think it was there pre debian 2.0) | 15:33 |
| norsetto | Dabian: yes, sorry if it wasn't clear. You may want to check those, they are all readable | 15:33 |
| norsetto | conffiles are special, by default they are not removed on removal or overwritten on upgrade! | 15:34 |
| Dabian | Its very clear .. I already know the .list files .. but I guess they don't show the conf files :) | 15:34 |
| norsetto | Dabian: they do actually, but not those which are created DURING the installation process | 15:35 |
| Dabian | Right | 15:35 |
| norsetto | Very good, lets see now what are the control files | 15:35 |
| norsetto | Control files are all the files in a debian package necessary for its installation and configuration | 15:35 |
| norsetto | Some examples of control files which you may see: | 15:36 |
| norsetto | conffiles (a list of all configuration files) | 15:36 |
| norsetto | we have seen this already | 15:36 |
| norsetto | config (debconf config script) | 15:36 |
| norsetto | templates (debconf templates) | 15:36 |
| norsetto | md5sums (md5sums for all files to be installed) | 15:36 |
| norsetto | shlibs (contain the mapping from shared libraries to the necessary dependency information) | 15:36 |
| norsetto | triggers (triggers list) | 15:37 |
| norsetto | control (binary control file) | 15:37 |
| norsetto | Be carefull about control file (singular) and control files (plural) | 15:37 |
| norsetto | and then all the maintainer scripts: postinst, postrm, preinst, prerm | 15:37 |
| norsetto | All these are in control.tar.gz, which we have seen before | 15:37 |
| norsetto | They are all unpacked in /var/lib/dpkg/info/ | 15:38 |
| norsetto | There, you will also find a <package>.list file, which will lists ALL files belonging to <package> | 15:38 |
| norsetto | strictly speaking <package>.list is not a control file though :-) | 15:38 |
| Dabian | Excluding those that are created during install :) | 15:39 |
| norsetto | You can check it out now if you want | 15:39 |
| norsetto | We have a nice utility that can be used to work with control files: dpkg-deb | 15:40 |
| norsetto | Also dpkg can be used since it can also act as a front end to dpkg-deb | 15:40 |
| norsetto | with dpkg-deb you can: | 15:40 |
| norsetto | print a summary of the content of a package as well as its control file | 15:40 |
| norsetto | print the content of control files | 15:40 |
| norsetto | print control file information: this is the binary control file | 15:40 |
| norsetto | Lets see some examples: | 15:41 |
| norsetto | dpkg-deb -e <package> : Extracts the control information files into a subdir of the current dir named DEBIAN | 15:41 |
| norsetto | dpkg-deb -I <package> : Print a summary of the content as well as the control file | 15:41 |
| norsetto | dpkg-deb -I <package> <control file>: Print content of one of the <control file>s | 15:41 |
| norsetto | dpkg-deb -f <package> : Print control file information | 15:42 |
| norsetto | If you have not yet downloaded the deb, now is the time to do it | 15:42 |
| norsetto | Everybody has it? | 15:42 |
| NielsE | yes | 15:42 |
| huats | yep | 15:42 |
| MikeMc | yes | 15:42 |
| Dabian | -e doesn't list anything? | 15:43 |
| norsetto | Good, now lets check some of these commands on the ucf package: | 15:43 |
| Dabian | For the example package, that is. | 15:43 |
| norsetto | Lets start with dpkg -e :-) | 15:43 |
| Dabian | ok :) | 15:43 |
| norsetto | dpkg -e ucf_3.005_all.deb | 15:43 |
| norsetto | Now you should have a DEBIAN subdirectory. Can you check what is in there? | 15:43 |
| Xk2c | norsetto: dpkg -e or dpkg-deb -e ? | 15:43 |
| NielsE | Dabian: -e creaes a directory | 15:43 |
| Dabian | oh | 15:43 |
| norsetto | Xk2c: the same | 15:43 |
| Xk2c | ic | 15:43 |
| Dabian | conffiles control md5sums postinst postrm preinst templates | 15:44 |
| norsetto | DEBIAN is nothing less nothing more than control.tar.gz again :-) | 15:44 |
| norsetto | Lets check the content of the control file | 15:44 |
| norsetto | That would be DEBIAN/control | 15:44 |
| Dabian | NielsE: Thank you .. I just reread norsettos explanation. :) | 15:45 |
| norsetto | do you see what is in there? | 15:45 |
| norsetto | As you can see its a mixed bag of things from debian/control and more | 15:45 |
| Xk2c | norsetto: that looks liek the output of aptitude show $PACKAGE | 15:45 |
| norsetto | Xk2c: indeed, thats where aptitude fetch that info | 15:45 |
| Xk2c | ic | 15:45 |
| norsetto | More info about the control file can be found in the deb-control man page (part of the dpkg-dev package) | 15:46 |
| norsetto | Now, lets see another example: | 15:46 |
| norsetto | dpkg-deb -I ucf_3.005_all.deb | 15:46 |
| norsetto | You will see the control file + a summary of all the control files, with some info about them, like size, number of lines, and if it is an executable what kind of executable it is | 15:47 |
| norsetto | Lets see a final example, how can we see what are the configuration files of ucf? | 15:47 |
| Xk2c | cat ucf.conffiles | 15:48 |
| Xk2c | :) | 15:48 |
| norsetto | Xk2c: well, yes, you can use dpkg(-deb) -e, but also dpkg -I | 15:48 |
| Dabian | There is a file named conffiles | 15:48 |
| Dabian | otherwise I guess you can do this: | 15:48 |
| ranf | dpkg-deb -I ucf_3.005_all.deb conffiles | 15:49 |
| Dabian | dpkg-deb <package> conffile | 15:49 |
| norsetto | ranf: yes | 15:49 |
| Dabian | oh .. I forgot -I | 15:49 |
| Xk2c | cool | 15:49 |
| norsetto | OK, we are finally coming to the crunch of the lecture. | 15:49 |
| Xk2c | :( sad | 15:49 |
| norsetto | Please let me know if there is anything unclear before we proceed | 15:50 |
| MikeMc | are conffiles always necessary? | 15:50 |
| norsetto | MikeMc: no, it depends on the package | 15:50 |
| norsetto | MikeMc: as you can see for ucf there is only one | 15:51 |
| MikeMc | thanks | 15:51 |
| norsetto | As we have seen, maintainer scripts are part of a package which the package management system will run for you when your package is installed, upgraded or removed. | 15:51 |
| norsetto | They are normally shell (/bin/sh) scripts but they can also be other shebang scripts | 15:51 |
| norsetto | Its normally better to use /bin/sh, otherwise you have to depends on bash/perl/python,etc. | 15:52 |
| norsetto | Also, POSIX shell or Bash are preferred to Perl/Python etc. since they enable debhelper to easily add bits to the scripts | 15:52 |
| norsetto | On Ubuntu (and now Debian) /bin/sh points to dash, so be careful, if you use bashisms, it will fail ... | 15:52 |
| norsetto | Lets check the postrm for ucf, how can we do that? | 15:53 |
| NielsE | cat postrm | 15:53 |
| norsetto | NielsE: :-) and without exctracting it? | 15:53 |
| Dabian | dpkg-deb -I <packagename> postrm | 15:54 |
| huats | dpkg-deb -I ucf_3.005_all.deb postrm | 15:54 |
| Dabian | ? | 15:54 |
| norsetto | indeed | 15:54 |
| Xk2c | dpkg-deb -I ucf_3.005_all.deb postrm | 15:54 |
| Dabian | Neat :) | 15:54 |
| norsetto | As you can see, its a pretty complex (or apparently so) script. Some general remarks: | 15:55 |
| norsetto | The policy requires that maintainer scripts must be idempotent, this means that you need to make sure nothing bad will happen if the script is called twice where it would usually be called once | 15:55 |
| norsetto | Standard input and output may be redirected (e.g. into pipes) for logging purposes, so don't rely on them being a tty | 15:55 |
| sistpoty|work | erm, just to clear up: you can use anything that has essential=yes set in maintainer scripts (and *don't need a depends*). Anything else is not always safe to use (e.g. a preinst won't have dependencies installed) | 15:56 |
| Dabian | idempotent .. thats a hard word. :) | 15:56 |
| Dabian | Or I guess, just unfamiliar, really. | 15:56 |
| Dabian | to me. :) | 15:56 |
| norsetto | sistpoty|work: yes, I will come to an example about that later on | 15:56 |
| sistpoty|work | :) | 15:56 |
| norsetto | All prompting or interactive configuration should be kept to a minimum. When it is necessary, you should use the debconf package for the interface. Remember that prompting in any case can only be in the configure stage of the postinst script | 15:57 |
| Dabian | sistpoty|work: That was greek to me. | 15:57 |
| norsetto | Keep the maintainer scripts as simple as possible, and don't assume that $PATH will allow you to use all commands (so use absolute and complete paths) | 15:57 |
| sistpoty|work | Dabian: norsetto will come to it laters ;) | 15:57 |
| Dabian | OK, thought so :) | 15:57 |
| norsetto | Now to the script, note the set -e at the top: this will make the script abort if any command returns an error value | 15:58 |
| norsetto | Very important, we don't want to continue if there is any error, and possibly cripple the user's machine | 15:58 |
| norsetto | But as important as well, we don't want to leave uninstalled/unconfigured stuff which may cripple the user's package management system! | 15:58 |
| norsetto | As we will see, there are ways to recover errors, but the most important way is to CHECK beforehand for things that can go wrong and abort GRACEFULLY if possible. | 15:59 |
| norsetto | You can use the usual test constructs, for example: | 15:59 |
| norsetto | if [ <test> ]; then | 15:59 |
| norsetto | print >&2 "Error. Please fix me." | 15:59 |
| norsetto | exit 0; | 15:59 |
| norsetto | fi | 15:59 |
| norsetto | Note that I use exit 0, so I let dpkg finish its job and at the same time I alert the user that something "strange" happened | 16:00 |
| norsetto | Lets download another example: | 16:00 |
| norsetto | wget http://archive.ubuntu.com/ubuntu/pool/universe/t/trousers/trousers_0.3.1-4_amd64.deb | 16:00 |
| * norsetto loves that package :-S | 16:01 | |
| norsetto | Now lets check the postinst script | 16:01 |
| norsetto | everybody has it? | 16:02 |
| NielsE | yes | 16:02 |
| MikeMc | yep | 16:02 |
| norsetto | Now, lets for a moment forget the first part of the script, we will see later what all these actions are | 16:02 |
| norsetto | In the second part, after the comment # Automatically added by dh_installinit you will see that it will call another script, which, in effect IS a maintainer script even though it will not be part of the control files | 16:02 |
| norsetto | In this particular case, its a configuration file, which makes it tricky since it might be difficult to remove it if it fails | 16:02 |
| norsetto | This is an init script, ie. a script used to launch a daemon at boot-up or stop it at shutdown | 16:03 |
| norsetto | It is standard practice to use this script as well to launch or stop the daemon during package upgrade/removal | 16:03 |
| norsetto | Now, lets check this init script. How could we do it? | 16:03 |
| MikeMc | try each line manually? | 16:04 |
| norsetto | MikeMc: sure, but I mean, how to see what is inside first ;-) | 16:04 |
| norsetto | Anyone? | 16:04 |
| Dabian | trying to catch up | 16:05 |
| tarvid | dpkg-deb -I trousers_0.3.1-4_amd64.deb postinst|less | 16:05 |
| norsetto | tarvid: that won't show the init script though, remember, its not a control file | 16:05 |
| MikeMc | done it thought you meant something else | 16:05 |
| norsetto | MikeMc: yes, my Englitalian its a bit funny ;-) | 16:06 |
| Xk2c | norsetto: $ dpkg -x trousers_0.3.1-4_amd64.deb | 16:06 |
| Dabian | Oh | 16:06 |
| norsetto | Xk2c: yes, that a good one | 16:06 |
| norsetto | And you will find it as etc/init.d/trousers | 16:06 |
| Xk2c | vim etc/init.d/trousers | 16:07 |
| norsetto | does everybody see what is the content of this file? | 16:07 |
| MikeMc | yes | 16:07 |
| Xk2c | yep | 16:07 |
| norsetto | As you can see this script does something very nice, before starting the daemon it checks if something which is necessary for the daemon to be successfull is there | 16:08 |
| norsetto | Do you see the test lines? | 16:08 |
| norsetto | If it isn't, it doesn't bail off, and hell breaks loose, it will print an error message and exit gracefully | 16:08 |
| norsetto | So, the user is informed and his system is not screwed up | 16:08 |
| Dabian | It checks if the binary exists and is excuteable by the current user? | 16:08 |
| norsetto | Dabian: yes, but the check I mean is about about the existance of the dev file, which means the tpm module is loaded into memory | 16:09 |
| Dabian | darn ... I think I have to run ... can I idle here ? | 16:09 |
| norsetto | Dabian: sure | 16:09 |
| Xk2c | Dabian: lines 26 to 31 | 16:10 |
| Dabian | Thanks for the lesson so far norsetto .. its been very enlightning :) | 16:10 |
| norsetto | Dabian: thx to you, I'm very glad you find it usefull | 16:10 |
| norsetto | Look also at the stop action | 16:10 |
| norsetto | The daemon is stopped with the --oknodo option, this makes return exit status 0 instead of 1 in case of errors | 16:11 |
| norsetto | Could we improve on that? For instance, catch obvious problems before they happen? | 16:11 |
| Xk2c | norsetto: if the daemon ist running is checked already? | 16:13 |
| Xk2c | with the "--pidfile /var/run/$NAME.pid " ? | 16:13 |
| norsetto | yes, but what again if /dev/tpm doesn't exist? | 16:13 |
| norsetto | Even better (and the package has later been patched like this) would be to have the same check that we have for the start action | 16:13 |
| norsetto | is everything clear? | 16:14 |
| MikeMc | yes | 16:15 |
| NielsE | yes | 16:15 |
| norsetto | OK, back to ucf, lets check again the postrm | 16:15 |
| norsetto | As you can see there are a number of actions in there | 16:15 |
| norsetto | All the possible actions are (hope I don't forget any): | 16:16 |
| norsetto | configure, abort-deconfigure, deconfigure-in-favour, install, abort-install, upgrade, abort-upgrade, failed-upgrade, remove, abort-remove, remove in-favour, purge, disappear | 16:16 |
| norsetto | ah, the beauty of cut and paste :-) | 16:16 |
| norsetto | These are all coming from dpkg | 16:16 |
| Xk2c_ | sorry network problems | 16:16 |
| === e-jat is now known as groups | ||
| === groups is now known as e-jat | ||
| norsetto | For the benefit of Xk2c_ | 16:17 |
| === e-jat is now known as fenris- | ||
| norsetto | configure, abort-deconfigure, deconfigure-in-favour, install, abort-install, upgrade, abort-upgrade, failed-upgrade, remove, abort-remove, remove in-favour, purge, disappear | 16:17 |
| norsetto | these are all the actions that dpkg can use when calling a maintainer script | 16:18 |
| norsetto | Some of these are pretty obscure and it is likely that you will never need to use them | 16:18 |
| norsetto | As you can see, the maintainer for ucf used some template (I guess coming from an old version of dh-make) where all the possible actions for a postrm script are already there, with comments which explain what they do | 16:18 |
| norsetto | All clear? Don't try to memorise all these! | 16:18 |
| norsetto | You have to know that when dpkg calls the maintainer scripts, its calls are of the type: script action [package|version] | 16:19 |
| norsetto | Each action will depend on what dpkg is trying to do | 16:19 |
| norsetto | Is it trying to remove the package? | 16:19 |
| norsetto | Is it trying to purge the package? | 16:19 |
| norsetto | Is it trying to recover from a failed remove? | 16:19 |
| norsetto | What dpkg will do depends on the request from the user and the status of the system | 16:20 |
| norsetto | For instance for postrm, dpkg could call it as follows: | 16:20 |
| norsetto | <postrm> remove: dpkg will call this after a removal | 16:20 |
| norsetto | <postrm> purge: dpkg will call this after a purge | 16:20 |
| norsetto | <old-postrm> upgrade <new-version>: dpkg will call this after removing an old package during an upgrade | 16:21 |
| norsetto | and an important one | 16:21 |
| norsetto | <new-postrm> failed-upgrade <old-version>: dpkg will call this if the postrm of the old package during upgrade failed | 16:21 |
| norsetto | The other cases are really very special or corner cases and we will skip them for this lecture | 16:21 |
| norsetto | Now, lets see an example for an upgrade | 16:21 |
| norsetto | There are some nice diagrams that could help you to visualise this in http://women.debian.org/wiki/English/MaintainerScripts | 16:22 |
| norsetto | geser already pointed these out to you | 16:22 |
| norsetto | Can you see the upgrade diagram in this page? | 16:22 |
| MikeMc | yep | 16:23 |
| NielsE | yes | 16:23 |
| norsetto | The nominal steps are as we explained briefly above (prerm->preinst->postrm->postinst), but if something goes wrong things get hairy, for instance: | 16:23 |
| norsetto | First dpkg will call prerm-from-old-package upgrade new-version | 16:24 |
| norsetto | If the script runs but exits with a non-zero exit status, dpkg will attempt: | 16:24 |
| norsetto | prerm-from-new-package failed-upgrade old-version | 16:24 |
| norsetto | Do you see this in the diagram? | 16:24 |
| MikeMc | yes | 16:24 |
| norsetto | If this works, the upgrade continues. If this does not work, it will continue with: | 16:24 |
| norsetto | postinst-from-old-package abort-upgrade new-version | 16:25 |
| norsetto | If this works, then the old-version is "Installed", if not, the old version is in a "Failed-Config" state. | 16:25 |
| norsetto | upgrade is really a complex example, but I wanted to show you what attempts dpkg will do in case something goes wrong | 16:25 |
| norsetto | you can make a clever use of these in your maintainer scripts, to ensure that dpkg will eventually be able ti install, or eventually to remove faulty packages | 16:26 |
| norsetto | Another example, configuration: | 16:26 |
| norsetto | postinst configure most-recently-configured-version | 16:27 |
| norsetto | That's it, no attempt is made to recover errors! If the configuration fails, the package is in a "Failed Config" state, and an error message is generated, so, be careful with that | 16:27 |
| norsetto | You can find these and all other possible dpkg calls in the debian policy, chapter 6 | 16:27 |
| norsetto | well, that was the hard bit, I hope your brain is not confused as mine :-) | 16:27 |
| * norsetto tries unsuccesfully to wake up huats | 16:28 | |
| norsetto | Usually one will not manually write these scripts, debhelper scripts will do that for us | 16:28 |
| huats | norsetto: (phone call...) | 16:28 |
| norsetto | For instance if our package installs icons and we want to update the Freedesktop icon caches, you will just add a dh_icons call in debian/rules, this will create appropriate maintainers scripts that will call update-icon-caches | 16:28 |
| norsetto | If you have several debhelper scriplets, all the fragments from each will be collated in a single maintainer script | 16:29 |
| norsetto | If you need to have a mixed script, ie. a part manually written and another generated by debhelper, you can use the #DEBHELPER# token | 16:29 |
| norsetto | During build, the token will be replaced by whatever "functions" debhelper needs to generate | 16:29 |
| norsetto | You have seen and example of this for trousers | 16:29 |
| norsetto | In that case there was a fragment that was inserted in the postinst by dh_installinit | 16:30 |
| norsetto | You can check it by looking at trousers.postinst in the debian dir of the trousers source package | 16:30 |
| norsetto | Even though you will rarely need to write maintainer scripts manually, it is VERY IMPORTANT to be able to debug failures coming from maintainer scripts | 16:30 |
| norsetto | This is because these may cause packages to be left in an non-desirable state and possibly screw up the whole package management | 16:30 |
| norsetto | You will see many bugs of this kind in the LP bug tracker | 16:31 |
| norsetto | It is also important to check an init script provided by upstream before installing it | 16:31 |
| norsetto | Since it will be a maintainer script we need to make sure that it doesn't fail, and if it does so, it does it gracefully | 16:31 |
| norsetto | If an init script fails, it may fail one of the maintainer scripts and leave a user's system in a mess | 16:32 |
| norsetto | On top of that, it is a conffile, so, not removed on upgrade | 16:32 |
| norsetto | Anybody know how we could solve such a situation? | 16:32 |
| norsetto | Anybody has ever seen or used dh_installinit? | 16:33 |
| huats | never user | 16:34 |
| huats | used | 16:34 |
| Xk2c_ | no | 16:34 |
| MikeMc | no sounds intersting... | 16:34 |
| norsetto | We have seen this for trousers, dh_installinit is the debhelper scriplet that installs init scripts into package build directories | 16:34 |
| norsetto | dh_installinit also automatically generates the postinst, postrm and prerm commands needed to set up the symlinks in /etc/rc*.d/ and to start and stop the init scripts | 16:34 |
| norsetto | there is a nice option that this debhelper function provides | 16:35 |
| norsetto | You can override the failure action by using the option --error-handler=function when calling dh_installinit in debian/rules | 16:35 |
| norsetto | In this case, the named shell function will be run if the init script fails | 16:35 |
| norsetto | This function should be provided in the prerm and postinst scripts, before the #DEBHELPER# token | 16:35 |
| norsetto | This will simply modify the snippets inserted by dh_installinit with something like "/etc/init.d/init_script action || function" instead of "/etc/init.d/init_script action || exit $?" | 16:35 |
| norsetto | so if "init_script action" fails it will execute funtion instead of exiting with an error <> 0 and failing the maintainer script | 16:36 |
| norsetto | For instance a simple --error-handler=true will just make the removal continue if the initscript fails stopping the daemon and you have | 16:37 |
| norsetto | a new version to install that fix that very bug! Without this, you will have to ask the user to manually modify the init script himself | 16:37 |
| norsetto | Another thing to be careful is that you cannot assume in your maintainer scripts that conffiles from another package are available | 16:38 |
| norsetto | Furthermore, you should not assume that a conffile being present means that some functionality are available, remember that conffiles are only removed on purge | 16:38 |
| norsetto | I have an example to show you this | 16:38 |
| norsetto | I stumbled against a problem of the first kind in a recent bug (bug 248150), which I hope I won't have to fix (hint to any member of motu-sru which might be around :-)) | 16:38 |
| ubottu | Launchpad bug 248150 in fcalendar "package r-cran-fcalendar 220.10063-1 failed to install/upgrade: " [Undecided,Confirmed] https://launchpad.net/bugs/248150 | 16:39 |
| norsetto | this is showing an example of sistpoty|work words of wisdom | 16:39 |
| sistpoty|work | <- isn't anywhere wise :P | 16:39 |
| huats | :) | 16:40 |
| norsetto | there is a very nice explanation by supermaster steve langasek in the debian bug which should enligthen you (it did it for me ;-)) | 16:40 |
| norsetto | do you see what the problem is ? | 16:42 |
| NielsE | I think I globally understand it | 16:43 |
| MikeMc | symlink? | 16:44 |
| norsetto | to tell you in a (hopefull) nutshell, the postrm of the old packages was calling a binary which relied on a conffile being present | 16:44 |
| norsetto | problem is, that that conffile was moved from one location to another, and a symlink installed to point to the new location | 16:45 |
| norsetto | now, when the package is upgraded, if the new package providing the binary is not yet installed, everything goes fine | 16:46 |
| norsetto | however, if the new package is installed, since it will not yet be configured, the symlink will be left dangling, and that binary will fail | 16:47 |
| norsetto | and fail 40+ other packages with it ... | 16:47 |
| norsetto | Don't be worried if you don't get it immediately, its not that simple :-) | 16:48 |
| norsetto | that was just to show you what kind of considerations one would have to make when doing maintainer scripts and what pain they can cause to debug | 16:49 |
| norsetto | Shall we do another simpler example? | 16:49 |
| MikeMc | yes | 16:49 |
| NielsE | yes | 16:49 |
| huats | norsetto: if you have one | 16:50 |
| huats | :) | 16:50 |
| norsetto | Lets see another example, bug 250088 | 16:50 |
| ubottu | Launchpad bug 250088 in psad "can't remove psad package" [Undecided,Fix released] https://launchpad.net/bugs/250088 | 16:50 |
| norsetto | huats: I have plenty ;-) | 16:50 |
| huats | I have doubts on the "simple" :) | 16:50 |
| norsetto | to understand this, download psad (the buggy version) and check what the prerm script does | 16:51 |
| norsetto | wget http://archive.ubuntu.com/ubuntu/pool/universe/p/psad/psad_2.1-1_amd64.deb | 16:53 |
| norsetto | dpkg -I psad_2.1-1_amd64.deb prerm | 16:54 |
| norsetto | do you see where it fails? | 16:54 |
| huats | the line find /var/run/psad/ -type f -exec rm -f {} \; | 16:54 |
| norsetto | yes, do you see why it fails? | 16:55 |
| huats | should test first the existence of /var/run/psad | 16:55 |
| norsetto | indeed, lets not forget that in ubuntu /var/run is mounted as tmpfs | 16:55 |
| huats | since if it does not exists, find will raise an error | 16:55 |
| norsetto | indeed, and fail the removal, leaving the user machine in a mess (recoverable, but still a mess) | 16:56 |
| huats | sure | 16:56 |
| NielsE | yes | 16:56 |
| norsetto | in this particular case I also believe that this will need to be done in the preinst and/or postinst (creating /var/run/psad if not existing) | 16:58 |
| huats | ok | 16:58 |
| norsetto | another example? | 16:59 |
| MikeMc | ok | 16:59 |
| huats | go ahead | 16:59 |
| huats | :) | 16:59 |
| norsetto | ok, lats one since its about two hours now :-) | 17:00 |
| norsetto | bug 251696 | 17:00 |
| ubottu | Launchpad bug 251696 in pygopherd "package pygopherd 2.0.17 failed to install/upgrade: subprocess post-installation script returned error exit status 1 (dup-of: 238755)" [Undecided,Incomplete] https://launchpad.net/bugs/251696 | 17:00 |
| ubottu | Launchpad bug 238755 in shadow "'Account has expired' message when adding a new user" [Undecided,Confirmed] https://launchpad.net/bugs/238755 | 17:00 |
| norsetto | I think pleia2 has recorded everything, so the logs will be available? | 17:02 |
| norsetto | do you see whats happening with this? | 17:03 |
| huats | the pb here seems that the postinst tests if /var/gopher exists | 17:03 |
| huats | and then copy in /var/gopher/gophermap | 17:03 |
| huats | no ? | 17:03 |
| norsetto | hmmm, no :-) | 17:03 |
| pleia2 | james_w typically posts logs following a session, we'll reply to the list with their location | 17:03 |
| norsetto | pleia2: okki, thanks | 17:03 |
| MikeMc | 'passwd -l' is broken? | 17:04 |
| norsetto | there are two problems actually, but one is not lethal (even though is silly) | 17:04 |
| NielsE | pleia2: the message goes to the motu-list? | 17:05 |
| norsetto | MikeMc: well, you could say that, other people may also disagree, but the issue is definetively in the chsh call | 17:05 |
| norsetto | you can all see that in the script | 17:05 |
| huats | when he grep the passd file | 17:06 |
| huats | he search for the HOMEDIR | 17:06 |
| huats | but he do not take into accoutn the result of HOMEDIREXISTS | 17:06 |
| norsetto | huats: he is using that, later on | 17:07 |
| Xk2c_ | Your account has expired; please contact your system administrator | 17:07 |
| Xk2c_ | from the original bug | 17:08 |
| norsetto | yes | 17:08 |
| norsetto | just immediately after resuming normal error checking | 17:08 |
| norsetto | he is calling "chsh -s /bin/sh gopher" | 17:08 |
| james_w | NielsE: yes, it will. | 17:08 |
| norsetto | that will fail if your account is locked, and fail everything else | 17:08 |
| Xk2c_ | yeah passwd -l is broken :( | 17:09 |
| norsetto | :-) | 17:09 |
| norsetto | somebody call that a feature ;-) | 17:09 |
| Xk2c_ | norsetto: if used at the right place | 17:10 |
| Xk2c_ | but an expired rootaccount causes headages | 17:10 |
| norsetto | Well, I really hope you enjoyed the lecture and I could help to shed some light on maintainer scripts and dpkg, any questions ? | 17:11 |
| norsetto | Anybody still alive? | 17:12 |
| NielsE | thanks for taking the time, you did a perfect job, no questions for me atm | 17:12 |
| Xk2c_ | norsetto: thank you | 17:12 |
| MikeMc | thanks, when is the next one? | 17:12 |
| huats | thanks a lot cesare | 17:13 |
| norsetto | MikeMc: thats for james_w to answer? I know he is working on solving the misteries surrounding kde packaging | 17:13 |
| james_w | thanks norsetto | 17:14 |
| james_w | I don't have another session lined up right now, but I hope to have one in two weeks time. | 17:14 |
| norsetto | thanks to everybody that partecipated, was a real pleasure to talk with you all | 17:15 |
| james_w | wiki.ubutntu.com/MOTU/School/Requests if you would like to request a session, or offer to present one. | 17:15 |
| james_w | thanks again norsetto, it was a great session. I'll post the logs a little later. | 17:15 |
| james_w | any help create documentation out of the logs would be appreciated. | 17:16 |
| NielsE | wiki.ubuntu.com/MOTU/School/Request does not exist... | 17:16 |
| NielsE | oh wait | 17:16 |
| NielsE | nvm :) | 17:17 |
| Xk2c_ | bye | 17:26 |
| NielsE | bye | 17:28 |
| prashant321 | whats going on in the session | 18:34 |
| e-jat | hi | 19:05 |
| === Kopfgeldjaeger2 is now known as Kopfgeldjaeger | ||
| word | are there any logs from the session earlier today? | 22:11 |
| pleia2 | word: https://wiki.ubuntu.com/MOTU/School/MaintainerScripts | 22:11 |
| word | thanks :) | 22:11 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!