| twb | In lucid, I have an upstart task that runs "iptables-restore /etc/iptab". | 04:08 |
|---|---|---|
| twb | If this fails, I want the system to freak out and disable all services -- say, switching to single-user mode. | 04:09 |
| twb | How would I configure this? | 04:09 |
| twb | Can I have two "pre-start exec" lines (instead of a pre-start script)? | 04:46 |
| mbiebl | twb: no | 06:06 |
| twb | Okey dokey | 06:40 |
| atpa8a | hello | 18:21 |
| atpa8a | any clues how i can enforce that the all of the network interfaces are up before sysvinit scripts run?.. | 18:21 |
| Keybuk | atpa8a: I've already answered your question three times | 18:28 |
| * ion notices Keybuk isn’t a Conan fan. | 18:29 | |
| Keybuk | Conan? | 18:29 |
| atpa8a | Keybuk: sorry, must've missed | 18:30 |
| Keybuk | atpa8a: to change the state in which sysvinit scripts are run, you edit the /etc/init/rc-sysinit.conf file | 18:30 |
| ion | keybuk: http://www.youtube.com/watch?v=iNu3MW-lb6Y | 18:30 |
| Keybuk | and change the "start on" kine | 18:30 |
| Keybuk | as you can see, right now it says to start when the *loopback* network interface is up | 18:31 |
| Keybuk | but no other | 18:31 |
| Keybuk | if you alter that start on line, you can delay sysvinit scripts as long as you like | 18:31 |
| Keybuk | e.g. until all network interfaces are up | 18:31 |
| Keybuk | or until the cows come home | 18:31 |
| atpa8a | Keybuk: i tried removing that IFACE= at all and tried making it IFACE=eth0 but nothing helped | 18:31 |
| atpa8a | still slapd fails to start because the network is not up yet | 18:32 |
| atpa8a | start on filesystem and net-device-up IFACE=eth0 | 18:33 |
| atpa8a | that's what i have there now | 18:33 |
| atpa8a | slapd[2826]: daemon: bind(9) failed errno=99 (Cannot assign | 18:35 |
| atpa8a | requested address) | 18:35 |
| Keybuk | atpa8a: is eth0 the right interface? | 18:36 |
| Keybuk | then it's likely that slapd is in fact waiting for something else | 18:36 |
| atpa8a | yes | 18:36 |
| atpa8a | well | 18:36 |
| atpa8a | slapd needs eth0:0 really | 18:37 |
| atpa8a | should i try that? | 18:37 |
| Keybuk | aha! bridges | 18:37 |
| atpa8a | aliases | 18:37 |
| Keybuk | right | 18:37 |
| Keybuk | so you need to add something like "and started networking" | 18:37 |
| Keybuk | e.g. | 18:37 |
| atpa8a | is there like... IFACE=all? | 18:37 |
| Keybuk | start on filesystem and net-device-up IFACE=lo and started networking | 18:37 |
| Keybuk | meaning filesystem ready, loopback device up, and all virtual devices | 18:38 |
| Keybuk | no, because then you'd have to define "all" | 18:38 |
| atpa8a | gotcha | 18:38 |
| atpa8a | started networking should work | 18:38 |
| Keybuk | e.g. on a laptop with a USB ethernet device, does "all" mean including that or not? | 18:38 |
| Keybuk | and if it means including that, what if it's not plugged in? | 18:38 |
| atpa8a | i see | 18:38 |
| Keybuk | you may want | 18:39 |
| Keybuk | start on filesystem \ | 18:39 |
| Keybuk | and net-device-up IFACE=lo \ | 18:39 |
| Keybuk | and net-device-up IFACE=eth0 \ | 18:39 |
| Keybuk | and started networking | 18:39 |
| Keybuk | because you'll want eth0 as well as the alises | 18:39 |
| atpa8a | can't i just use 'and started networkin' without net-device-up?.. | 18:40 |
| SpamapS | started networking just means ifup -a returned | 18:40 |
| atpa8a | ok... so that would supercede net-device-up, right? | 18:40 |
| Keybuk | no | 18:40 |
| Keybuk | they do different things | 18:41 |
| SpamapS | nope.. ifup just starts the things that configure the ifaces | 18:41 |
| Keybuk | physical network devices, for which you have hardware, are up when you get net-device-up | 18:41 |
| Keybuk | non-physical network devices (virtual, bridges, aliases) are up on the ifup -a | 18:41 |
| atpa8a | got it | 18:41 |
| atpa8a | trying | 18:42 |
| SpamapS | Tho it does mean all statically configured devices are up.. right? | 18:42 |
| atpa8a | thanks a bunch | 18:42 |
| atpa8a | hmm | 18:42 |
| Keybuk | SpamapS: no, because at the point networking runs, the actual devices may not have modules loaded yet ;) | 18:42 |
| Keybuk | SpamapS: so you can't guarantee that | 18:42 |
| atpa8a | should i... just in case add eth1 as well? | 18:42 |
| Keybuk | atpa8a: at some point it's worth filing a bug that slapd should use IP_FREEBIND | 18:42 |
| SpamapS | Keybuk: so ifup -a may fork modprobes and such, even for iface eth0 inet static ? | 18:42 |
| Keybuk | but that's for another day ;) | 18:42 |
| Keybuk | SpamapS: ifup doesn't fork modprobes unless you've stuck modprobe in a pre-up ;) | 18:43 |
| atpa8a | Keybuk: thought about just filing a bug with ubuntu to fix rc-sysvinit.conf | 18:43 |
| Keybuk | atpa8a: fix? | 18:43 |
| Keybuk | atpa8a: this is not considered a bug in Ubuntu | 18:43 |
| Keybuk | Ubuntu has *never* delayed starting services until after network interfaces are configured | 18:43 |
| Keybuk | this is because the default out-of-the-box behaviour in Ubuntu isn't to configure network interfaces until *user login* | 18:43 |
| Keybuk | services are supposed to work without a configured network interface | 18:44 |
| SpamapS | Keybuk: so started networking should mean all statically configured, non transient hardware, has been configured. yes? | 18:44 |
| Keybuk | thus the IP_FREEBIND | 18:44 |
| Keybuk | SpamapS: yes | 18:44 |
| Keybuk | SpamapS: assuming you have such a thing | 18:44 |
| atpa8a | hmm | 18:44 |
| SpamapS | Keybuk: thinking about servers mostly | 18:44 |
| SpamapS | Is IP_FREEBIND POSIX or a linux innovation? | 18:45 |
| atpa8a | so if my interfaces are all static, 'and started networking' should be enough? | 18:45 |
| Keybuk | Linux I believe | 18:45 |
| Keybuk | atpa8a: yes | 18:45 |
| SpamapS | I could see it being a problem for some upstreams if its linux only. | 18:45 |
| Keybuk | atpa8a: though see notes about modprobe above | 18:45 |
| Keybuk | SpamapS: #ifdef is their friend | 18:45 |
| SpamapS | Right, most will be fine w/ that | 18:46 |
| atpa8a | Keybuk: sure, thanks a bunch | 18:46 |
| SpamapS | some will be adamantly opposed. | 18:46 |
| atpa8a | i've a feeling slapd might be one of those | 18:46 |
| Keybuk | SpamapS: if you can find any upstream not using at least one Linux-specific API, I'd be impressed ;-) | 18:46 |
| Keybuk | and that's why distros carry patches | 18:46 |
| Keybuk | after all, as soon as you see setsockopt() or ioctl() or anything like that - you're in linux-specific-land | 18:46 |
| SpamapS | Keybuk: I've recently proposed hanging rc-sysinit to start on filesystem and started networking rather than net-device-up IFACE=lo so that users can expect that any static interfaces will always be configured before runlevel 2.... given what we've talked about here.. I think that its still a good idea. | 18:47 |
| SpamapS | s/hanging/changing/ | 18:47 |
| Keybuk | SpamapS: the trouble there is you run the risk of locking the boot | 18:47 |
| Keybuk | because people use sysvinit to bring up the dependencies of static interfaces | 18:47 |
| Keybuk | in theory though I have no problem with delaying rc-sysinit as long as you like | 18:48 |
| Keybuk | because things should not be using rc-sysinit | 18:48 |
| Keybuk | slapd should have its own .conf etc. | 18:48 |
| SpamapS | Hanging it or failing a bunch of stuff.. I'll choose failing a bunch of stuff. But we need a generic solution. | 18:48 |
| SpamapS | Keybuk: its own.conf that will 'start on runlevel [2345]' ? ;) | 18:49 |
| Keybuk | we don't even *have* rc-sysinit on Chromium OS after all ;-) | 18:49 |
| Keybuk | SpamapS: I thought you were introducing jobs to replace those? :) | 18:49 |
| SpamapS | I am.. definitely. Haven't gotten much traction on those. | 18:49 |
| Keybuk | I always take no traction as silent assent ; | 18:49 |
| Keybuk | ;) | 18:49 |
| SpamapS | Right.. my DMB app has been delayed twice unfortunately. | 18:49 |
| atpa8a | upstart scrips for slapd and a few other services are nowhere to be found for ubuntu | 18:50 |
| SpamapS | otherwise I might have uploaded network-services.conf and wait-for-state.conf already. :) | 18:50 |
| atpa8a | i did consider making my own but... i'm not an experienced SA | 18:50 |
| Keybuk | atpa8a: bug! :p | 18:50 |
| atpa8a | don't know the intricacies | 18:50 |
| atpa8a | Keybuk: there's a bug already but it's on hold | 18:50 |
| * atpa8a is java dev | 18:50 | |
| atpa8a | SA is a hobby :P | 18:51 |
| atpa8a | bb | 18:52 |
| atpa8a | testing the script | 18:52 |
| Keybuk | ion: you realise that killed my X server and I hate you, right? :-) | 18:55 |
| ion | Eh :-D | 18:56 |
| Keybuk | flash seems to hate dual-head non-twinview nvidia | 18:56 |
| wraiden | hello, is anyone actively working on /proc/PID/oom_score_adj support? /proc/PID/oom_adj is deprecated and upstarts oom stanza uses this interface... | 19:00 |
| Keybuk | wraiden: nobody is actively working on it | 19:00 |
| wraiden | i ask because i don't want to do duplicated work on this if anyone has picked the issue up allreda | 19:00 |
| wraiden | k | 19:00 |
| Keybuk | so please feel free to pick it up | 19:00 |
| wraiden | k | 19:01 |
| Keybuk | I believe both APIs are still supported, right? | 19:01 |
| wraiden | will take a similar aproach as udev | 19:01 |
| wraiden | yes | 19:01 |
| Keybuk | try and write to oom_score_adj | 19:01 |
| wraiden | but it polutes dmesg with a warning | 19:02 |
| Keybuk | if that fails with EEXIST, fall back to oom_adj ? | 19:02 |
| wraiden | yes | 19:02 |
| wraiden | and no | 19:02 |
| wraiden | udev checks for filepointer lower than 0 | 19:02 |
| wraiden | so other errorcodes are picked up to | 19:02 |
| wraiden | i think | 19:03 |
| Keybuk | right, but you also want to check errno | 19:03 |
| Keybuk | if you got EPERM from oom_score_adj you shouldn't fall back to oom_adj | 19:03 |
| wraiden | i cook something up over the weekend | 19:03 |
| Keybuk | because you'll just get the same error ;-) | 19:03 |
| Keybuk | EEXIST should be the only reason to fallback | 19:03 |
| Keybuk | but please do cook a patch up, and send it to the ML | 19:03 |
| Keybuk | gratefully accepted | 19:03 |
| wraiden | aye sir ;) | 19:04 |
| wraiden | the only thing is that the score interface has grater values | 19:05 |
| wraiden | so the off case must be -1000 | 19:05 |
| wraiden | the testsuite tests the values... could be problematic... as it will produce falsepositives on the offbound value checks | 19:06 |
| Keybuk | that is why we have a test suite ;-) | 19:06 |
| Keybuk | decide on appropriate behaviour, write tests for it, etc. | 19:06 |
| wraiden | mhm | 19:06 |
| atpa8a | hello again | 19:08 |
| atpa8a | those changes seem to hange the boot... | 19:08 |
| atpa8a | hang that is | 19:08 |
| Keybuk | atpa8a: that implies to me that your network interfaces are (at least, in part) brought up by sysvinit scripts | 19:08 |
| Keybuk | so your boot is: | 19:08 |
| Keybuk | - waiting for networking to start sysvinit | 19:08 |
| Keybuk | - unable to start networking because it's waiting for sysvinit | 19:08 |
| atpa8a | ok... | 19:08 |
| Keybuk | ie. deadlock | 19:08 |
| Keybuk | so here's your next trick: | 19:09 |
| Keybuk | remove the bits you added to rc-sysinit | 19:09 |
| Keybuk | now create a new job | 19:09 |
| Keybuk | /etc/init/slapd-interlock.conf | 19:09 |
| Keybuk | start on slapd-interlock and net-device-up IFACE=eth1 and started networking | 19:09 |
| Keybuk | (that's it) | 19:09 |
| Keybuk | and in /etc/init.d/slapd, in the start case, add | 19:09 |
| Keybuk | initctl emit slapd-interlock | 19:09 |
| atpa8a | hmm | 19:10 |
| Keybuk | you may need some protection here to make sure it only runs on actual boot, e.g. by checking $RUNLEVEL or some shit - I haven't fiddled with sysvinit scripts in a while to remember what that is | 19:10 |
| Keybuk | what this will do is: | 19:10 |
| Keybuk | sysvinit scripts will start as normal | 19:10 |
| Keybuk | slapd will start | 19:10 |
| Keybuk | it will run "initctl emit slapd-interlock" and wait for that event to complete | 19:10 |
| Keybuk | the .conf file in Upstart will pick up that event, and block it | 19:10 |
| Keybuk | while it also waits for net-device-up IFACE=eth0 and started networking | 19:10 |
| atpa8a | (i think) i'm following... | 19:11 |
| Keybuk | and then when all three happen, that ".conf" is running, so the event is unblocked, so initctl emit finished, so slapd carries on | 19:11 |
| atpa8a | let me try\ | 19:11 |
| atpa8a | thanks a bunch | 19:11 |
| atpa8a | hmm | 19:15 |
| atpa8a | seems like i cannot boot into single user mode either... | 19:16 |
| wraiden | keybuk: is it ok if i change the tests for offbound values in the testsuite to values for the score interface? the will be offbound in booth cases then and we can avoid to check two interfaces with distinct test code... | 19:20 |
| atpa8a | livecd ftw! | 19:21 |
| Keybuk | atpa8a: single user mode is a sysvinit thing ;-) | 19:23 |
| Keybuk | wraiden: you can change the tests, but be careful to not change the thing they're testing | 19:23 |
| Keybuk | current job files should work unchanged | 19:23 |
| atpa8a | hmm | 20:03 |
| atpa8a | Keybuk: would you say... it'd be better to call it networking-interlock.conf? | 20:03 |
| atpa8a | kinda a common thing for all services that face the same situation | 20:04 |
| atpa8a | and emit networking-interlock as well? | 20:04 |
| Keybuk | yes, but the first job would release the interlock | 20:04 |
| Keybuk | the second would hang | 20:04 |
| Keybuk | which is why it's specific to slapd | 20:04 |
| atpa8a | i see | 20:05 |
| atpa8a | can it be made generic?.. | 20:05 |
| Keybuk | not without converting slapd to upstart | 20:09 |
| atpa8a | gotcha | 20:11 |
| atpa8a | hmm | 20:15 |
| atpa8a | just to confirm... i don't need 'start on filesystem' in slapd-interlock.conf, right? | 20:15 |
| Keybuk | right | 20:17 |
| atpa8a | thanks | 20:18 |
| atpa8a | what would happen if i just do /etc/init.d/slapd restart when there's the emit in it? | 20:22 |
| atpa8a | nothing happened... | 20:24 |
| atpa8a | ok... | 20:25 |
| atpa8a | trying booting now | 20:25 |
| atpa8a | hmm | 20:31 |
| atpa8a | something didn't work... | 20:31 |
| atpa8a | heh | 20:33 |
| atpa8a | typos :P | 20:33 |
| atpa8a | ok | 20:35 |
| atpa8a | now that i fixed it... it's not working when starting manually | 20:35 |
| atpa8a | gotta do the emit conditionally... | 20:35 |
| atpa8a | [ $RUNLEVEL ] && initctl emit slapd-interlock | 20:46 |
| atpa8a | that should do it\ | 20:46 |
| atpa8a | testing | 20:46 |
| wraiden | mhm, the changed valid range of values for the oom handling isn't easy to handle... | 20:58 |
| wraiden | if i change it do accept 1000 to -1000 in the parser and change it by divide with 59 in the fallback (ugly hack, i know) we could get away wir | 21:00 |
| wraiden | *with a string change for any supported locale... | 21:00 |
| wraiden | another approach would be to check for the existence of /proc/self/oom_score_adj in init startup code, set a flag according to existence and use that to change behavior accordingly wich would eliminate ugly hacks to adapt the value in fallback case | 21:04 |
| wraiden | but that would complicate the testcase... | 21:05 |
| wraiden | btw: isn't EEXIST the error for "i want to crate a file but it exists" and not "file i want to open don't exist" ? | 21:07 |
| wraiden | posix manpage for open (which fopen refers to) defines it as the fist rather than the second... | 21:08 |
| wraiden | i'll take the flag oom_score_adj and check for the flag road... | 21:24 |
| wraiden | +existence | 21:24 |
| atpa8a | hmm | 21:37 |
| atpa8a | something isn't working... | 21:37 |
| atpa8a | http://pastie.org/1661174 | 21:37 |
| atpa8a | but it says initctl status slapd-interlock ~ | 21:51 |
| atpa8a | slapd-interlock start/running | 21:51 |
| atpa8a | hmm | 22:09 |
| atpa8a | fixed it by using 'net-device-up IFACE=eth0:3 \' | 22:10 |
| atpa8a | working now | 22:10 |
| atpa8a | thank you gents | 22:10 |
| atpa8a | Keybuk: thank you much! | 22:10 |
| Keybuk | oh cool | 22:10 |
| wraiden | isn't networking enough to check for? | 22:10 |
| Keybuk | I didn't even know you could do that :p | 22:10 |
| atpa8a | wraiden: apparently not | 22:11 |
| wraiden | y? | 22:11 |
| atpa8a | http://pastie.org/1661174 | 22:11 |
| atpa8a | that was before i used IFACE=eth0:3 and it didn't work | 22:11 |
| wraiden | y do you add the interface up events? | 22:12 |
| atpa8a | hmm :P Keybuk recommended | 22:12 |
| wraiden | arnt they prerequisites for networking? | 22:12 |
| atpa8a | no | 22:13 |
| wraiden | ah | 22:13 |
| atpa8a | if you have a wiki i can drop this solution into, i'll document it | 22:14 |
| atpa8a | putting it in my private wiki now | 22:14 |
| === atpa8a_ is now known as atpa8a | ||
| atpa8a | thank you folkes | 22:44 |
| atpa8a | very helpful | 22:44 |
| wraiden | Keybuk: first compiletested only version without messing with the testsuite is here: http://pastie.org/1661386 | 22:48 |
| Keybuk | my concern about that approach is that it changes the behaviour of Upstart if you reboot into an older kernel | 22:51 |
| Keybuk | suddenly jobs go from running to "parse error" | 22:51 |
| wraiden | yes | 22:52 |
| wraiden | problem here is that the interfaces arnt compatible with respect to value range ... | 22:53 |
| Keybuk | one approach to that would be | 22:54 |
| wraiden | a possible way out would be to accept the new range by default and | 22:54 |
| Keybuk | oom NN = old range | 22:54 |
| Keybuk | oom score NN = new range | 22:54 |
| wraiden | divide the value | 22:54 |
| Keybuk | and accept both | 22:54 |
| Keybuk | accepting the new range only would change the behaviour of existing jobs | 22:54 |
| Keybuk | (though I bet there are none out there using the score rather than just "never") | 22:54 |
| wraiden | that was my thinking eighter... | 22:55 |
| wraiden | a percent value would be nice | 22:57 |
| wraiden | could be used as operator to calculate the right value for each interface | 22:57 |
| wraiden | -operator + base | 22:57 |
| Keybuk | that's no real difference than just sticking to the old vange | 22:58 |
| Keybuk | range | 22:58 |
| Keybuk | more fun, "oom less than <other job>" and let upstart pick the values? :P | 22:58 |
| wraiden | that would by far overwehlm my rusty c skills *hehe* | 22:59 |
| bigdata | how do I set an environment variable in a job file? | 23:00 |
| wraiden | by reading: man 5 init | 23:02 |
| wraiden | search foe env | 23:02 |
| wraiden | *for | 23:02 |
| wraiden | i'll go for the "oom score =" one i think ... | 23:04 |
| Keybuk | well, no = in upstartyland but I think that makes the most sense | 23:04 |
| wraiden | jepp | 23:05 |
| wraiden | we need to support both interfaces, as noone upstream boundling conf files for upstart can predict which interface a user might have avaiable... | 23:06 |
| wraiden | that leads to the question: is any upstream package shipping upstart definitions? | 23:07 |
| wraiden | anyway a nice to have | 23:09 |
| Keybuk | wraiden: there are definitely both upstreams and distributions shipping upstart config files | 23:10 |
| Keybuk | so they must not be broken | 23:10 |
| wraiden | what to do in a case where the new interface is usable but only the old one has defined value? | 23:10 |
| Keybuk | multiply it by sufficient numbers to turn it into the new value | 23:10 |
| wraiden | + and the old one is not available (after its removal) | 23:11 |
| Keybuk | the behaviours are the same, just the ranges have changed | 23:11 |
| wraiden | k | 23:11 |
| Keybuk | likewise if you only have new value in the .conf and only the old interface is available, you divide | 23:11 |
| wraiden | will hack that together... | 23:11 |
| Keybuk | (I'd multiply when parsing the old value so that the oom score in the job is always -1000..1000) | 23:11 |
| Keybuk | then divide again when writing to the old interface | 23:12 |
| wraiden | mhm | 23:12 |
| Keybuk | I wonder just how many bugs on LP, at this point, are about pre-stop being fucking busted? | 23:25 |
| wraiden | whats busted with it? | 23:27 |
| wraiden | the multiplying and dividing isn't easily possible... | 23:28 |
| Keybuk | why not easily possible? | 23:29 |
| Keybuk | there's an operator for them in C and everything <g> | 23:29 |
| wraiden | the values on the old interface are asyncronous reguarding positive and negative values | 23:29 |
| Keybuk | normalise the base, adjust the scale | 23:29 |
| wraiden | you cant get -16 | 23:29 |
| Keybuk | maths 101 | 23:29 |
| Keybuk | -16 is "the minimum" and -17 is "never" with +15 being "always" | 23:30 |
| Keybuk | so I guess you make +15 be 1000, -17 be -1000 | 23:30 |
| wraiden | i would have used a simple * 62.5 | 23:30 |
| Keybuk | optionally -16 is either -999 or -1000-1000/17 | 23:31 |
| Keybuk | look at what the kerneld oes | 23:31 |
| Keybuk | ie. the code that deals with writing to the deprecated interface | 23:31 |
| Keybuk | I'm sure it already has the exact algorithm to scale to the new one | 23:31 |
| wraiden | if (task->signal->oom_adj == OOM_ADJUST_MAX) | 23:35 |
| wraiden | task->signal->oom_score_adj = OOM_SCORE_ADJ_MAX; | 23:35 |
| wraiden | else | 23:36 |
| wraiden | task->signal->oom_score_adj = (oom_adjust * OOM_SCORE_ADJ_MAX) / | 23:36 |
| wraiden | -OOM_DISABLE; | 23:36 |
| * wraiden feeds his cats... | 23:37 | |
| bigdata | still trying to get the env setup up, I did read 'man 5 init', but the script being called does not see the PERL5LIB, any ideas http://pastebin.com/VDRdWM3V | 23:47 |
| Keybuk | it should | 23:49 |
| Keybuk | (though the "export PERL5LIB" there doesn't do what you think) | 23:49 |
| Keybuk | keybuk /root# | 23:51 |
| Keybuk | keybuk /root# cat /etc/init/test.conf | 23:51 |
| Keybuk | env DEMO=this:is:a:test | 23:51 |
| Keybuk | exec echo $DEMO > /tmp/demo | 23:51 |
| Keybuk | keybuk /root# start test | 23:51 |
| Keybuk | test start/running, process 13507 | 23:51 |
| Keybuk | keybuk /root# cat /tmp/demo | 23:51 |
| Keybuk | this:is:a:test | 23:51 |
| wraiden | how about adding a oom_score_adj int to job_class so that we only need to calculate in the conf parsing case and just use what we have calculated? | 23:54 |
| Keybuk | that's what I meant | 23:54 |
| bigdata | your right, it did work | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!