=== Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === jelmer [n=jelmer@157pc196.sshunet.nl] has joined #upstart === Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === int0x0c [n=ben@161.253.14.157] has joined #upstart === GodEater_ [n=bryan@rockbox/staff/GodEater] has joined #upstart === juergbi [n=juerg@80-219-21-79.dclient.hispeed.ch] has joined #upstart === Md [i=md@freenode/staff/md] has joined #upstart === Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === Md [i=md@freenode/staff/md] has joined #upstart === Amaranth [n=travis@ubuntu/member/Amaranth] has joined #upstart === Keybuk [n=scott@wing-commander.netsplit.com] has joined #upstart === Amaranth_ [n=travis@ubuntu/member/Amaranth] has joined #upstart [03:02] hmm, momentary thoughts on bug tracking [03:02] is "logd not working" a bug? [03:03] or is it simply wishlist [03:03] is it a regression? [03:03] if it never worked, its a wishlist IMO [03:03] it worked once, then we disabled it [03:03] the reason I wonder is that the "triaged" state doesn't make sense for it [03:03] urrrgh [03:04] (or some of the other bugs in there) [03:04] Confirmed - there's enough information to replicate the bu [03:04] Triaged - there's enough information to fix the bug [03:04] but for things like that, neither makes sense [03:04] which is why I wonder whether it's a wishlist [03:05] its a regression so in one sense its a bug. but its not really fatal to anything so more of a wishlist item.. [03:05] yeah, tricky innit [03:06] aye [03:06] toss a coin? [03:06] heads its a wishlist [03:06] ;) [03:07] had any more upstart related thoughtlets? [03:09] not yet [03:11] im not sure what to do for my needs :( [03:12] what were those again? [03:13] one example is for syncing windows mobile devices [03:13] synce.org has a couple of daemon type progs it needs [03:13] i dont really like the idea of them running constantly [03:13] so wanted to try using hal events to start them [03:14] i had been thinking hal -> upstart [03:18] on top of that, there are quite a few devices that need you to ppp them before they are useful. network manager could be used for that in the future, but i dont know how far off such things are... [03:23] ah yes, I remember [03:26] months off, at least [03:37] ideally i'd like to have this stuff auto-magicked by then [03:37] i think the guy i was talking to on ubuntu-devel-discuss was hoping to have something pretty for HH... [03:38] oh, who was that? [03:39] i think Martin Owens [03:40] we had cultural differences between yorkshire and liverpool.. [03:40] :) [03:41] the sync project stuff? [03:44] aye [03:45] yeah, there's a few people working on that for 8.04 [03:47] nice that they are getting in touch... [03:47] how do you mean? [03:48] i've not heard anything from martin or anyone else and theres been no mention of it on ubuntu dd [03:48] wouldn't be yet [03:48] we're still in the 7.10 release cycle [03:48] ahh [03:48] 8.04 doesn't begin for another couple of weeks [03:49] i know that theres a RH guy really interested in Conduit, so i was hoping to get a bit of cross-distro love [03:49] but i really want some flashy "here i am plugging in my phone and it just working" demo before i go pimping again! :) [03:50] the trouble with cross-distro love is that the distros don't tend to love each other ;) [03:50] aye :( [03:51] i'll love the distro that loves me i guess [03:52] unfortunately there arent any local (at least inside UK) get togethers i know of in the near term where I can pimp Conduit and/or get help with the fiddly polishy bits needed to support some of the devices we want to support [03:53] i think the nearest is going to be, ironic as its so far, linux conf au [03:53] or lug radio us === Jc2k is still angry he missed guadec [04:16] Keybuk: think i'm going to resort to "pupstart" unless you have any ideas. [04:16] i.e. a subset of upstart in python solely focused on my simple needs [04:17] init can't be written in Python [04:17] you'll end up with a machine full of zombie processes, and a non-functioning init :p [04:17] (not without rewriting the C bits of Python, anyway) [04:18] eh [04:18] it wouldnt be init [04:19] what would it be then? [04:19] some event -> start a program, some event -> stop a program [04:19] primarily hal events [04:20] if it's not an init daemon, please don't call it "upstart" or anything even similar [04:20] and able to deal with multiple copies of same programs [04:20] otherwise that will just confuse people [04:20] ok [04:20] you already have that though -- it's called D-BUS :p [04:20] eh [04:20] really? [04:21] that does most of it already [04:21] do i have to add dbus to everything though [04:21] probably [04:21] this is not a bad thing [04:22] alas a bit much for my weak C skills [04:22] might be doable for the synce case though [04:23] Perhaps gutsy just contains too old versions of opensync and conduit, but i dont seem to be able to sync two evolutions over the network using either one. :-\ === Keybuk hasn't tried that yet [04:24] conduit | 0.3.2-1 | http://gb.archive.ubuntu.com gutsy/universe Packages [04:24] i wet myself when i made the video. [04:24] opensync | 0.19-1.2ubuntu1 | http://gb.archive.ubuntu.com gutsy/main Sources [04:24] (d be nice if conduit and opensync were integrated somehow, btw.) [04:24] its *really* sweet [04:24] conduit is up to 0.3.4 [04:24] the network sync hasn't been released, you need to run svn [04:25] opensync, 0.19 is ancient. [04:25] ion_: i'm working to let opensync be used from conduit [04:25] Nice [04:25] nothing newer in Debian [04:25] atm i am able to use opensync plugins in conduit [04:25] and hoping to support multiple sync engines soon too [04:26] meant to be going to germany to collaborate with them soon [04:27] anyway, if there are any conduit questions im happy to answer on #conduit (GIMPnet), but back on topic [04:28] doing this all in dbus has implications on the scope of upstart? [04:28] e.g. equally some of boot up (sh/w)ould be handled by the dbus system activation patches? [04:29] yes [04:30] in a 100% new-world desktop, the only init supervised services would be udev/hal, d-bus and gdm [04:30] (though that ignores things like mounting filesystems, etc. - but then RH have a single shell script for all that) [04:30] what about crashes [04:31] should some of these dbus triggered services self-heal or just wait to be needed [04:31] who knows [04:31] :) [04:31] this is why Upstart development has stalled for a while [04:31] figuring out how it fits in [04:32] the thing i still dont understand in the new-world desktop is how services die. [04:32] i want things to go away.. [04:34] I think we have to consider D-BUS signals a key part of the system [04:34] which means that they need to be able to start and stop processes on their own [04:35] it makes sense to have a ServiceManager daemon that handles this [04:35] which makes room for Upstart [04:35] (it then makes sense for D-BUS and HAL to use that ServiceManager for their own activation purposes) [04:35] the difficulty comes in two stages: [04:35] 1) how are D-BUS signals captured, and converted into useful environment for the service being started [04:35] 2) what about events that aren't D-BUS signals yet? [04:37] it depends on what information will be needed for (2), but from the outsider position wrapper seems doable to inject faux dbus signals [04:37] the annoying thing is how hard D-BUS signals are to define [04:37] aye [04:38] perhaps the hal path, key, value approach would work? [04:39] the signal would always be a path, key and value [04:39] or perhaps a list of [04:39] The DpmsModeChanged signal in the org.gnome.PowerManager interface emitted from the /org/gnome/PowerManager object by the org.gnome.PowerManager service === Jc2k grumbles at dbus [04:40] i was thinking we'd be able to control all the signals, but of course we might want to respond to other signals [04:40] (and that's for objects you can name! You can't reliably name objects from HAL!) [04:40] so artificially restricting the kinds of signals that are interesting is not possible... [04:41] yeah [04:41] is dbus introspectable enough from C land that we could have a FDI-alike? [04:42] yes [04:42] but then we're in FDI hell [04:42] with great power comes great pains in the butt [04:42] and sometimes a signal is just a signal [04:42] and contains no useful information [04:43] you need to make an object call to obtain the information about the signal [04:43] and at this point we have a desire for a minimal scripting language [04:44] and the equal opposite desire for fear of the untold danger involved in such a thing [04:44] so you can see why this needs careful thought :p [04:44] indeed. [04:44] hopefully pestering you will help you decide whats best ;) [04:45] heh [04:45] I haven't had time to sit down and understand D-BUS [04:45] and more particularly, how it's used [04:46] if I had a list of the signals, methods, etc. that all the usual desktop bits generated, then I would have a better idea [04:46] since that would infer what to listen for [04:46] eh [04:46] i've tried to understand it [04:47] and i hurted :P [04:47] the current case seems to be that the initial uses of dbus were a bit crap and not how the API was intended to be used [04:47] but that this is improving [04:48] yeah [04:48] but they won't drop them :p [04:48] :p [05:22] i had an idea [05:22] then i tried to replay it in my head [05:22] and forgot the best bit [05:22] -_- [05:22] i hate jet lag [05:22] 15:17:12> +init can't be written in Python [05:22] 15:17:25> +you'll end up with a machine full of zombie processes, and a non-functioning init :p [05:22] ^^ pardus wrote their init in python ;) [05:23] AlexExtreme: i knew there was one distro that did, forgot its name.. thx [05:23] wait [05:23] although, it's rather buggy :P [05:24] AlexExtreme: have i seen you in #ylug or #lugradio at all? [05:24] I don't think so [05:24] at least, I don't remember ever being in either of those [05:24] hmmm [05:24] its going to bother me now :) [05:25] AlexExtreme: heh [05:25] the Python problem is that it doesn't handle signals properly [05:25] it handles them well enough for user space [05:25] but init has different signal semantics [05:26] yes === Jc2k tries to remember his idea [05:28] i was thinking along the lines of mapping the entire boot sequence and saying which things belonged to dbus activation and which things still need init and any other categories along the way [05:29] i was also thinking that this whole stalemate is because of the big picture [05:29] trying to make sure upstart is ready for everything, without even knowing what everything is [05:31] on the python thing [05:32] could init be chained? [05:32] as in, the actual init just starts a python init which is then "orginary" [05:32] perhaps the "raw" upstart has the service start/stop api [05:33] e.g. org.something.ServiceManager [05:33] and then python pokes the other services [05:34] the python init is still not ordinary [05:34] since it's still pid #1 [05:34] eh [05:35] unless the python process was forked [05:35] in which case, you lose the special powers you get as pid #1 [05:35] which are the reason to be pid #1 in the first place [05:35] i was going for making the special powers to be dbus exposed [05:35] splitting Upstart into a ServiceManager component and a ServiceActivator component has been discussed [05:35] the thought was that /sbin/init would be pid #1 [05:36] and would expose the org.freedesktop.ServiceManager interface [05:36] and then you'd start other things that used that interface to start and stop processes [05:36] though it's not clear how you handle blocking in that scenario [05:36] (where when you start apache, it blocks waiting for other services to start first) [05:38] and presumably in that case the ServiceActivator is "normal"? [05:39] dont really grok the apache case. are you saying that apache shouldnt start until other things are ready, or that other things shouldnt start until apache is ready? [05:40] both [05:40] Jc2k: Both [05:40] heh [05:40] if you've got tomcat installed, it should be started if apache is starting [05:41] and apache should wait until tomcat is started before continuing [05:41] how is that handled in the current case? the init script doesnt exit until apache is ready? [05:41] sorry for nub questions btw [05:41] current case? [05:42] as in, why does moving to ServiceManager ServiceActivator make this different? [05:42] presumably ServiceManager can just issue a signal when a process is loaded? [05:43] kind of like Twisted has Defer's [05:43] but then ServiceManager has to block :) [05:43] ie. when does it decide to carry on [05:43] when the signal fires? [05:44] i imagine it would be painful in C land, but twisted python is pretty good at pausing where its at until a signal completes [05:45] but D-BUS has no notion of signal completion [05:45] it'd actually need to be a method [05:45] which is somewhat kooky, since who is the method against? [05:45] or fire a 'finished' signal [05:45] maybe "completes" was a bad choice [05:46] i call StartProcess and then attach to a signal called ProcessStarted [05:47] also dbus does have a way to specify a function to call when the method completes [05:47] so you can StartProcess(some, params, reply_handler=self.Started_Cb) [05:47] where self is an object containing enough info to carry on [05:48] err? [05:48] sorry, not following [05:48] 1s [05:59] Keybuk: http://pastebin.com/d80f7e06 <- make any sense? [06:00] aside from the fact that its not even close to valid python... [06:03] it's pretty close :) [06:03] Amaranth: i puked when i re-read it. [06:10] Jc2k: the problem is, how does the StupidManager send out the message that the job is starting [06:10] and thereby give other processes the option to handle that message, and delay the starting of the job [06:10] oh, uh, ouch [06:12] Keybuk: i guess that such delays should be handled before you go to ServiceManager [06:12] ? [06:14] actually no [06:15] so theres a difference between the Apache job and the apache process? [06:15] dunno [06:15] you start ./apache and ./tomcot before the apache "job" is ready [06:15] *tomcat even === Jc2k abuses ./ for visual clarity rather than noobness [06:21] Keybuk: are you going to be near IRC tonight? [06:21] yeah probably [06:21] okies [06:22] i have to commute for a bit now, so i'll think about another version of my pastebin [06:22] it will treat a job as something different from a process [06:22] which i think is the key to it satisfy your needs [06:40] interesting [06:40] so GNOME Power Manager is entirely implemented through HAL D-BUS stuff [06:41] signal sender=:1.10 -> dest=(null destination) path=/org/freedesktop/PowerManagement; interface=org.freedesktop.PowerManagement; member=OnBatteryChanged [06:41] boolean false [06:42] actually, sorry, that's wrong [06:42] since that's not coming from HAL [06:42] even though the docs say it will === Keybuk frowns [06:46] GNOME developers should be banned from uploading anything without docs [06:47] (says the author of the most undocumented init system there is :p) [06:56] :-) [06:57] though I'm making some kind of process [06:58] e.g. "running while a battery is present" [06:59] => listen to HAL for DeviceAdded/DeviceRemoved signals, take the object path and then check for the "battery" capability, it has it, check the "battery.present" property is "true" [07:00] (also obviously listen for changes to that property, as well as the NewCapability signal in case some existing object becomes a battery) [07:01] so this requires a combination of method calls and signals [07:06] this strongly implies the need for upstart-addon-hal [07:10] likewise similar for avahi [07:10] since for that you need a ServiceBrowser for each thing you want to watch for === Md [i=md@freenode/staff/md] has joined #upstart === bleinmono [n=toffel@ppp91-76-108-30.pppoe.mtu-net.ru] has joined #upstart === AlexExtreme [n=AlexExtr@frugalware/developer/AlexExtreme] has joined #upstart === bleinmono [n=toffel@ppp91-76-108-30.pppoe.mtu-net.ru] has left #upstart [] === Jc2k is back home at last [09:11] Keybuk: still not grokking the big problem [09:12] it's quite complex to explain :-/ [09:12] tell me about it [09:12] i feel like im missing something obvious [09:13] before upstart i implemented an event based job manager for my old company [09:13] events were simpler there [09:13] timer events, file change events and "job x finished" events [09:13] each job was dependant on multiple events [09:14] and then i had a set of god awful queries to work out which jobs were due to run [09:14] right [09:14] that could handle all kinds of edge cases [09:14] it's the god awful bit that causes us issues [09:14] well in my case, the god awful came from having to build it out of ms access and vbscript :P [09:14] heh [09:15] ok, let me try and quickly explain [09:15] jobs can be manually or automatically started and stopped [09:15] manual is by initctl, automatic is by events [09:15] jobs also generate events themselves [09:15] four of them [09:15] starting = I'm about to start, but nothing has been run yet [09:15] started = I'm now running and ready for action [09:16] stopping = I'm about to stop, but nothing has been killed yet [09:16] stopped = I'm no longer running [09:16] thus the events that jobs can be automatically started/stopped by aren't just external forces like battery status, but can also be based on other jobs [09:16] so if HAL depends on D-BUS, it can be started by the "started dbus" event and stopped by the "stopping dbus" event [09:17] that way HAL is never running when d-bus isn't [09:17] grokking so far [09:17] events in Upstart have the notion of completion [09:17] initctl emit foo will actually wait until that event emission has completed [09:17] the completion of an event is when all jobs that reacted to it have reached their goal state [09:17] (running for services, stopped again for tasks) [09:18] so if I have a service that's started "on foo", "initctl emit foo" will actually show the progress of that service, and not return until that service is running [09:18] this means events also have failure [09:18] if that service fails to start, initctl emit foo will actually exit 1 [09:18] (if no services are affected by the event, the event completes immediately) [09:19] ok? [09:19] grokking :) [09:19] good so far [09:19] right [09:20] so upstart uses this event completion internally for the starting and stopping job events [09:20] when the service emits the starting event, it will not proceed until the starting event has completed [09:20] likewise for the stopping event [09:20] so if I have a small task that's run "on stopping mysql" (e.g. to backup the database), mysql *will not be stopped* until that task completes [09:21] since services only need to reach running or stopped, this also provides upstart's dependency mechanism [09:21] so even though mysqld isnt running, the mysql "job" is still "stopping" [09:21] no [09:21] mysqld *IS* running [09:21] upstart hasn't killed it yet [09:21] ahh [09:21] ok [09:22] (or it might not be, if mysqld died, but that's a different story :p) [09:22] lol [09:22] the event does actually tell you why mysql is stopping [09:22] on stopping mysql failed main [09:22] EXIT_SIGNAL=SEGV [09:22] for example [09:23] anyway, digressing [09:23] since services only need to reach running or stopping, this provides Upstart's dependency system [09:23] the tomcat server, if installed, is a dependency of Apache [09:23] (web sites use it) [09:23] so the tomcat job can specify: [09:23] start on starting apache [09:23] stop on stopped apache [09:24] now, if you "start apache" the following will happen: [09:24] the apache job emits the starting apache event [09:24] which starts the tomcat job (event is now blocked) [09:24] the tomcat job issues starting tomcat, runs its various scripts, and eventually the tomcat daemon is running [09:24] tomcat job emits started tomcat [09:24] and the event is now unblocked [09:24] and thus the event completes [09:24] apache can now start, run its various scripts, and the httpd daemon [09:25] so the tomcat job, by being started when apache is starting (rather than started) can make itself a dependency of apache [09:25] (and rather usefully, it becomes tied to the lifecyle of apache - you don't have to think of it as a separate service) [09:26] got it [09:27] so with start on starting apache we make sure tomcat starts before apache [09:27] exactly [09:27] so i cant actually do my python demo without mixing in dependencies [09:27] this is the bit that's hard to implement with a separated service manager and automation manager core [09:28] since the service manager needs to be able to block the service on things that the automation manager knows [09:29] i still think i can do it with async dbus [09:29] though i worry i missed something [09:29] the problem with dbus is that you need to specify someone to ask :) [09:29] signals have no completion [09:29] so the starting/stopping events can't be signals [09:29] (we'd never know whether something cared) [09:29] they have to be methods [09:29] and you can't have an open method [09:29] you have to know who you're asking [09:30] at which point, why have two processes since they're inherently coupled anyway? [09:30] are we saying that the problem is that when ServiceManager raises a signal you no longer know what that signal is about? [09:31] no [09:31] that it doesn't know whether anyone cared about the signal [09:31] or whether those who cared about the signal did anything about it [09:31] this is the bit where i dont get it :) [09:32] im ok up to where i said "i worry i missed something" [09:32] then my world fell apart [09:32] heh [09:32] which bit don't you get? [09:32] you understand how Upstart events work wrt completion?> [09:33] apache is starting, but tomcat has to start first, so tomcat starts. apache is still in the starting state until tomcat is started.. then apache finishes starting and it is started... [09:33] all that stuff makes sense [09:34] right [09:34] i dont understand how it breaks down when the service is split in 2 [09:34] i dont understand what is lost [09:35] the way that events work :) [09:35] because D-BUS doesn't support that at all [09:36] i see that it doesnt support that natively [09:36] exactly [09:36] you can't use signals to do it [09:36] you have to use methods [09:36] does that make sense? [09:37] but most of the state should be in the automation manager.. [09:37] if i can prototype it, you buy me beer? [09:37] :) [09:37] sure [09:37] does it make sense that you can't use D-BUS signals? [09:37] because they are fire-and-forget [09:38] kind of [09:38] im going to try to use async method calls first [09:39] so the problem with method calls is that, in D-BUS, you must know your target [09:39] I can't tell the world that a job is starting [09:39] I can only tell one person [09:39] (I can tell the world with signals, but I cannot find out whether the world cared) [09:40] with a method, I can tell one person, and they can give me a result [09:40] but I have to hard-code that one person [09:40] right [09:40] now i understand. [09:40] and i scratch my freshly shaved beard. [09:41] but... [09:41] so since the service manager would have the name of the policy bit hard-coded [09:41] why have them separate at all? :p [09:41] no. [09:41] brain failing... holding on to thread of something... [09:41] i think we are talking about different services [09:42] in my mind i was assuming that pid 1 *soley* starts and stops single processes [09:42] but in actual fact [09:42] pid 1 starts and stops jobs [09:42]