=== koolhead11|afk is now known as koolhead11 [08:35] Morning === freeflyi1g is now known as freeflying === koolhead11 is now known as koolhead17 [13:22] sidnei, nice logo ;-) [13:27] :D [13:27] :) [13:28] alienpulse: so you were interested in writing an ensemble formula :) [13:28] welcome [13:28] alienpulse: we just had a private chat .. but let's continue talking over here [13:28] kim0, okay no prob. [13:28] so basically the best way to get started is to start reading the couple of tutoirals [13:28] https://ensemble.ubuntu.com/docs/user-tutorial.html [13:28] ill [13:28] https://ensemble.ubuntu.com/docs/write-formula.html [13:29] with getting started [13:29] maybe checkout the 2 vids we have to get a feel [13:29] then pick something you'd like to ensemblize [13:29] and start hacking [13:29] :) [13:29] thanks a lot .. [13:29] ill work on that ASAP ! [13:30] alienpulse: in every step along the way .. if you have questions ask here in this channel and ping me if you dont get an answer .. I'll always help [13:30] alienpulse: AWESOME :) [13:30] \o/ [13:30] woohoo [13:30] kim0, Cool :) [13:30] \m/ [13:31] alienpulse: once you decide which package you'd like to do .. let me know .. rock n roll [13:32] kim0, okay ill do .. i think im gonna go for Drupal [13:33] alienpulse: nice choice .. I've written the writing a formula tutorial on drupal .. so you can start from there [13:33] alienpulse: I didn't really link the formula to principia .. so you can do it [13:33] kim0, ohh okay [13:33] also, I used drush to keep things too simple .. but you might want to do it differently [13:34] Have fun [13:34] emm [13:34] ok [13:34] drupal is a good choice to start with :) [13:35] alienpulse: what's your LP username [13:35] alienpulse: I'll create a bug and assign to you so you :) [13:37] kim0, PM :) [13:37] alienpulse: just assigned :) [13:39] kim0, thanks a lot ... [13:39] :) [13:39] alienpulse: https://bugs.launchpad.net/principia/+bug/805955 [13:39] <_mup_> Bug #805955: Formula needed: Drupal < https://launchpad.net/bugs/805955 > [13:40] alienpulse: I think the best way, is to assemble the drupal formula from the tutorial, get it working, then start hacking on it to make it a "real" formula :) [13:40] enjoy [13:41] kim0, okay man ill do my best ,,, thanks again , [13:41] Awesome .. and remember for any questions .. we're all here for you [13:41] rock on [13:41] kim0, am not able to move anywer :( [13:42] koolhead17: don't feel bad :) [13:42] koolhead17: you're the best .. are you hitting a problem with phpmyadmin .. or just not enough time ? [13:43] i would say i need to learn bit more about dbconfig-common :) [13:45] the preseed thing ? [13:45] I dont really know much about that :) [13:47] hmm. i will try to nail it tonight [13:50] kim0, BTW people are coming up with nice Ensemble logo`s E for Ensemble :) [13:52] koolhead17: hehe yeah :) [14:03] <_mup_> ensemble/auto-peer r264 committed by kapil.thangavelu@canonical.com [14:03] <_mup_> ensemble deploy automatically adds peer relations. [14:04] <_mup_> ensemble/test-api r236 committed by kapil.thangavelu@canonical.com [14:04] <_mup_> a new test api for construction of environments. [14:53] <_mup_> ensemble/auto-peer r265 committed by kapil.thangavelu@canonical.com [14:53] <_mup_> riak peer rel test for deploy [14:54] Woot [14:55] niemeyer, its in review now as well [14:55] hazmat: Awesome, thanks a lot [14:56] niemeyer, for the relation-broken cli api.. i'm thinking just return empty values for relation-list, relation-get (except for self which returns valid values), relation-set is unclear, i think it should error [14:57] hazmat: It depends a bit on the use case [14:57] hazmat: I think Adam had in mind doing actions depending on the remote data, I'd think [14:58] hazmat: It's not clear if should handle this case, or advise to store any _needed_ information locally [14:58] niemeyer, hmm.. there is no guarantee that remote data even exists at that point [14:58] hazmat: Yeah, that's the key point to ponder [14:58] its assymetric cleanup across the relation endpoints [14:59] hazmat: The question is whether we should really do this or should allow querying of last available state [14:59] niemeyer, access to local data is fine, and we could even store it (i'd prefer not to... but it could be valid).. the relation data is effectively dead, even if the relation is resurrected (it will be as a new relation) [14:59] s/store/allow writes to local data [15:00] hazmat: Nah, that'd be weird.. it's a clean up really [15:00] niemeyer, agreed [15:00] hazmat: We should talk to Adam to see what he's trying to do [15:03] niemeyer: he wanted to do things like remove grants in the local db when the remote side goes away [15:04] bcsaller: Morning! [15:04] bcsaller: bcsaller: Yeah, that's what I got from the bug description too [15:04] morning [15:04] bcsaller: The question is what is he trying to fetch from remote side [15:04] bcsaller: Why is relation-get relevant in this context? [15:05] Maybe all he needs is a relation id [15:05] yeah, I think he had to know which relationship was broken so he could map the data [15:05] I suspect thats correct [15:05] niemeyer, hmm. clint put in a bug last month about wanting to get the service name of the remote side for cases like this [15:05] In which case we'd be fine with the already filed bugs [15:05] so he could cleanup local resources for the remote side [15:05] hazmat: Yeah, I've linked the relevant bugs on Adam's bug too [15:05] niemeyer: we didn't talk about it at the rally but we should still talk about the bug where relations should get ids [15:05] hazmat: None of the other bugs require access to the data, though [15:05] niemeyer, its unclear if the relation-id would always contain the relevant data [15:06] I wasn't clear on the use case in the sense that I didn't know which tools would take those ids to do anything useful [15:06] bcsaller: Yeah, we have to implement this [15:06] niemeyer, ie. if the remote side specifically requested a particular principal/db name etc [15:06] bcsaller: There are a few use cases described in the bug [15:06] hazmat: Well, that'd be weird either way [15:07] hazmat: That kind of cleanup should happen locally [15:07] hazmat: Rather than trusting the remote data (which may have change) [15:07] d [15:08] true, but its a communication protocol, i can definitely see cases where local cleanup might want remote state, although i think it could be argued that state should be copied local if its needed for cleanup [15:10] hazmat: Right.. it's a bit strange in that the relation was broken, and one wants to talk to the remote besides that. [15:11] Either way, let's look at the use case [15:20] hmm... i think i need to add a generic file manifest to formulas to be able to do clean upgrades ( remove obsolete files) [15:29] hazmat: Interesting idea [15:29] hazmat: We probably don't need an actual file, though [15:30] hazmat: The content of the formula bundle is its own manifest [15:30] niemeyer, yeah.. its computed on demand [15:39] * niemeyer => lunch ftw [15:43] hi can someone pls check this http://paste.ubuntu.com/638462/ [15:44] * kim0 looks [15:45] oh more licensing :) [15:45] * hazmat looks as well [15:46] daker, unless your working at canonical and doing this for work, i'm pretty sure you need the canonical copyright statement, ie. is it an original work, on a not for hire basis [15:47] er.. you don't need [15:51] hazmat, you mean the first section ? [15:55] daker, any of the sections that reference canonical copyright would be dependent on this being done as work for hire, contribution to ensemble itself would require copyright assignment per canonical policy, but formulas are effectively data inputs to ensemble, and are licensed/copyrighted by the author as an independent work. If the author is performing the work as a result of a fiduciary agreement, the terms of such agreement come into play. for mo [15:55] st common situations this results in the work being considered a work for hire, with copyright assignment (and thus licensing choice) to the hiring entity. If you do work at canonical, i'm actually unclear what our formula licensing policy is.. ensemble itself is AGPL. [15:58] hazmat, no, i don't do work for canonical [15:59] daker, than your free to claim copyright solely for yourself if its an independent work, and license as you see fit, the canonical copyright statements are nesc in that case. [16:00] UGH.. sorry are not nesc in that csae [16:00] kim0, we should get this stuff in a FAQ and vetted [16:01] hazmat, good now http://paste.ubuntu.com/638471/ ? [16:02] daker, looks good [16:10] hazmat: thanks .. taking note and acting [16:10] <_mup_> ensemble/formula-upgrade-removes-obsolete r264 committed by kapil.thangavelu@canonical.com [16:10] <_mup_> dynamically computed manifests for formulas [16:18] kim0, also added links to your screencasts to the wiki front page.. i don't see an easy way to embed them directly without a moin plugin. [16:18] <_mup_> Bug #806037 was filed: FAQ needs updating for formulas licensing < https://launchpad.net/bugs/806037 > [16:18] hazmat: Yeah doing that one too [16:19] kim0, cool, i just finished adding them their near the bottom [16:19] ah ok [16:19] it would be nice to have them embedded closer to the top [16:19] but i think that requires some ops support, perhaps niemeyer would know more re the site setup and options for moin plugins [16:19] yes .. I looked at that before [16:19] it needs a moin plugin indeed [16:20] I think we're getting a new site that's more than a wiki [16:20] so that's probably enough for now [16:21] I saw people working on a new website during the past sprint [16:26] hmm.. i think i'm making this way more complicated than needed (clean formula upgrades), i could just remove the whole formula dir instead of just the stale entries, and extract the new [16:27] hazmat: does that look good http://paste.ubuntu.com/638479/plain/ [16:28] btw you can buy the cloud portal from me :D [16:28] Mostly punctuating your wording :) [16:28] daker: lol :) [16:28] hehee [16:29] kim0, looks fine to me, but it would be nice to have this vetted by the org [16:29] hmm ok [16:33] jamespage: hey, trying out your new jenkins formulas. i've probably forgotten how to do it right, but i'm not seeing any plugins available? [16:33] (wanted to test with git) [16:34] hmm I tried robbie, but he seems to be OoO till next week .. was wondering who can approve this formula licensing question .. any thoughts are welcome [16:35] kim0: clint and dustin had some examples [16:35] (biab) [16:37] guess I'll just push the faq entry in the review queue for now .. then once vetted can be merged [16:37] m_3: thanks man [16:37] kim0: clint's updated mysql copyright... http://pastebin.com/X5EnhCxt [16:38] SpamapS: If you have a sample formula license for an independant contributor .. let me know so I'd link to it [16:38] m_3: problem is that it's copyright Canonical [16:38] m_3: which shouldn't be the case for most people [16:38] at least that's my current understanding [16:39] I'm basically trying to get approval over http://paste.ubuntu.com/638479/plain/ .. [16:41] kim0: yeah, that was my understanding... dunno who can approve [16:42] hazmat: There's a major revamp of the website front page coming [16:42] kim0 knows about it too [16:42] Yeah I told him [16:42] niemeyer, nice [16:45] serge_: are you just getting an empty list? [16:48] jamespage: yeah [16:48] (just commented in the bug) [16:49] serge_: its probably unrelated [16:49] sometimes the plugin manager can be a bit flakey [16:49] ok [16:50] so, i could add a relation, destroy the relation, and i can destroy the jenkins-slave service. [16:50] BUt what is the ensemble command to just remove one machine from a servcie? [16:51] serge_: remove-unit [16:52] jamespage: d'oh! i was looking at the ensemble help output list, and somehow never saw that one [16:52] thanks! [16:53] is there an easy to set up distcc plugin? so i can fire up 3 slaves and get fast kernel compiles? [16:54] (that's a jenkins q, not ensemble :) [16:55] serge_: nope - you would have to setup distcc sep and then call from jenkins [16:57] SpamapS: around? [17:01] serge_: BTW I added a bit of intelligence into the slave setup - it now calculates the number of executors based on the number of cores the system has [17:02] jamespage: SpamapS is out this week [17:02] m_3: bah [17:02] not sure why that did not show on my calendar [17:02] oh - it did I just missed it :-) [17:03] jamespage: dunno... he was gonna have spotty coverage.. in-laws in central europe [17:03] its no problem really [17:03] I've been working on upgrading the zookeeper package in Debian/Ubuntu and wanted to catchup [17:04] serge_: I can't reproduce your issue - I can see plugins just fine. [17:05] jamespage: excellent [17:13] jamespage: my new instance has plugins. yay! (wonddr what happened before) [17:13] jamespage, awesome re zk, it sounded like the debian version was going to need a new maintainer (the old one bowing out), the latest release of zk is packaged in zk, its a pretty minimal diff to the packaging of the older version in debian [17:13] serge_: hmm - sometimes it take a while to scan the update site [17:14] hazmat: well I'm the new maintainer for my sins :-) have the package prepped and have done a basic test via PPA [17:14] jimbaker: hazmat bcsaller niemeyer SpamapS m_3 .. Could you guys please send me your take on Ensemble happenings of past week. Any decisions at the sprint? any new bits, formulas or bugs that you hacked, please send me asap so I can tell the world about it :) [17:15] This also goes to anyone in this channel .. let me know cool things you hacked [17:15] jamespage, right on.. now i know who to bug ;-) [17:16] * jamespage is looking forward to being bugged already :-) [17:17] serge_: hey - one other thing - the python-jenkins package is in the new queue for oneiric [17:17] kim0: will do [17:17] m_3: Thanks man :) [17:17] I've stuck it in a PPA for natty; but I guess once we can use oneiric AMI's we should update the formula to check [17:17] * kim0 poinst to the dev team :) [17:17] points even [17:17] jamespage: cool. any roadblocks right now to its inclusion? [17:18] serge_: I don't think so (fingers crossed :-)) [17:28] <_mup_> ensemble/formula-upgrade-removes-obsolete r265 committed by kapil.thangavelu@canonical.com [17:28] <_mup_> stale formula files are removed on formula upgrade. [17:31] hazmat: about removing stale formulas... [17:31] hazmat: what does that do to a rolling migrations? [17:31] m_3, its more about removing stale dirents in a formula when upgrading [17:31] m_3, nothing [17:31] s/migrations/migration/ [17:31] ah, ok... misunderstood then [17:32] m_3, for rolling upgrades, i'm thinking its something that's more formula implemented with ensemble providing a cli api for something like leader election (which is also useful for other contexts) [17:33] although that has implications in that implies potentially long running hooks. [17:33] hazmat: just was imagining a situation where there are multiple versions deployed at the same time [17:34] hazmat: yeah, we're still just hitting the surface of the tools (like leader-elect) needed by formulas [17:34] m_3, indeed that is possible with the current upgrade scheme, each of the units is upgrade independently although concurrently in practice, we stop hook execution during upgrades atm [17:35] m_3, yeah.. i wrote up some docs on more advanced upgrade scenarios, which aren't covered by the simple formula upgrade that's in place right now [17:35] hazmat: thanks man... still learning and need to test these scenarios [17:35] https://ensemble.ubuntu.com/docs/upgrades.html [17:35] hazmat: cool thanks! [17:35] m_3, the current upgrade mechanism is mainly about supporting the formula development/iteration story [17:36] hazmat: right [17:36] at least that was the primary use case i was targeting, simple service upgrades are possible with it, but more complex ones will definitely find it lacking without additional support [17:37] i think effectively we'll need to support perhaps some different policies, like rolling migration, or upgrade one with the rest stopped, etc [17:37] hazmat: yeah, I started thinking about this in your 'Service Upgrades' section [17:38] hazmat: with a rails app that pulled from source too... so even more complex [17:38] kim0: Maybe put it online somewhere so people can review what you have at the moment [17:38] kim0: I think you were present in most of the key decisions we've made [17:39] hazmat: agree about policies... until then, we'll try to produce example scenarios [17:41] kim0, i've mostly been doing bug fixes, but all my work from last week is still in the review queue, the highlights are more robust handling of transient network issues, ability to use standard ubuntu images (really any cloud-init enabled ubuntu/deb image), hooks executing inside of formula dirs, peer relations automatically added on deploy of a service, fixes for ensemble status with peer relations.. that's all i recall atm [17:42] m_3, yeah.. even a simple wordpress service with a db migration runs into this [17:43] hazmat: most of the issues mentioned on your last post affect me. Any idea when will the changes go through ? [17:43] negronjl, as soon as they get reviewed [17:43] hazmat: lol... I guess the question is when will they be reviewed [17:44] negronjl, i [17:44] negronjl, btw end of the week at latest i'd assume, they would have gotten reviewed last week except for the sprint/rally i think [17:44] what does this error mean? "2011-07-05 13:44:13,333 ERROR Connection was refused by other side: 111: Connection refused." [17:45] zul: ensemble is not ready for you yet :) [17:45] zul, typically it means ssh isn't active on yet on the remote side even though the machine is up [17:45] negronjl: well duh :) [17:45] kim0, oh and i wrote a security specification for how we'll be approaching security wrt to zk === sidnei is now known as sidnei-away [17:45] zul: wait longer [17:45] hazmat: so wait and try agian? [17:45] negronjl: Today [17:45] negronjl: I mean, they'll be reviewed today, at least once [17:45] niemeyer: thx...that's blocking a few things on my side. [17:46] zul, yeah.. which command where you running? i think we try to auto-retry most of the time [17:46] hazmat: ensemble bootstrap [17:46] zul: This is a known issue, actually.. it should really be waiting by itself [17:46] hmm... that's very odd [17:47] bootstrap doesn't talk to any of the ensemble machines.. just the machine provider [17:47] hazmat: thanks for the list .. that's awesome :) [17:47] zul, is this against openstack? [17:47] hazmat: yep but there is somethign wrong with my install though euca-describe-images is taking a long time [17:47] ic [17:48] i think i found it [17:51] niemeyer, re populating hostname automatically for relations, doing so effectively assumes its static, and if that's the case we might as well populate it directly to the machine state, so the unit agents can avoid having to query the machine provider [17:54] hazmat: Why does it assume it's static? [17:55] niemeyer, hmm.. good question.. we don't really capture any event for the unit to correct it upon change [17:56] hazmat: If the hostname in a relation change, things should just work normally.. relation-changed hooks firing, etc [17:57] niemeyer, the question is how do we know it changed to update the relation data [17:57] hazmat: That's a separate problem [17:57] its effectively an internal detail of the machine provider [17:57] afaik most of them give out statics for the lifetime of the instance [17:58] subject to reuse upon termination [17:58] hazmat: It's not a detail of the machine provider.. it's a detail of the machine [17:58] hazmat: and the fact IaaS'es don't change it right now doesn't really affect the design [17:59] hazmat: It just feels like a pretty convenient way to get such information [17:59] hazmat: and one that allows for it to change as well, without any effort [18:00] niemeyer, agreed, we can propogate it nicely upon change if know about the change, but it sounds like your suggesting we cron to check on the machine.. else we don't have a way of detecting and propogating the change [18:00] hazmat: I'm suggesting a design that allows for us to change it.. I'm not suggesting we address the change right now [18:00] or cron internal to the agent [18:01] niemeyer, sounds good, so unit agents directly pulling ip address from the machine and populating hostname for relation data when instantiating the unit in the relation [18:01] hazmat: +1 [18:01] Man.. I almost feel bad to say that now.. damn Google. [18:02] <_mup_> Bug #806098 was filed: fact repository for Ensemble < https://launchpad.net/bugs/806098 > [18:05] is there a query syntax for launchpad bugs.. stuff like AND OR etc? [18:10] out to run some errands bbiab [18:11] hazmat: Don't think so [18:11] hazmat: But there are some advance options [18:11] hazmat: See ya [19:30] Got a broken pipe at home.. the joys of assembling a house together. [19:37] broken pipe .. for a moment I was thinking SIGPIPE :) [19:37] it's an actual pipe [19:53] kim0: Hehe, yeah [19:53] kim0: A real SIGPIPE [19:54] hehe :) [20:16] hazmat: ping [20:21] hazmat_: ping [20:21] Here's our weekly report .. edits are most welcome http://pad.ubuntu.com/ensemble-week-5th-July [20:22] niemeyer: pong [20:22] hazmat_: Hey [20:22] hazmat_: Was reviewing the session-event-handling conversation again [20:22] I'll publish in an hour .. make sure your edits go there [20:22] unity seems to have issues for me with multiple screens, i'm currently in a VT on my laptop working on my screen session from my desktop.. sadness [20:22] niemeyer: awesome [20:22] hazmat_: Regarding your questioning on [4], what's your take on it? [20:23] hazmat_: I read your answer on it, but it's not clear what you have in mind to sort out the fact there's background logic that won't complete [20:24] kim0: Cool, thanks [20:24] great [20:24] kim0: The LXC stuff should probably not be there.. [20:24] too early ? [20:24] kim0: At least not with that wording [20:24] niemeyer: so there are two options, leave it as is.. or on fatal errors (session expired) propogate to both [20:24] kim0: People will want to try that, and it's not integrated yet [20:25] kim0: You can mention the experiment was successful and all [20:25] both being the connection level handler and the extant deferred [20:25] ok, I'll change to "work has started on ..." [20:25] niemeyer: the thing is we don't really expect the extant watchers to do anything except die, which makes for some additional unhandled error logging, or storm of such when those events happen [20:26] hazmat_: This is the problem.. [20:27] hazmat_: The logic simply _stops_. There's no chance for error handling or anything [20:27] hazmat_: Like, oh, if that doesn't work I'll log a warning, or.. if that doesn't work, remove the temporary directory, or anything really [20:27] the connection level handlers allow us to centralize the handling, which i see as a good thing, watch level error handling is going to be more error prone imo and its going to be subject to stampede effects [20:28] hazmat_: I can accept that as an ugly hack we're putting in place knowing it's a terrible way to program. [20:28] hazmat_: and with a known way out [20:28] niemeyer: how about i put an option to allow for either [20:29] ie. fatal session events propogated to watchers [20:29] those deferreds are effectively dead in that case.. the problem is that its tricky, the fatal error might come in response to a synchronous call not tied to a session event [20:29] hazmat_: But it's important to make it very clear that killing _any_ action is simply not feasible.. it's like crashing, except the application continues running [20:29] er. async call that is [20:47] <_mup_> Bug #806184 was filed: peer interface relationship data not propagating < https://launchpad.net/bugs/806184 > [20:48] hmm [20:53] Disregard bug: 806184 [20:54] I marked it invalid. [22:15] negronjl: Cool [22:16] I've been talking to hazmat_ for something like two hours already.. we'll write down some notes when we're done [23:01] <_mup_> Bug #806241 was filed: It should be possible to collocate service units < https://launchpad.net/bugs/806241 > [23:30] is there any way of having a service not start after the install hook? other than having a noop install hook? [23:41] adam_g: ensemble is just executing the named hooks, what you do in them is up to you [23:41] adam_g: some packages automatically start the service/daemon after install a pkg [23:42] adam_g: you can switch them to be manually started with an upstart override and set to manual.. or just stop them after install (if its the package) [23:42] probably a few other ways