/srv/irclogs.ubuntu.com/2011/07/28/#ubuntu-ensemble.txt

SpamapShaha00:00
jimbakeri'm pretty much convinced that Twisted is the best choice for this in the python frameworks00:00
SpamapSIt drives me *batty*00:00
jimbakerSpamapS, oh sure, it definitely does do that00:02
jimbakeri personally think that we could write equally robust and deterministic tests using threaded code, along with some conventions. the advantage over twisted would being able to use any arbitrary library. but we would have to come up with some good conventions that at the end of the day do look like go00:03
SpamapSGo is a nice idea.. but at what point do you stop forward progress to rewrite in it?00:04
jimbakerSpamapS, progress is so slow right now because testing is currently too hard00:05
jimbakerso i think that answers it for me00:05
SpamapSI found it rather easy to do tests for LXC.. but.. thats because I made LXC block.00:05
jimbakerSpamapS, correct. blocking is much easier to work with. that's why i mentioned using threads. (i was thinking of the paper you shared)00:06
SpamapSYeah it pretty much blows up all of the arguments I've heard for event based programming00:08
jimbakerthe inline callbacks model and how it supports coroutines through a trampoline basically gives us what we want there, except that constantly need to remember our calling conventions. so stuff randomly blows up if we forget. and we lose the stack.00:11
jimbakerin comparison goroutines are very nice00:12
SpamapSWhats the obsession w/ concurrency in a single process? I honestly can't see more than 1000 things happening at any one time in any one of the agents or clients... thats well within the capabilities of multi-processing.00:13
jimbakerwell, i'm quite certain that multiprocessing would be the wrong way to architect it00:16
jimbakerwe have i/o concurrency here00:16
jimbakerso not compute bound00:16
kirklandm_3: around?00:20
_mup_ensemble/robust-hook-exit r291 committed by jim.baker@canonical.com00:24
_mup_Doc strings and comments about exit vs end00:24
_mup_ensemble/robust-hook-exit r292 committed by jim.baker@canonical.com00:25
_mup_Merged trunk00:25
_mup_ensemble/robust-hook-exit r293 committed by jim.baker@canonical.com00:39
_mup_PEP800:39
_mup_ensemble/trunk r286 committed by jim.baker@canonical.com00:43
_mup_merge robust-hook-exit [r=niemeyer,hazmat][f=810808]00:43
_mup_Distinguishes between a process exits and the process has closed its00:43
_mup_file descriptors, so that hook output is captured properly without00:43
_mup_unnecessarily waiting on misbehaving hooks with child processes that00:43
_mup_haven't closed their FDs.00:43
jimbakernegronjl, fix released into trunk00:46
negronjljimbaker:  Thanks man00:47
jimbakernegronjl, cool, hopefully this works well as we expect for cloudfoundry00:47
_mup_ensemble/states-with-principals r299 committed by kapil.thangavelu@canonical.com01:40
_mup_integrate principal creation/group management into relevant domain states (service, unit, machine)01:40
hazmatjimbaker, i think the greenlet approach might be a bit nicer01:42
jimbakerhazmat, the problem i see with that is you cannot switch across threads with greenlets. but it's required to use different threads when working with ZK (or at least the current c api)02:06
jimbakerobviously it can be made to work, but so much of the greenlet advantage is seen when using switch02:07
m_3kirkland: hey man... ssup?02:28
hazmatjimbaker, you can definitely switch across threads with greenlets02:49
hazmatjimbaker, you can wait on another greenlet to finish and capture its return value and you can yield control of a thread02:49
sidneiSpamapS, do you happen to be around?03:16
jimbakerhazmat: http://packages.python.org/greenlet/#greenlets-and-python-threads - "Greenlets can be combined with Python threads; in this case, each thread contains an independent "main" greenlet with a tree of sub-greenlets. It is not possible to mix or switch between greenlets belonging to different threads."03:44
SpamapSsidnei: here now, sup?03:50
sidneiSpamapS, oh, was checking #736216, then i realized i enabled proposed but never actually installed the package from proposed.03:51
_mup_Bug #736216: bzr crashed with error in _curl_perform(): (28, 'SSL connection timeout at 298225') connecting to Launchpad <amd64> <apport-crash> <error-reporting> <https> <natty> <running-unity> <verification-needed> <Bazaar:Invalid> <Launchpad itself:Invalid> <bzr (Ubuntu):Invalid> <curl (Ubuntu):Fix Released> <bzr (Ubuntu Maverick):Invalid> <curl (Ubuntu Maverick):In Progress> <bzr (Ubuntu Natty):Invalid> <curl (Ubuntu Natty):Fix Committed>03:51
sidneiincidentally seems like niemeyer was hitting it today with bzr03:52
SpamapSsidnei: Cool. bzr has a micro-release exception, so not all the bugs have to be verified. The package just has to sit in -proposed for 7 days and pass the test suites.03:54
sidneiSpamapS, yes, but the bug is in curl, and bzr is affected due to python-pycurl linking against libcurl3-gnutls03:55
SpamapSOH03:55
sidneianyway, i enabled proposed but forgot to install the upgrade from proposed until today. seems like the updated package fixes the problem, but i will leave landscape-client running overnight and check the logs in the morning to be real sure.03:57
=== daker_ is now known as daker
=== kooolhead11|afk is now known as kooolhead11
* hazmat wonders if his diy nas will boot13:51
jcastroSpamapS: good luck today on your talk, knock em dead!14:01
kooolhead11hi all14:02
kim0hazmat: I'm interested to know more about ur nas14:04
hazmatkim0, nutshell. recycling old diy desktop (core2duo + 4g ram), into a cooler master centurion 590 case, a pair of 5x3 drive cages, and 8 port sata controller, i'm pretty set on a zfs setup at this point (freenas), although i'm going to see if i can test btrfs before i do. sadly my first problem just cropped, need an add on video card, no mb graphics.14:07
kim0hazmat: was just checking out btrfs since I converted my /home to that (getting too many bad sectors lately). so btrfs can't raid5/6 yet, no working fsck, and no raid conversion yet. Otherwise, seems functional :)14:09
niemeyerGood mornings!14:10
kim0Morning 14:11
fwereadeniemeyer: heyhey14:11
hazmatkim0, i have hopes for btrfs, and keeping check in on the dev, but it still seems a bit experimental for trustworthy storage.. i'm planning on using  raidz2 (3 drive fail to loose data) mode of zfs across eventually 14 drives (starting with 6 atm), in two vdevs of 5-7 each14:16
kim0yeah although fedora is switching to it, I think it's too early to trust it with family photos :)14:17
SpamapSjcastro: thanks.. I may still change the pics if I have some time.. but for now.. its lolcats ftw14:19
jcastroSpamapS: heh, when in doubt, go with what you know14:19
kim0hehe14:19
niemeyerhazmat: Physical drivers?14:50
niemeyerhazmat: Physical drives?14:50
hazmatniemeyer, 6 atm (5 2tb, 128 gb ssd from old laptop).. total capacity is 14 with the 8-port sata card (not raid) and 6 mb sata slots14:51
hazmatas far as picking a sata controller with good foss driver support, i found this blog post very helpful.. http://blog.zorinaq.com/?e=1014:53
fwereadehey all, I'm confused about something14:53
hazmatniemeyer, re security discussion from yesterday, let me know when you've got time to pick it back up.. 14:54
hazmatfwereade, what's up?14:54
fwereadeyou know we get authorized keys and send them to instances... what do we actually use them for?14:54
fwereadeI thought we were meant to be doing all communication via ZK?14:54
hazmatfwereade, ssh into the machine14:54
niemeyerhazmat: Ok.. I have time right now, but I think I'd like to spend some more time with it if you don't mind, so that I can be more helpful.14:54
fwereadeis it only a sort of emergency damn-I've-screwed-up channel14:54
niemeyerhazmat: I've thought further about the problem yesterday, and have some ideas, but these ideas have holes still14:55
hazmatfwereade, we use that for the debug hooks command and ssh, and in future scp14:55
hazmatniemeyer, i've got a nice solution14:55
niemeyerhazmat: I need to stand next to a white board for a moment14:55
fwereadehazmat: ah, cool14:55
niemeyerhazmat: Oh, what's that?14:55
fwereadehazmat: thanks :)14:55
hazmatfwereade, np14:55
hazmatniemeyer, well first.. i think its useful to think of the topology as an identity db, not particularly relevant to the solution.. but most sequence nodes don't have identity prior to their existance in the topology14:56
hazmatfor example look at a failure mode for creating a machine, it leaves an unused sequence node14:57
niemeyerhazmat: Ok14:57
hazmatniemeyer, so in particular, i just backed off doing acls for these nodes at creation time, and instead update the acl using policy rules after they've been added14:57
hazmatto the topology14:58
niemeyerhazmat: Yeah, that was the direction of my thinking as well14:58
niemeyerhazmat: Have you figured what's the problem with this too?14:58
hazmatniemeyer, haven't run into anything yet, do you foresee an issue?14:59
niemeyerhazmat: Yeah.. what's the reason why we create the entry before putting it in the topology?14:59
hazmatniemeyer, ah.. yes there is a gap i did notice that14:59
hazmatbefore we apply acls15:00
niemeyerhazmat: Right, we're going back to the original problme15:00
hazmatniemeyer, not really15:00
niemeyerhazmat: Why not?15:00
hazmatniemeyer, the one acl that the standard policy can put forth is an owner policy, which free us from worrying about an acl security gap, so we can modify it post topology add.15:01
niemeyerhazmat: THe reason why we create the entry before putting it in the topology is because once the topology is touched agents will notice that and will act accordingly15:02
hazmatniemeyer, as to your question of why create before topology, so we can have an atomic update of the topology which drives  identity/rel watches and that's its reflective of what's present15:02
niemeyerhazmat: So to make it "atomic" we change the topology once the system is ready to take it15:02
niemeyerhazmat: If we change in the suggested way, we'll put in the topology something that is not usable15:02
hazmatniemeyer, yeah.. the gap15:02
hazmathmm15:02
hazmatniemeyer, so that brings up back to nodes should have identity upon creation via contents, but that's complicated as well for the named principal which is going to be reference the sequence value15:05
niemeyerhazmat: Yeah, I feel like this is an ugly hack we shouldn't dive into15:06
hazmatniemeyer, alternatively we could write support for the new zk multi-node api into the libzk and zkpython15:06
hazmatniemeyer, yeah.. i'd prefer to avoid content sniffing as well15:06
hazmatbummer15:07
hazmatyeah.. a whiteboard sounds good15:07
hazmatand some coffee15:07
niemeyerhazmat: I think your suggestion is going into the right direction.. let's just ponder for a second on the new problem that arrives from that.  I have a vague idea, just need to see if it's reasonable or not.  Give me a few minutes to brainstorm on this15:08
hazmatniemeyer, so the failure modes on these from a topo conflict are pretty much non-existant15:10
hazmatits just a retry15:10
hazmatthe sequence nodes are guaranteed unique15:11
hazmatwe could have a local topology for the policy to utilize before saving it to the final topology, effectively making the acl update part of the topo change function, that finishes prior to return contents15:12
hazmats/finishes/modifies the acl prior to returning the topology contents15:12
hazmathmm.. it also needs to do the otp there, as the rules will likely reference15:14
niemeyerhazmat: Why?15:15
niemeyerhazmat: The OTP doesn't have to change15:15
niemeyerhazmat: on every iteration of the retry, that is15:15
jcastrojamespage: hey do you think it's feasable to replace pad.ubuntu.com with -lite deployed via ensemble? That would be a nice dogfood for next UDS.15:15
jamespagejcastro: hmm - maybe; its quite bleeding edge ATM and requires stuff which is not in the archive15:16
* jcastro nods15:16
jamespagejcastro: will probably hit the 'is-it-packaged' challenge 15:16
jcastroyeah but if we have an ensemble formula we should stop caring about that right?15:17
jamespagejcastro: :-)15:17
jamespagejcastro: well the formula pull in and builds quite a few bits and pieces not from Ubuntu15:17
jcastroI thought that was a feature, but yeah, I can see that.15:18
jamespagejcastro: just pushed an initial version of the formula BTW15:18
jcastrojamespage: I saw, that's what prompted me to ask.15:18
jcastrowell, we can certainly dogfood it during ensemble team events. :)15:19
jamespagedon't try it with haproxy yet......15:19
jamespagebut works OK with a mysql backend15:19
jcastrook15:19
jcastroyeah this is one of those that just will demo great and let people try it immediately.15:19
jamespageagreed15:21
hazmatniemeyer, true the otp identity is stable and doesn't ref the topology.15:39
_mup_ensemble/debug-log-relation-settings-changes r275 committed by jim.baker@canonical.com15:47
_mup_Merged trunk and resolved conflicts15:47
robbiewm_3: hey...did you ever hear back from the MediaWiki folks?16:12
m_3robbiew: nopw16:17
m_3I'll bug Ryan again16:17
robbiewm_3: cool...I think SpamapS was planning on stopping by the offices...but not sure when16:18
m_3yeah, #wikimedia-tech's quiet at the moment... no sign of ryan16:18
robbiewprobably OSCon related16:19
m_3hopefully16:19
m_3it's still early16:19
niemeyerLunch time16:27
=== kooolhead11 is now known as kooolhead11|afk
niemeyerfwereade: ping17:45
=== daker is now known as daker_
hazmatbcsaller, jimbaker, fwereade, niemeyer weekly meeting?17:59
bcsaller trying g+ again?18:00
niemeyer+118:02
niemeyerjimbaker: ping?18:05
jimbakerniemeyer, hi18:24
jimbakerniemeyer, so is this g+ meeting happening?18:30
niemeyerjimbaker: Yeah, it's rolling18:32
niemeyerjimbaker: We'll invite you18:32
jimbakerniemeyer, ok18:32
niemeyerjimbaker: What's your gmail address18:32
m_3SpamapS: break a leg!18:52
SpamapS:)18:53
SpamapSt minus 3 hours 16 minutes18:54
SpamapSerr18:54
SpamapS4 hours18:54
m_3time to enjoy the conf18:54
SpamapSs/enjoy/document/ ;)18:55
m_3niemeyer: please add me to the right lp group to edit the ensemble.ubuntu.com wiki when you get a chance18:55
niemeyerm_3: Done18:55
m_3thnsk18:55
jimbakerfyi, james.edward.baker@gmail.com19:18
hazmatniemeyer, one problem i have with current review queue stackand security.. the fixes for security-rules depend depend on stuff latter in the stack.19:45
hazmatthe rules are standalone as is, so i think addressing some of the comments, and then moving forward with the rest of it, till a latter point in the branch stack probably makes the most sense19:45
niemeyerhazmat: How do you mena?19:46
niemeyerhazmat: When there is a dependency chain like this, the natural thing to do is to review them in order, apply comments in order, and merge in order19:47
niemeyerhazmat: Jumping back and forth will make things more complex19:47
hazmatniemeyer, right.. security rules are an integration piece though, i'm doing integration work and branches right now, but the rest of the branches in the stack that's part of review, including those that follow security-rules are implementation focused. some of them may contain fixes that will be needed to finish up security-rules. currently its standalone, effectively unused (though tested), but unfortunately merged into the rest of the implementati19:49
hazmaton stack of branches19:49
hazmati guess its doable as is, it might just be a while till everything gets merged, which means i'll have a lot of pending branches19:50
niemeyerhazmat: Yeah, that's why I say that when the dependency has been introduced and they are in a chain, the natural thing to do is to apply reviews and move forward19:50
niemeyerhazmat: My suggestion is to get the bottom of the stack, fix it, and merge it19:50
niemeyerhazmat: and so on19:50
hazmatniemeyer, yeah.. its an artificial dependency at this point which is going to hold up the merge of the other 4 branches in review19:50
niemeyerhazmat: and then move forward with everything in trunk19:50
niemeyerhazmat: It is, but it's there..19:51
niemeyerhazmat: The problem you have is that you're trying to push things forward while there are reviews pending19:51
niemeyerhazmat: If you take care of the branches that are in review and get them merged, the problem disappear19:52
niemeyers19:52
hazmatniemeyer, perhaps.. the real problem is that its an unrelated branch, which introduced before the implementation of other features, ie. an artifical dependency... i probably shouldn't have pushed it to review.. but i wanted to get feedback on the approach... not a big deal, it can be redressed in the current process.19:54
jcastroany idea what could cause this? http://pastebin.ubuntu.com/653982/20:26
jcastrooh nevermind20:27
jcastroit's that minute delay thing20:27
fwereadegents, I'm very sorry I missed you20:40
fwereadein my defence, it did start almost 2 hours after the theoretical end of my day 20:41
niemeyerfwereade: Not a problem.. we were entirely aware of that20:42
niemeyerfwereade: We decided to go forward just because we've been missing it again and again..20:43
niemeyerfwereade: We should do another one tomorrow, or very soon anyway, with you20:43
fwereadeniemeyer: anything I should know from it?20:43
niemeyerfwereade: Nothing critical to what you've been doing.. hazmat is sending notes to the list20:43
niemeyerfwereade: Btw,20:43
fwereadeniemeyer: cool20:43
niemeyerfwereade: Just as a note for the start of your day tomorrow20:43
niemeyerfwereade: It looks like some of the branches you've been pushing for review are missing part of the workflow20:44
niemeyerfwereade: They're not showing in the Kanban20:44
niemeyerfwereade: No rush, but worth checking early tomorrow so they get into the review flow20:44
fwereadeniemeyer: hmm, lack of bugs perhaps?20:44
niemeyerfwereade: Quite possibly20:45
niemeyerfwereade: Or not assigned to the right milestone20:45
niemeyerfwereade: Or not in progress.. or.. :)20:45
fwereadeniemeyer: hmm, probably all 3 for most of them20:45
niemeyerfwereade: Again, no big deal.. just worth checking20:45
* fwereade looks shifty20:45
* hazmat lol20:45
fwereadeniemeyer: cheers :)20:45
niemeyerfwereade: Sorry for missing you on the meeting20:45
fwereadeniemeyer: no worries :)20:46
niemeyerfwereade: I hope we automate that at some point.. I'm conscious this procedure is too heavy20:46
fwereadeniemeyer: it's sweetness and light compared to what I'm used to :)20:47
fwereadeniemeyer: however, I think what I'm used to gave me a mild phobia of kanban boards20:48
niemeyerLOL20:48
fwereadeniemeyer: you'd almost think the software was written by anti-agile activists, it seemed to make a point of putting you off20:48
niemeyerfwereade: "Wait.. I see you've pushed too branches in a row.. why are you doing things so fast!?"20:48
fwereadeniemeyer: haha20:49
_mup_Bug #817732 was filed: can't communicate with an orchestra server running cobbler <Ensemble:New> < https://launchpad.net/bugs/817732 >20:54
niemeyerbcsaller: re. the order of things on LXC, some vague ideas:20:55
niemeyer<niemeyer> 1) Make multiple units work on a single machine across the board (no LXC)20:55
niemeyer<niemeyer> 2) Make local deployments work with one or multiple units (no LXC)20:55
niemeyer<niemeyer> 3) Make LXC work to deploy units 20:55
niemeyerbcsaller: Of course, each of these can be split further20:55
bcsalleryeah20:55
niemeyerbcsaller: But these high-level steps feel like making things more manageable20:55
bcsallersounds like a good place to start20:56
_mup_Bug #817735 was filed: cobbler system names can change <Ensemble:In Progress by fwereade> < https://launchpad.net/bugs/817735 >20:57
hazmatbcsaller, i'm writing up meeting notes, is your next post-config item spec for containers or co-location?21:00
bcsalleryes21:00
hazmat"or" }21:00
hazmat?21:00
hazmatbcsaller, both?21:00
bcsallerof, well, specs for both, but co-location is first21:00
hazmatbcsaller, cool, thanks21:00
_mup_Bug #817736 was filed: FileStorage interface is inconsistent <Ensemble:In Progress by fwereade> < https://launchpad.net/bugs/817736 >21:01
_mup_Bug #817738 was filed: duplicated code in ec2 LoadState and SaveState operations <Ensemble:In Progress by fwereade> < https://launchpad.net/bugs/817738 >21:03
_mup_Bug #817739 was filed: no FileStorage implementation for orchestra <Ensemble:In Progress by fwereade> < https://launchpad.net/bugs/817739 >21:05
_mup_Bug #817740 was filed: orchestra provider can't discover running zookeepers  <Ensemble:In Progress by fwereade> < https://launchpad.net/bugs/817740 >21:08
_mup_Bug #817743 was filed: orchestra provider can't launch an instance that does anything useful <Ensemble:In Progress by fwereade> < https://launchpad.net/bugs/817743 >21:10
fwereadeniemeyer: btw, re: orchestra development strategy21:16
fwereadeniemeyer: you're absolutely right about the shadow-trunk, that's a nicer model for future work21:17
fwereadeniemeyer: but I think I'm missing something: how does that help us help us transition the code already in adres' branch?21:18
fwereadeniemeyer: well, I guess it doesn't have to21:18
fwereadeniemeyer: it's just that it works, and I'd prefer to transition the working ocde smoothly if I can21:18
fwereadeniemeyer: ...and as soon as I say it in public, it seems much dumber than it did in my head21:19
fwereadewith one more branch (in progress), I think we'll have something that's a viable base for RoAkSoAx to branch from21:20
niemeyerfwereade: There's no way to transition his work into something compatible with trunk in a mergeable way besides doing it piece by piece21:23
fwereadeniemeyer: yep, I think you're right, it just seems like a shame :)21:25
niemeyerfwereade: Spikes are spikes.. we have to recognize them by the benefit they provide, but transitioning from a spike into a stable implementation takes known pain21:26
fwereadeniemeyer: and, to be fair, less total pain than "it works? SHIP IT!"21:27
niemeyerfwereade: Exactly21:27
* fwereade has an uncomfortable feeling he may have quoted directly from someone at his first job21:30
robbiewlol21:31
SpamapScrazy idea22:00
SpamapSwe should have a "sneakernet" provider22:00
SpamapSall it does is create nodes in zookeeper...22:00
SpamapSand feed you back admin credentials to spawn machine agents22:01
SpamapSthat way you can join existing machines to ensemble22:01
niemeyerSpamapS: As a first step, defining a formula for an external service would probably work22:07
robbiewSpamapS:  done your talk yet?23:05
adam_grobbiew: starts in 2 min23:09
robbiewadam_g: ah...how's the attendance?23:10
robbiewI noticed it was still in the OS track b/c of the previous upstart topic23:11
robbiewnot cloud :/23:11
_mup_ensemble/debug-log-relation-settings-changes r276 committed by jim.baker@canonical.com23:40
_mup_Fixed faulty merge23:40

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!