=== andrewsmedina_ is now known as andrewsmedina
=== andrewsmedina_ is now known as andrewsmedina
TheMuerog: morning08:54
rogTheMue: hiya08:54
=== Leseb_ is now known as Leseb
TheMueniemeyer: hiya13:25
TheMueniemeyer: my propose of the continued machines based on yesterdays charms (has been a branch of that) is in13:25
niemeyerTheMue: Thanks13:33
niemeyerTheMue: I may only get to it tomorrow13:33
niemeyerTheMue: I'll try to stick to store work today, since I've been clearly failing at it13:33
TheMueniemeyer: ok, no prob, better keep concentration13:34
TheMueniemeyer: would part of state should I take next?13:34
rog(outside in the sun!)13:39
niemeyerTheMue: Not sure..13:43
TheMuerog: enjoy. sun? (envy) here it is grey and wet13:43
niemeyerTheMue: Actually, wasn't there something to be done as a left over from the current work?13:43
niemeyerI recall something like that, but can't remind of it now13:43
niemeyerTheMue: Ah, yes.. zk => st13:44
TheMueniemeyer: state into all entities, but that's only very small13:44
niemeyerTheMue: Yeah, but has to be done13:44
TheMueniemeyer: yep, exactly13:44
niemeyerTheMue: Besides that, just pick something that moves the state forward13:44
TheMueniemeyer: ok, will do so13:44
niemeyerTheMue: Preferably in areas that do not conflict with changes going on in Python (relations)13:45
niemeyerandrewsmedina: Any progress there?13:45
andrewsmedinaniemeyer: yes13:45
niemeyerandrewsmedina: Nice13:45
TheMueniemeyer: yep, no relations13:45
andrewsmedinaniemeyer: I'm finishing the tests for port charm metadata subordinate to Go13:46
niemeyerandrewsmedina: Very neat13:55
niemeyerandrewsmedina: Thanks for working on that13:55
andrewsmedinaniemeyer: :D13:58
rogwondering if anyone is up for sanity-checking a possible approach to making my ec2test fake ec2 server simulate eventual consistency issues14:26
rogfwereade, jimbaker?14:26
fwereaderog, I'm trying to finish something off atm, I'll ping you when I'm free14:27
rogfwereade: thanks14:27
=== Leseb_ is now known as Leseb
niemeyerLunch time here..15:50
fwereaderog, sorry that took so long; can I be of service?15:53
rogfwereade: yes, thanks. gimme 1 minute.15:54
rogfwereade: bakc15:56
rogfwereade: so...15:56
rogfwereade: the object is to simulate the "eventual consistency" semantics of the ec2 server.15:56
rogfwereade: to that end, the idea is that all operations operate on an out-of-date version of the server data.15:56
rogfwereade: when a client makes a call, if it's a read-only request, we return the results from the out-of-date version.15:57
rogfwereade: if it's a modification request, we allocate any unique ids (instance ids, group ids etc) from the current version. then we queue up the operation to be executed later.15:57
rogfwereade: after every call, we execute one operation from the queue.15:57
rogfwereade: does that sound reasonable to you?15:57
fwereaderog, yes, I think so15:57
rogfwereade: i can't help feeling that there might be a large bear lurking in the room somewhere15:58
rogguess i can only try and see15:59
fwereaderog, I'm wondering though whether it might be good to have some way of varying the behaviour -- sometimes immediate consistency, sometimes irritatingly delayed15:59
rogfwereade: the idea is to allow the length of the queue to be specified when the server starts15:59
rogfwereade: i don't think having random behaviour is going to be good for reproducing failed tests15:59
fwereaderog, the trouble is you don't *really* want to do that *randomly*, or you're on a miserable path to unreproducible test failures15:59
fwereadesnap :p16:00
rogfwereade: so i think we'd try all tests with "totally consistent" and "very delayed" behaviour16:00
rogfwereade: one question i can't quite decide on is if there are significant tests that might fail with random consistency but neither of the above scenarios16:01
fwereaderog, that's what's bugging me16:01
fwereaderog, I can see a plausible route towards repeatable random tests16:01
jimbakerrog, this sounds like what we do in the testing in the python code. not uniformly mind you16:02
jimbakerin terms of delays16:02
rogfwereade: what about concurrency?16:02
fwereaderog, where we store the seed that generated the queue-frobbing and can attach that16:02
rogfwereade: that assumes we generate requests deterministically.16:03
fwereaderog, ok, close-enough-to-repeatable to probably be a bit easier to reproduce16:03
rogfwereade: hence my question "what about concurrency"?16:03
fwereaderog, quite so; it may very well be that the difference is slight16:03
rogfwereade: i dunno.16:03
fwereaderog, and doesn't justify the effort16:03
rogfwereade: the other thing is that the above approach doesn't work if you can jump backwards and forwards in time.16:04
fwereaderog, in what sense?16:04
rogfwereade: because you need to keep several versions of the server state16:04
rogfwereade: with this approach we just keep one version and apply deltas16:04
fwereaderog, ah got you16:04
rogfwereade: hmm16:05
rogfwereade: although16:05
rogfwereade: if delta application is deterministic16:06
rogfwereade: then we can keep several versions and apply deltas to each of them as appropriate16:06
fwereaderog, sorry, I think I'm lost...16:07
rogfwereade: we could have several versions of the state, each representing the state at some point within the queue of pending operations16:07
rogfwereade: as deltas move through the queue, they get applied to each state in turn. or maybe not in turn... to simulate the random ordering16:08
rogfwereade: assuming s1+delta1+delta2 == s1+delta2+delta1 it should be ok16:09
fwereaderog, yes, I understand that bit; ah, yes, ok, we can have a separate queue of operations for each... er, time-slice of the server16:09
rogfwereade: unfortunately that last isn't quite true, although it is for most things AFAICS16:09
fwereaderog, thinking more though; it might be good to propose it with just the discussed boundary-condition checks, which will be useful if not complete16:10
rogfwereade: for instance, if the user does AddSecurityGroupAuth followed by RevokeSecurityGroupAuth, what's the result if the ops get reordered?16:10
fwereaderog, finding some happy medium between randomness and repeatability can wait16:11
fwereaderog, I have no idea, and that worries me16:11
rogfwereade: yeah, i think so. and that probably makes it easier to do something more sophisticated later on if we deem it useful16:11
rogfwereade: i *wish* there was some documentation on what semantics you can expect from ec216:11
fwereaderog, hopefully16:12
rogfwereade: i can't find anything at all. even discussions.16:12
rogfwereade: (which i find hard to believe - surely people encounter this stuff all the time?)16:12
fwereaderog, I think the semantics are "you'll get what you're given, and you'll like it"16:12
jimbakerrog, maybe take a look at http://www.allthingsdistributed.com/ ?16:12
jimbakeralso there is some docs on expected semantics for s3, fwiw16:13
rogjimbaker: did you have any particular blog post in mind?16:14
fwereaderog, heh, what if it's just that it's a *really* rare and unreproducible occurrence, so nobody ever hears about it or quite believes it when it happens?16:14
rogfwereade: it's not rare!16:14
* fwereade dons tinfoil hat16:14
jimbakerrog, unfortunately no, but it is the sort of thing werner discusses on his blog16:14
rogfwereade: it happens a lot of the time in my tests16:14
rogjimbaker: ok, thanks.16:14
fwereaderog, ah, ok, I didn't realise16:14
rogfwereade: otherwise i don't think i'd've know16:15
rogfwereade: one minor comment on https://codereview.appspot.com/5672067/16:43
fwereaderog, not convinced by that; how about a "checkedClock" (or something) field on the type, as a compromise?16:45
rogfwereade: i'm not sure that's right. if the file did not at first exist, you never need to check the clock.16:46
fwereaderog, hm, good point16:46
fwereaderog, Idon;t really like the idea of threading it through all those calls though :/16:47
rogfwereade: i think that the calls in Alive and AliveW are in the unique position of knowing that the clock needs to be checked, hence my suggestion.16:47
rogfwereade: it's only two levels...16:47
* fwereade looks at the code again16:47
fwereaderog, yeah, it turns out ok16:56
rogfwereade: cool16:56
fwereaderog, how about calling setAlive setState instead? it now always sets both .alive and .timeout16:56
* rog has a look16:58
rogfwereade: seems reasonable. (although i think it always has set both alive and timeout)17:01
fwereaderog, it went through a brief recent incarnation in which it didn't even bother to read the timeout more than once17:02
rogfwereade: ah17:02
fwereaderog, changing it back made the name change seem like a good idea17:02
rogfwereade: yeah, i think setState is a better name for what it does17:03
fwereaderog, cheers, reproposed ;)17:04
fwereaderog, EOD I'm afraid; if you have a spare moment I would appreciate it if you would have a little look at https://codereview.appspot.com/5695082 and let me know the perceived crackfulness of the general idea17:08
rogfwereade: cool!17:08
rogfwereade: yes, i'll do that17:09
fwereaderog, er, expect another one in a sec, I appear to have ballsed up a bit17:09
rogfwereade: looks better with firstTime, to my mind - that logic becomes a bit more obvious.17:09
rogfwereade: thanks for changing it.17:09
fwereaderog, a pleasure :)17:11
fwereaderog, huh, no I didn't balls it up; somehow I was trying to link against something stale, I guess17:12
fwereaderog, so don't expect an update soon17:13
rogfwereade: ok. i'll try to ignore the extraneous diffs17:13
rogfwereade: where's the implementation of relation-get etc again?17:14
fwereaderog, in python?17:14
* fwereade racks brains17:14
fwereaderog, juju.hooks.commands17:14
rogfwereade: no, that wasn't the question.17:15
rogfwereade: i was wondering what files the new code for " Rough proposal for implementation of relation-get and similar commands." was in17:15
rogfwereade: presuming it's somewhere in that CL17:15
fwereaderog, ah sorry: none of them are *implemented* yet17:15
rogfwereade: server/protocol looks new17:15
fwereaderog, but it provides a mechanism by which a command like relation-get can get the unit agent (which knows all the appropriate state) to execute the command for it and pipe back the results17:16
rogfwereade: yeah, that's what i was looking for. that's in cmd/server, right?17:16
fwereaderog, yeah17:17
rogfwereade: instant initial reaction: why not use the rpc package?17:18
rogfwereade: rather than doing our own marshalling etc17:18
fwereaderog, huh, because I ballsed up on due diligence :p17:18
rogfwereade: i think this code can be really really small17:19
fwereaderog, then that's doubly awesome17:19
fwereaderog, it was very instructive writing it all but less code is always better than more code ;)17:19
rogfwereade: the unix socket stuff will stay :-)17:19
fwereaderog, cool17:20
rogfwereade: but yeah, i think a few methods exported on a relation type and passed to rpc.Server.Register will probably do the job17:22
rogfwereade: the only thing that won't give you will be commands that stream.17:23
rogfwereade: are there any like that?17:23
fwereadefwereade, some certainly dump data back on stdout17:23
fwereadeer, *rog*, ^^17:24
rogfwereade: lots and lots of data? or don't-care-too-much-if-it's-all-in-memory data?17:24
fwereaderog, and in general I'd expect/hope that --verbose and --debug should work (and thereby potentially generate monster stderrs too)17:25
fwereaderog, not sure how big it could get17:25
rogfwereade: i've an idea for how that could work ok with rpc17:26
fwereaderog, well, we could do exactly the same in terms of how we pipe stuff back17:27
rogfwereade: yeah17:27
fwereaderog, the streams feel to me like a good thing17:27
rogfwereade: if you're around for a minute or so more, i'll paste a little example17:27
fwereaderog, please do17:28
rogfwereade: http://paste.ubuntu.com/860798/17:31
rogfwereade: slightly contorted by the necessary form of rpc methods17:31
rogfwereade: but the idea is that you call New which gives you a handle on a running command. then you can call other methods to get info on that command, including reading from its output streams17:32
rogfwereade: Cmd is the type that would be registered as an rpc handler17:32
rogfwereade: maybe Cmd.Result might be better named Cmd.Wait17:33
rogfwereade: and if we were concerned about malicious clients, we could put a random capability string in the Running id.17:34
=== Leseb_ is now known as Leseb
fwereaderog, seems less clear to me but that's probably a matter of familiarity; I shall meditate upon it17:38
rogfwereade: i'll quickly sketch an implementation17:39
rogfwereade: i think it'll be pretty small.17:39
fwereaderog, don't rush, it has been noticed by others that it's my eod17:40
rogfwereade: no probs.17:40
rogfwereade: i've only got 20 minutes myself...17:40
rogfwereade: see ya tomorrow morning17:40
fwereaderog, and you :)17:41
andrewsmedinaniemeyer: hi18:39
niemeyerandrewsmedina: yo18:41
andrewsmedinaniemeyer: for port this test (http://dpaste.org/9MrTY/) I need create another dummy charm?18:43
niemeyerandrewsmedina: You can copy and change an existing one.. some tests already have the needed utilities to do that easily18:44
niemeyerandrewsmedina: Hold on.. I'll find an example18:45
andrewsmedinaniemeyer: dont have yet in Go a equivalent for change_sample18:46
niemeyerandrewsmedina: We actually do18:46
niemeyerandrewsmedina: Check charm_test.go18:46
niemeyerandrewsmedina: If you have a reader r, you can do something like18:47
niemeyerhackYaml := ReadYaml(r)18:47
niemeyerhackYaml["subordiate"] = true18:47
niemeyernewr := hackYaml.Reader()18:48
andrewsmedinaniemeyer: I will try this18:49
niemeyerandrewsmedina: It's not as practical as change_sample yet, but feel free to implement a function so that it is18:49
niemeyerandrewsmedina: For example: changeSample(func(yh *yamlHacker) { yh["subordinate"] = true })18:49
andrewsmedinaniemeyer: is missing only this use case18:51
niemeyerSummer storm..19:07
andrewsmedinaniemeyer: can you help me?22:39
niemeyerandrewsmedina: Maybe.. :) What's up?22:39
andrewsmedinaniemeyer: All tests are ok22:40
andrewsmedinaniemeyer: but22:40
andrewsmedinaniemeyer: in python (http://dpaste.org/9MrTY/) when a charm is not a proper subordinate, is returned the charm path in the excetion22:42
hazmatandrewsmedina, and?22:47
hazmatthis is problematic because?22:47
andrewsmedinahazmat: I dont have path in go implementation22:48
niemeyerandrewsmedina: Sounds fine.. just don't return it22:48
andrewsmedinaniemeyer: I return a simple error message?22:49
niemeyerandrewsmedina: Simplicity FTW22:49
andrewsmedinaniemeyer: I will send the commit in minutes :p22:50
niemeyerandrewsmedina: Superb :)22:50
andrewsmedinaniemeyer: sorry for my noob questions22:50
niemeyerandrewsmedina: Totally fine22:50
andrewsmedinaIm a noob in go =/22:50
niemeyerandrewsmedina: Not to worry.. there are a handful of people only that know Go for more than a couple of years, so you're not too far behind :)22:53

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!