| === andrewsmedina_ is now known as andrewsmedina | ||
| === andrewsmedina_ is now known as andrewsmedina | ||
| TheMue | rog: morning | 08:54 |
|---|---|---|
| rog | TheMue: hiya | 08:54 |
| === Leseb_ is now known as Leseb | ||
| niemeyer | Alow | 13:15 |
| TheMue | niemeyer: hiya | 13:25 |
| TheMue | niemeyer: my propose of the continued machines based on yesterdays charms (has been a branch of that) is in | 13:25 |
| niemeyer | TheMue: Thanks | 13:33 |
| niemeyer | TheMue: I may only get to it tomorrow | 13:33 |
| niemeyer | TheMue: I'll try to stick to store work today, since I've been clearly failing at it | 13:33 |
| TheMue | niemeyer: ok, no prob, better keep concentration | 13:34 |
| TheMue | niemeyer: would part of state should I take next? | 13:34 |
| rog | lunch | 13:38 |
| rog | (outside in the sun!) | 13:39 |
| niemeyer | TheMue: Not sure.. | 13:43 |
| TheMue | rog: enjoy. sun? (envy) here it is grey and wet | 13:43 |
| niemeyer | TheMue: Actually, wasn't there something to be done as a left over from the current work? | 13:43 |
| niemeyer | I recall something like that, but can't remind of it now | 13:43 |
| niemeyer | TheMue: Ah, yes.. zk => st | 13:44 |
| TheMue | niemeyer: state into all entities, but that's only very small | 13:44 |
| niemeyer | TheMue: Yeah, but has to be done | 13:44 |
| TheMue | niemeyer: yep, exactly | 13:44 |
| niemeyer | TheMue: Besides that, just pick something that moves the state forward | 13:44 |
| TheMue | niemeyer: ok, will do so | 13:44 |
| niemeyer | TheMue: Preferably in areas that do not conflict with changes going on in Python (relations) | 13:45 |
| niemeyer | andrewsmedina: Any progress there? | 13:45 |
| andrewsmedina | niemeyer: yes | 13:45 |
| niemeyer | andrewsmedina: Nice | 13:45 |
| TheMue | niemeyer: yep, no relations | 13:45 |
| andrewsmedina | niemeyer: I'm finishing the tests for port charm metadata subordinate to Go | 13:46 |
| niemeyer | andrewsmedina: Very neat | 13:55 |
| niemeyer | andrewsmedina: Thanks for working on that | 13:55 |
| andrewsmedina | niemeyer: :D | 13:58 |
| rog | wondering if anyone is up for sanity-checking a possible approach to making my ec2test fake ec2 server simulate eventual consistency issues | 14:26 |
| rog | fwereade, jimbaker? | 14:26 |
| fwereade | rog, I'm trying to finish something off atm, I'll ping you when I'm free | 14:27 |
| rog | fwereade: thanks | 14:27 |
| === Leseb_ is now known as Leseb | ||
| niemeyer | Lunch time here.. | 15:50 |
| fwereade | rog, sorry that took so long; can I be of service? | 15:53 |
| rog | fwereade: yes, thanks. gimme 1 minute. | 15:54 |
| rog | fwereade: bakc | 15:56 |
| rog | ck | 15:56 |
| rog | fwereade: so... | 15:56 |
| rog | fwereade: the object is to simulate the "eventual consistency" semantics of the ec2 server. | 15:56 |
| rog | fwereade: to that end, the idea is that all operations operate on an out-of-date version of the server data. | 15:56 |
| rog | fwereade: when a client makes a call, if it's a read-only request, we return the results from the out-of-date version. | 15:57 |
| rog | fwereade: if it's a modification request, we allocate any unique ids (instance ids, group ids etc) from the current version. then we queue up the operation to be executed later. | 15:57 |
| rog | fwereade: after every call, we execute one operation from the queue. | 15:57 |
| rog | fwereade: does that sound reasonable to you? | 15:57 |
| fwereade | rog, yes, I think so | 15:57 |
| rog | fwereade: i can't help feeling that there might be a large bear lurking in the room somewhere | 15:58 |
| rog | guess i can only try and see | 15:59 |
| fwereade | rog, I'm wondering though whether it might be good to have some way of varying the behaviour -- sometimes immediate consistency, sometimes irritatingly delayed | 15:59 |
| rog | fwereade: the idea is to allow the length of the queue to be specified when the server starts | 15:59 |
| rog | fwereade: i don't think having random behaviour is going to be good for reproducing failed tests | 15:59 |
| fwereade | rog, the trouble is you don't *really* want to do that *randomly*, or you're on a miserable path to unreproducible test failures | 15:59 |
| fwereade | snap :p | 16:00 |
| rog | fwereade: so i think we'd try all tests with "totally consistent" and "very delayed" behaviour | 16:00 |
| rog | fwereade: one question i can't quite decide on is if there are significant tests that might fail with random consistency but neither of the above scenarios | 16:01 |
| fwereade | rog, that's what's bugging me | 16:01 |
| fwereade | rog, I can see a plausible route towards repeatable random tests | 16:01 |
| jimbaker | rog, this sounds like what we do in the testing in the python code. not uniformly mind you | 16:02 |
| jimbaker | in terms of delays | 16:02 |
| rog | fwereade: what about concurrency? | 16:02 |
| fwereade | rog, where we store the seed that generated the queue-frobbing and can attach that | 16:02 |
| rog | fwereade: that assumes we generate requests deterministically. | 16:03 |
| fwereade | rog, ok, close-enough-to-repeatable to probably be a bit easier to reproduce | 16:03 |
| rog | fwereade: hence my question "what about concurrency"? | 16:03 |
| fwereade | rog, quite so; it may very well be that the difference is slight | 16:03 |
| rog | fwereade: i dunno. | 16:03 |
| fwereade | rog, and doesn't justify the effort | 16:03 |
| rog | fwereade: the other thing is that the above approach doesn't work if you can jump backwards and forwards in time. | 16:04 |
| fwereade | rog, in what sense? | 16:04 |
| rog | fwereade: because you need to keep several versions of the server state | 16:04 |
| rog | fwereade: with this approach we just keep one version and apply deltas | 16:04 |
| fwereade | rog, ah got you | 16:04 |
| rog | fwereade: hmm | 16:05 |
| rog | fwereade: although | 16:05 |
| rog | fwereade: if delta application is deterministic | 16:06 |
| rog | fwereade: then we can keep several versions and apply deltas to each of them as appropriate | 16:06 |
| fwereade | rog, sorry, I think I'm lost... | 16:07 |
| rog | fwereade: we could have several versions of the state, each representing the state at some point within the queue of pending operations | 16:07 |
| rog | fwereade: as deltas move through the queue, they get applied to each state in turn. or maybe not in turn... to simulate the random ordering | 16:08 |
| rog | fwereade: assuming s1+delta1+delta2 == s1+delta2+delta1 it should be ok | 16:09 |
| fwereade | rog, yes, I understand that bit; ah, yes, ok, we can have a separate queue of operations for each... er, time-slice of the server | 16:09 |
| rog | fwereade: unfortunately that last isn't quite true, although it is for most things AFAICS | 16:09 |
| fwereade | rog, thinking more though; it might be good to propose it with just the discussed boundary-condition checks, which will be useful if not complete | 16:10 |
| rog | fwereade: for instance, if the user does AddSecurityGroupAuth followed by RevokeSecurityGroupAuth, what's the result if the ops get reordered? | 16:10 |
| fwereade | rog, finding some happy medium between randomness and repeatability can wait | 16:11 |
| fwereade | rog, I have no idea, and that worries me | 16:11 |
| rog | fwereade: yeah, i think so. and that probably makes it easier to do something more sophisticated later on if we deem it useful | 16:11 |
| rog | fwereade: i *wish* there was some documentation on what semantics you can expect from ec2 | 16:11 |
| fwereade | rog, hopefully | 16:12 |
| rog | fwereade: i can't find anything at all. even discussions. | 16:12 |
| rog | fwereade: (which i find hard to believe - surely people encounter this stuff all the time?) | 16:12 |
| fwereade | rog, I think the semantics are "you'll get what you're given, and you'll like it" | 16:12 |
| jimbaker | rog, maybe take a look at http://www.allthingsdistributed.com/ ? | 16:12 |
| jimbaker | also there is some docs on expected semantics for s3, fwiw | 16:13 |
| rog | jimbaker: did you have any particular blog post in mind? | 16:14 |
| fwereade | rog, heh, what if it's just that it's a *really* rare and unreproducible occurrence, so nobody ever hears about it or quite believes it when it happens? | 16:14 |
| rog | fwereade: it's not rare! | 16:14 |
| * fwereade dons tinfoil hat | 16:14 | |
| jimbaker | rog, unfortunately no, but it is the sort of thing werner discusses on his blog | 16:14 |
| rog | fwereade: it happens a lot of the time in my tests | 16:14 |
| rog | jimbaker: ok, thanks. | 16:14 |
| fwereade | rog, ah, ok, I didn't realise | 16:14 |
| rog | fwereade: otherwise i don't think i'd've know | 16:15 |
| rog | n | 16:15 |
| rog | fwereade: one minor comment on https://codereview.appspot.com/5672067/ | 16:43 |
| fwereade | rog, not convinced by that; how about a "checkedClock" (or something) field on the type, as a compromise? | 16:45 |
| rog | fwereade: i'm not sure that's right. if the file did not at first exist, you never need to check the clock. | 16:46 |
| fwereade | rog, hm, good point | 16:46 |
| fwereade | rog, Idon;t really like the idea of threading it through all those calls though :/ | 16:47 |
| rog | fwereade: i think that the calls in Alive and AliveW are in the unique position of knowing that the clock needs to be checked, hence my suggestion. | 16:47 |
| rog | fwereade: it's only two levels... | 16:47 |
| * fwereade looks at the code again | 16:47 | |
| fwereade | rog, yeah, it turns out ok | 16:56 |
| rog | fwereade: cool | 16:56 |
| fwereade | rog, how about calling setAlive setState instead? it now always sets both .alive and .timeout | 16:56 |
| * rog has a look | 16:58 | |
| rog | fwereade: seems reasonable. (although i think it always has set both alive and timeout) | 17:01 |
| fwereade | rog, it went through a brief recent incarnation in which it didn't even bother to read the timeout more than once | 17:02 |
| rog | fwereade: ah | 17:02 |
| fwereade | rog, changing it back made the name change seem like a good idea | 17:02 |
| rog | fwereade: yeah, i think setState is a better name for what it does | 17:03 |
| fwereade | rog, cheers, reproposed ;) | 17:04 |
| fwereade | rog, EOD I'm afraid; if you have a spare moment I would appreciate it if you would have a little look at https://codereview.appspot.com/5695082 and let me know the perceived crackfulness of the general idea | 17:08 |
| rog | fwereade: cool! | 17:08 |
| rog | fwereade: yes, i'll do that | 17:09 |
| fwereade | rog, er, expect another one in a sec, I appear to have ballsed up a bit | 17:09 |
| rog | fwereade: looks better with firstTime, to my mind - that logic becomes a bit more obvious. | 17:09 |
| rog | fwereade: thanks for changing it. | 17:09 |
| fwereade | rog, a pleasure :) | 17:11 |
| fwereade | rog, huh, no I didn't balls it up; somehow I was trying to link against something stale, I guess | 17:12 |
| fwereade | rog, so don't expect an update soon | 17:13 |
| rog | fwereade: ok. i'll try to ignore the extraneous diffs | 17:13 |
| rog | fwereade: where's the implementation of relation-get etc again? | 17:14 |
| fwereade | rog, in python? | 17:14 |
| * fwereade racks brains | 17:14 | |
| fwereade | rog, juju.hooks.commands | 17:14 |
| rog | fwereade: no, that wasn't the question. | 17:15 |
| rog | fwereade: i was wondering what files the new code for " Rough proposal for implementation of relation-get and similar commands." was in | 17:15 |
| rog | fwereade: presuming it's somewhere in that CL | 17:15 |
| fwereade | rog, ah sorry: none of them are *implemented* yet | 17:15 |
| rog | fwereade: server/protocol looks new | 17:15 |
| fwereade | rog, but it provides a mechanism by which a command like relation-get can get the unit agent (which knows all the appropriate state) to execute the command for it and pipe back the results | 17:16 |
| rog | fwereade: yeah, that's what i was looking for. that's in cmd/server, right? | 17:16 |
| fwereade | rog, yeah | 17:17 |
| rog | fwereade: instant initial reaction: why not use the rpc package? | 17:18 |
| rog | fwereade: rather than doing our own marshalling etc | 17:18 |
| fwereade | rog, huh, because I ballsed up on due diligence :p | 17:18 |
| rog | fwereade: i think this code can be really really small | 17:19 |
| fwereade | rog, then that's doubly awesome | 17:19 |
| fwereade | rog, it was very instructive writing it all but less code is always better than more code ;) | 17:19 |
| rog | fwereade: the unix socket stuff will stay :-) | 17:19 |
| fwereade | rog, cool | 17:20 |
| rog | fwereade: but yeah, i think a few methods exported on a relation type and passed to rpc.Server.Register will probably do the job | 17:22 |
| rog | fwereade: the only thing that won't give you will be commands that stream. | 17:23 |
| rog | fwereade: are there any like that? | 17:23 |
| fwereade | fwereade, some certainly dump data back on stdout | 17:23 |
| fwereade | er, *rog*, ^^ | 17:24 |
| rog | fwereade: lots and lots of data? or don't-care-too-much-if-it's-all-in-memory data? | 17:24 |
| fwereade | rog, and in general I'd expect/hope that --verbose and --debug should work (and thereby potentially generate monster stderrs too) | 17:25 |
| fwereade | rog, not sure how big it could get | 17:25 |
| rog | fwereade: i've an idea for how that could work ok with rpc | 17:26 |
| fwereade | rog, well, we could do exactly the same in terms of how we pipe stuff back | 17:27 |
| rog | fwereade: yeah | 17:27 |
| fwereade | rog, the streams feel to me like a good thing | 17:27 |
| rog | fwereade: if you're around for a minute or so more, i'll paste a little example | 17:27 |
| fwereade | rog, please do | 17:28 |
| rog | fwereade: http://paste.ubuntu.com/860798/ | 17:31 |
| rog | fwereade: slightly contorted by the necessary form of rpc methods | 17:31 |
| rog | fwereade: but the idea is that you call New which gives you a handle on a running command. then you can call other methods to get info on that command, including reading from its output streams | 17:32 |
| rog | fwereade: Cmd is the type that would be registered as an rpc handler | 17:32 |
| rog | fwereade: maybe Cmd.Result might be better named Cmd.Wait | 17:33 |
| rog | fwereade: and if we were concerned about malicious clients, we could put a random capability string in the Running id. | 17:34 |
| === Leseb_ is now known as Leseb | ||
| fwereade | rog, seems less clear to me but that's probably a matter of familiarity; I shall meditate upon it | 17:38 |
| rog | fwereade: i'll quickly sketch an implementation | 17:39 |
| rog | fwereade: i think it'll be pretty small. | 17:39 |
| fwereade | rog, don't rush, it has been noticed by others that it's my eod | 17:40 |
| rog | fwereade: no probs. | 17:40 |
| rog | fwereade: i've only got 20 minutes myself... | 17:40 |
| rog | fwereade: see ya tomorrow morning | 17:40 |
| fwereade | rog, and you :) | 17:41 |
| andrewsmedina | niemeyer: hi | 18:39 |
| niemeyer | andrewsmedina: yo | 18:41 |
| andrewsmedina | niemeyer: for port this test (http://dpaste.org/9MrTY/) I need create another dummy charm? | 18:43 |
| niemeyer | andrewsmedina: You can copy and change an existing one.. some tests already have the needed utilities to do that easily | 18:44 |
| niemeyer | andrewsmedina: Hold on.. I'll find an example | 18:45 |
| andrewsmedina | niemeyer: dont have yet in Go a equivalent for change_sample | 18:46 |
| niemeyer | andrewsmedina: We actually do | 18:46 |
| niemeyer | andrewsmedina: Check charm_test.go | 18:46 |
| niemeyer | andrewsmedina: If you have a reader r, you can do something like | 18:47 |
| niemeyer | hackYaml := ReadYaml(r) | 18:47 |
| niemeyer | hackYaml["subordiate"] = true | 18:47 |
| niemeyer | newr := hackYaml.Reader() | 18:48 |
| andrewsmedina | niemeyer: I will try this | 18:49 |
| niemeyer | andrewsmedina: It's not as practical as change_sample yet, but feel free to implement a function so that it is | 18:49 |
| niemeyer | andrewsmedina: For example: changeSample(func(yh *yamlHacker) { yh["subordinate"] = true }) | 18:49 |
| andrewsmedina | niemeyer: is missing only this use case | 18:51 |
| niemeyer | Booom! | 19:07 |
| niemeyer | Summer storm.. | 19:07 |
| andrewsmedina | niemeyer: can you help me? | 22:39 |
| niemeyer | andrewsmedina: Maybe.. :) What's up? | 22:39 |
| andrewsmedina | niemeyer: All tests are ok | 22:40 |
| andrewsmedina | niemeyer: but | 22:40 |
| andrewsmedina | niemeyer: in python (http://dpaste.org/9MrTY/) when a charm is not a proper subordinate, is returned the charm path in the excetion | 22:42 |
| hazmat | andrewsmedina, and? | 22:47 |
| hazmat | this is problematic because? | 22:47 |
| andrewsmedina | hazmat: I dont have path in go implementation | 22:48 |
| niemeyer | andrewsmedina: Sounds fine.. just don't return it | 22:48 |
| andrewsmedina | niemeyer: I return a simple error message? | 22:49 |
| niemeyer | andrewsmedina: Simplicity FTW | 22:49 |
| andrewsmedina | niemeyer: I will send the commit in minutes :p | 22:50 |
| niemeyer | andrewsmedina: Superb :) | 22:50 |
| andrewsmedina | niemeyer: sorry for my noob questions | 22:50 |
| niemeyer | andrewsmedina: Totally fine | 22:50 |
| andrewsmedina | Im a noob in go =/ | 22:50 |
| niemeyer | andrewsmedina: Not to worry.. there are a handful of people only that know Go for more than a couple of years, so you're not too far behind :) | 22:53 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!