| rogpeppe | fwereade: good morning | 08:19 |
|---|---|---|
| fwereade | rogpeppe, heyhey | 08:19 |
| rogpeppe | fwereade: back home now? | 08:19 |
| fwereade | rogpeppe, yeah, it's quite a relief :) | 08:20 |
| rogpeppe | fwereade: parents can be stressful... | 08:20 |
| fwereade | rogpeppe, my parents are fine really... it's the aggregate of travelling around and england in general and having to see far too many people in too few days | 08:21 |
| rogpeppe | fwereade: yeah, i can see that... | 08:21 |
| rogpeppe | fwereade: still, you got some good weather... | 08:22 |
| fwereade | rogpeppe, oh, indeed, it was lovely... but I'm still really happy to be back home :) | 08:22 |
| rogpeppe | fwereade: exciting event of the morning: i've just submitted the branch that actually runs the Go s/w on the server... | 08:25 |
| rogpeppe | fwereade: as of 30s ago | 08:25 |
| * fwereade whoops, hollers | 08:25 | |
| rogpeppe | fwereade: bootstrap phase 1, complete | 08:26 |
| rogpeppe | fwereade: i thought i'd move on to some constraint stuff now | 08:27 |
| fwereade | rogpeppe, that sounds like a good plan | 08:27 |
| rogpeppe | fwereade: unfortunately i lost all the notes i made over UDS | 08:28 |
| rogpeppe | fwereade: not that there were many, mind. i *think* i remember the general gist. | 08:29 |
| fwereade | rogpeppe, heh :) hopefully the approximate parmeters remain accurate | 08:29 |
| fwereade | rogpeppe, if I can be of any assistance let me know | 08:29 |
| rogpeppe | fwereade: thanks. i think i'll take a dive into the python code for a little, just to see. have you got a suggestion as to the crux of the contraint code there? | 08:30 |
| fwereade | rogpeppe, basically everything core is is juju/machine/constraints.py | 08:31 |
| rogpeppe | fwereade: lovely, thanks | 08:31 |
| fwereade | rogpeppe, the big thing that should definitely change behaviour-wise is the unit-level constraint stickiness | 08:32 |
| rogpeppe | fwereade: is that something we can change without breaking things? | 08:32 |
| fwereade | rogpeppe, yeah, I think so -- it just removes a guarantee that I don't think is made explicitly anywhere | 08:33 |
| rogpeppe | fwereade: ok. i take it that the current code implements unit-level stickiness, and you're saying that it should be removed, as per discussions here earlier. | 08:34 |
| rogpeppe | ? | 08:34 |
| fwereade | rogpeppe, it *is* a guarantee that I assumed we'd need to provide, and perhaps some users share my assumptions | 08:34 |
| fwereade | rogpeppe, yeah | 08:34 |
| fwereade | rogpeppe, but it doesn't fit niemeyer's model and I think he is right about it :) | 08:36 |
| rogpeppe | fwereade: yeah, probably. i remember the discussion. | 08:36 |
| rogpeppe | fwereade: can i treat this as the authoritative spec? https://juju.ubuntu.com/docs/constraints.html | 08:38 |
| rogpeppe | fwereade: for instance, can i assume that no value will contain white space? | 08:39 |
| fwereade | rogpeppe, I think you should be safe there, yes | 08:39 |
| fwereade | rogpeppe, anything related to units should be dropped, and you should maybe hold off on the provider-badged constraints until you've had a chance to sync up with niemeyer | 08:40 |
| rogpeppe | fwereade: seems good | 08:41 |
| fwereade | rogpeppe, (IIRC he wants ec2-zone to be availability-zone... but I'm not sure how long-term that plan is) | 08:41 |
| fwereade | rogpeppe, (and who knows what maas constraints we'll end up with) | 08:41 |
| rogpeppe | fwereade: so, the plan is to have a single struct that represents constraints across all providers. | 08:43 |
| rogpeppe | fwereade: i guess that means that conflicts are provider non-specific too | 08:44 |
| fwereade | rogpeppe, I *think* the conflicts were always fundamentally non-specific, am I missing something? | 08:45 |
| fwereade | rogpeppe, ah, I think I see | 08:45 |
| rogpeppe | fwereade: we should get the python version to change ec2-zone to availability-zone so that the Go version doesn't break things | 08:45 |
| fwereade | rogpeppe, what actually conflicts with what may take some more thinking now I guess | 08:45 |
| fwereade | rogpeppe, that sort of question is why I feel we should sync up with niemeyer | 08:45 |
| * bigjools has a couple of MPs up if someone could take a looksie please | 08:46 | |
| fwereade | rogpeppe, and I feel it'd be best to start with a constraints struct that skips all the potentially contentious ones | 08:46 |
| rogpeppe | fwereade: i think what i'll do as a first step is move the existing "constraint" stuff out of ec2 and into environs | 08:46 |
| rogpeppe | fwereade: so we've got the skeleton types and API there | 08:46 |
| fwereade | rogpeppe, great | 08:46 |
| fwereade | bigjools, I'll see if I can manage a quick review at some stage but I'm not *really* meant to be thinking about python... so you might end up having to wait for the US to wake up | 08:47 |
| fwereade | bigjools, sorry :( | 08:47 |
| bigjools | fwereade: poor Python! | 08:50 |
| bigjools | they are small, if that helps :) | 08:50 |
| * fwereade is flattered :) | 08:50 | |
| fwereade | bigjools, cool | 08:50 |
| rogpeppe | bigjools: lucky Go! :-) | 08:50 |
| bigjools | are you planning to switch to Go and forget the Python one at some point? | 08:51 |
| rogpeppe | bigjools: that's the plan | 08:51 |
| rogpeppe | fwereade: just looking at the INSTANCE_TYPES list in providers/ec2/utils.py. where did you get the stats from. the memory numbers look a bit weird. what units are they in? | 08:54 |
| fwereade | rogpeppe, IIRC they're MB; and, er, I forget :/. I've seen a couple of different sets of numbers quoted... amazon is not much help, they just say things like "1.6GB" which is tricky to translate with certainty | 08:55 |
| rogpeppe | fwereade: ok, thanks. | 08:55 |
| fwereade | rogpeppe, I guess the *other* issue with constraints is whether or not the resource-map thing is planned for 12.10 | 08:55 |
| rogpeppe | fwereade: BTW with conflicts, i'm wondering if the best approach is simply to let a conflicting requirement be satisfied by nothing. | 08:56 |
| fwereade | rogpeppe, because I'm pretty sure we don't want hardcoded numbers; but nor do we rally want to saddle go with new features | 08:56 |
| fwereade | rogpeppe, please expand re conflicts | 08:57 |
| fwereade | rogpeppe, I don't think what you said works | 08:57 |
| rogpeppe | fwereade: currently, if you specify two conflicting constraints, it's an error, right? | 08:57 |
| fwereade | rogpeppe, yeah | 08:57 |
| rogpeppe | fwereade: so i'm wondering if it would be easier if it wasn't an error, but simply resulted in a constraint that was impossible to satisfy | 08:58 |
| rogpeppe | fwereade: which means that conflicts don't need to be defined in a provider-non-specific way | 08:59 |
| fwereade | rogpeppe, not sure that works... consider an m1.small with at least 512MB. that's not unsatisfiable in any sane universe; but it does I think betray confusion on the user's part, which should be flagged an whined about | 09:00 |
| rogpeppe | fwereade: and also means that, potentially, some attributes may conflict only if they have some values | 09:00 |
| fwereade | rogpeppe, I fear that that in particular will become hideously complex and easy to screw up | 09:00 |
| rogpeppe | fwereade: i'm not sure. | 09:01 |
| rogpeppe | fwereade: i think that if we have tags, for example, a tagged class of machine may or may not conflict | 09:01 |
| rogpeppe | fwereade: with some other attribute | 09:01 |
| rogpeppe | fwereade: and if someone tries to deploy an m1.small with at least 512MB, they'll quickly find out it's impossible... | 09:02 |
| fwereade | rogpeppe, I don't see how that's an improvement over immediately finding it's ill-specified | 09:02 |
| rogpeppe | fwereade: dunno, just thinking | 09:02 |
| fwereade | rogpeppe, consider also arch/instance-type interactions | 09:03 |
| fwereade | rogpeppe, it's not insane to ask for an i386 m1.small, but it is to ask for an i386 cc2.8xlarge | 09:04 |
| fwereade | rogpeppe, the first cut had arch conflicting with instance-type, and we dropped that conflict for exactly that reason | 09:05 |
| rogpeppe | fwereade: i think the difficulty with the "immediately finding it's ill-specified" thing is that that there's an overlap between several constraining attributes in terms of the machines that can satisfy them, and that overlap won't always be clear-cut, i think | 09:05 |
| fwereade | rogpeppe, and so that falls back to "we can't satisfy those constraints" vs "that request doesn't even make sense, pick one or the other" | 09:05 |
| rogpeppe | fwereade: ok, so that's precisely my point | 09:05 |
| rogpeppe | fwereade: i don't mind if we always do the former | 09:06 |
| fwereade | rogpeppe, I see your point of view | 09:07 |
| rogpeppe | fwereade: the reason we're able to do the conflict think with instance-type is that it specifies almost everything. but it seems to me that it's a special case. | 09:07 |
| rogpeppe | fwereade: hmm, i see the advantage though of being able to say to the user "a constraint with these *never* be able to be satisfied, regardless of the values you choose". | 09:09 |
| rogpeppe | fwereade: but even then that's not actually true. | 09:09 |
| fwereade | rogpeppe, I think I may still be missing some details... how should we then handle a case where the env specified 1386 and a service asks for cc2.8xlarge? | 09:09 |
| rogpeppe | fwereade: it would fail to choose any instances (assuming cc2.8xlarge is always amd64) | 09:10 |
| fwereade | rogpeppe, the conflict mechanism currently drops the more-general arch constraint because it conflicts with a more-specific instance-type constraint | 09:10 |
| fwereade | rogpeppe, if we're not using conflicts at all, I'm not sure how we deal with that situation | 09:10 |
| rogpeppe | fwereade: i would treat all the constraints as independent | 09:10 |
| rogpeppe | fwereade: given that ec2 is filtering instance types, not machines, i'd start with a list of all instance types, filter out all those without instance-type=cc2.8xlarge, then filter out all those with arch!=i386, leaving nothing. | 09:12 |
| fwereade | rogpeppe, so you'd just break it? :p | 09:12 |
| rogpeppe | fwereade: no need to encode any knowledge of constraint conflicts AFAICS - they just come out in the wash, by resulting in nothing | 09:12 |
| rogpeppe | fwereade: it's broken by the user specifying an impossible constraint | 09:13 |
| rogpeppe | fwereade: same as if specified mem=80000 | 09:13 |
| rogpeppe | s/if/if you/ | 09:13 |
| fwereade | rogpeppe, I don't think they're the same situation | 09:14 |
| fwereade | rogpeppe, more realistic example | 09:14 |
| fwereade | rogpeppe, env: "instance-type=m1.small"; service "mem=8G" | 09:14 |
| rogpeppe | fwereade: oh yeah "setting an instance-type constraint will clear out any inherited cpu or mem values, and vice versa," | 09:15 |
| fwereade | rogpeppe, it's entirely unhelpful for us to whine about not having an m1.small with 8G, when IMO it's perfectly clear that the service-level 8G should result in an instance type that does satisfy | 09:15 |
| fwereade | rogpeppe, exactly | 09:15 |
| rogpeppe | fwereade: i think that actually makes constraints harder to reason about. | 09:16 |
| rogpeppe | fwereade: BTW what other constraints other than instance-type might result in conflicts? | 09:16 |
| fwereade | rogpeppe, the other uses were orchestra-name/orchestra-classes, but orchestra-name got dropped | 09:17 |
| fwereade | rogpeppe, in general I think that name constraints will conflict with just about everything else | 09:17 |
| rogpeppe | fwereade: i think i'm coming to the thought that certain attributes are symbolic aliases for a set of other attribute values. | 09:17 |
| fwereade | rogpeppe, and while I understand the arguments against name constraints I also think we may be stuck with them | 09:17 |
| rogpeppe | Aram: yo! | 09:18 |
| Aram | morning. | 09:18 |
| fwereade | rogpeppe, hmm, I'm not certain the linked attribute values are *necessarily* knowable | 09:19 |
| fwereade | Aram, heyhey | 09:19 |
| rogpeppe | fwereade: yeah, but perhaps that's true of the attributes they conflict with too. | 09:19 |
| rogpeppe | fwereade: i.e. on some providers, instance-type *may* conflict with arch | 09:20 |
| fwereade | rogpeppe, I'm thinking about MAAS here -- we can't currently find out cpu/mem, but then that's why we don't even expose them, so probably that will all come out in the wash when MAAS gets more sophisticated | 09:20 |
| fwereade | rogpeppe, I think that's a dangerous road to go down... that was what we had, and we dropped it because we don't want the conflict mechanism to come into play unless specifying one side *always* implicitly specifies the other | 09:21 |
| rogpeppe | fwereade: oh, interesting. i thought current behaviour was entirely provider-specific | 09:22 |
| fwereade | rogpeppe, I guess it is but only as a side-effect of different providers exposing different constraint sets | 09:22 |
| rogpeppe | fwereade: yeah | 09:23 |
| fwereade | rogpeppe, it's an accidental property, not an essential one | 09:23 |
| rogpeppe | fwereade: ok. | 09:23 |
| rogpeppe | fwereade: 'cos we're proposing to go with a single type for constraints across all architectures | 09:23 |
| rogpeppe | s/architectures/providers/ | 09:23 |
| fwereade | rogpeppe, indeed | 09:23 |
| fwereade | rogpeppe, and while this makes me a little uncomfortable I don't have any solid arguments against doing so :) | 09:24 |
| rogpeppe | fwereade: which means that the conflicts logic needs to happen in a provider-independent way | 09:24 |
| fwereade | rogpeppe, yeah; and I suppose it comes down to what precisely an instance-type does specify | 09:24 |
| rogpeppe | fwereade: yup | 09:25 |
| fwereade | rogpeppe, AFAICT it's just cpu/mem/local-storage | 09:25 |
| rogpeppe | fwereade: and currently we're taking that cue from ec2 | 09:25 |
| fwereade | rogpeppe, and we don't handle local-storage at the moment anyway | 09:25 |
| rogpeppe | fwereade: but it might be different on other providers | 09:25 |
| fwereade | rogpeppe, having poked around a little, I think some providers may have tighter requirements (on foo-cloud, perhaps, instance-type will always specify arch) | 09:26 |
| rogpeppe | fwereade: so do we treat instance-type as conflicting with arch on those providers? | 09:26 |
| rogpeppe | fwereade: does instance-type overwrite arch on those providers? | 09:26 |
| fwereade | rogpeppe, personally I think it shouldn't, because that feels like a coincidence they could drop at any time | 09:27 |
| rogpeppe | fwereade: i'm sure it's too late now, but i *think* things would be much more straightforward if all constraints were totally independent, and constraints unknown to a provider were ignored. | 09:27 |
| fwereade | rogpeppe, like amazon actually did -- originally the only arch variation was with t1.micro, which we were happy to special-case because nobody's going to be deploying important services to those anyway | 09:27 |
| fwereade | rogpeppe, I think the instance-type concept becomes meaningless once it's divorced from cpu/mem | 09:29 |
| rogpeppe | fwereade: it just seems a bit weird we're proposing a general rule ("instance-type implies cpu/mem but not arch") based only on the way that amazon has specified instance types | 09:29 |
| fwereade | rogpeppe, it's intended to be the minimum attribute set implies by instance-type | 09:30 |
| rogpeppe | fwereade: across all providers? surely some providers might not use instance-type to imply either cpu or mem? | 09:31 |
| fwereade | rogpeppe, that's the question... what then does instance-type actually mean? | 09:31 |
| rogpeppe | fwereade: indeed | 09:31 |
| rogpeppe | fwereade: in the aws case, it's a provider tag. | 09:32 |
| rogpeppe | fwereade: ah, no it's not. | 09:32 |
| rogpeppe | fwereade: because if we say cg1.4xlarge, we might not get the GPU h/w | 09:33 |
| fwereade | rogpeppe, sorry, not following you there | 09:33 |
| rogpeppe | fwereade: if it was a tag, then any machine we get would be guaranteed to get that tag. | 09:33 |
| fwereade | rogpeppe, if we ask for cg1.4xlarge, we will surely get GPUs | 09:34 |
| rogpeppe | fwereade: but as far as i can see, all the constaints specified by cg1.4xlarge are also satisfied by cc2.8xlarge | 09:34 |
| fwereade | rogpeppe, if we ask for cpu/mem such that cg1.4xlarge is the cheapest way of satisfying them, we will also get the GPUs, but nobody cares | 09:34 |
| rogpeppe | fwereade: but i'm presuming from the comment that that instance type doesn't have the fancy GPU h/w | 09:35 |
| rogpeppe | fwereade: if we ask for cg1.4xlarge, are we guaranteed to not get cg1.8xlarge ? | 09:35 |
| fwereade | rogpeppe, er, yes, I think so | 09:35 |
| fwereade | rogpeppe, barring crackfulness on amazon's side | 09:35 |
| rogpeppe | fwereade: ah, so an instance-type isn't just an alias for the mem, cpu etc attributes of that instance-type | 09:36 |
| rogpeppe | fwereade: it is really a tag. | 09:36 |
| fwereade | rogpeppe, still not following... isn't it an alias for a certain amount of GPU as well? | 09:37 |
| fwereade | rogpeppe, it's just that we don't happen to expose gpu | 09:37 |
| rogpeppe | fwereade: it implies that, yeah. | 09:37 |
| fwereade | rogpeppe, back to instance-type, briefly... across all providers I'm aware of, instance-type implies cpu/mem | 09:39 |
| fwereade | rogpeppe, in many cases it implies more | 09:39 |
| fwereade | rogpeppe, but the common properties are cpu/mem, and I'm having trouble imagining an instance-type that doesn't | 09:39 |
| fwereade | rogpeppe, can you flesh out your earlier what-if with a plausible scenario? | 09:40 |
| rogpeppe | fwereade: i guess i'm uncomfortable with that as a minimum. for instance, on some providers, instance-type may imply arch, but arch won't be cleared out when we specify instance-type | 09:42 |
| rogpeppe | fwereade: and in that case, we'll fall back to the situation i'm suggesting we do for everything - it just won't satisfy anything | 09:42 |
| rogpeppe | fwereade: hold on, there's parcel at the door | 09:43 |
| fwereade | rogpeppe, it's a first layer of defence against knowably nonsensical requests... we don't have any guarantees that anything will actually be provisionable ever | 09:45 |
| fwereade | rogpeppe, and it gives us sl. more helpful behaviour when dealing with the interplay between env and service constraints | 09:46 |
| fwereade | rogpeppe, I think the second point is the one you need to address if you want to drop the conecpt | 09:47 |
| rogpeppe | fwereade: it's more than a layer of defense, i think - the "setting an instance-type constraint will clear out any inherited cpu or mem values" thing is awkward | 09:47 |
| rogpeppe | fwereade: yeah, that is the second point, i think | 09:48 |
| fwereade | rogpeppe, indeed... it may be awkward for us but I think it's useful for actual users in realistic scenarios | 09:48 |
| rogpeppe | fwereade: hmm, not entirely convinced. in that scenario earlier, it would be easy for the deployer to do "instance-type= mem=8g" | 09:50 |
| rogpeppe | fwereade: i.e. to explicitly clear out the instance-type constraint rather than remembering that mem will override instance-type. (and arch won't...) | 09:51 |
| fwereade | rogpeppe, it forces them to think more to get the same result | 09:51 |
| rogpeppe | fwereade: really? i think it gives more "stuff" to remember. | 09:51 |
| rogpeppe | fwereade: i.e. there's interplay between the attributes as well as the inheritance of attributes. | 09:52 |
| fwereade | rogpeppe, the intent is that users shouldn't actually have to worry about it... if they want a machine with 8G, we give them one, without forcing them to take additional context into account | 09:53 |
| rogpeppe | fwereade: but that's the context that they themselves have specified... | 09:53 |
| fwereade | rogpeppe, that they or someone else specified at some point arbitrarily long ago... | 09:53 |
| rogpeppe | fwereade: also, when they get told that no machines will be deployed, that's a pretty big hint they've got something wrong... | 09:54 |
| rogpeppe | fwereade: anyway, i've missed the boat here. i think we have to implement the current python semantics | 09:55 |
| fwereade | rogpeppe, the point is to reduce the incidence of hat sort of situation | 09:55 |
| rogpeppe | fwereade: i'm not sure that adding more magic helps here | 09:55 |
| fwereade | rogpeppe, and, yes, I think so; but tbh if we're planning to change ec2-zone to availability-zone, et cetera, I feel like we've reopened the whole thing really :/ | 09:56 |
| rogpeppe | fwereade: the user will encounter this situation with other conflicting attributes, so they've got to know how to deal with this kind of situation anyway | 09:56 |
| fwereade | rogpeppe, they may do, but reducing the incidence of "oh ffs, wasn't it obvious what I meant?"s feels worthy to me | 09:57 |
| fwereade | rogpeppe, it's not as if it's an unpredictable scenario | 09:58 |
| * rogpeppe finds value in a simple conceptual model, even if it makes some things a little harder. | 09:58 | |
| fwereade | rogpeppe, so, we should just drop instance-type entirely? :p | 10:00 |
| rogpeppe | fwereade: no. | 10:00 |
| rogpeppe | fwereade: just treat it as independent | 10:00 |
| rogpeppe | fwereade: if someone wants to specify instance-type=t1.micro mem=8G, let them do so | 10:00 |
| rogpeppe | fwereade: they won't get many machines working for them | 10:01 |
| rogpeppe | fwereade: and i think, on further reflection, i'd have providers treat any constraints unknown to them as unsatisfiable. | 10:01 |
| fwereade | rogpeppe, unknown-constraints=unsatisfiable definitely works for me | 10:03 |
| fwereade | rogpeppe, the issue is really that it's not always obvious that you're specifying instance-type=t1.micro at the same time as mem=8G | 10:04 |
| fwereade | rogpeppe, sure, some users will have to dig in and figure out the details at some point | 10:04 |
| fwereade | rogpeppe, delaying the point at which they do feels to me like a good thing :) | 10:04 |
| rogpeppe | fwereade: we could make it very easy for clients to find out the total set of constraints applying to a service. a tool to debug this stuff would seem very useful to me. | 10:05 |
| fwereade | rogpeppe, well, yeah, get-constraints exists | 10:05 |
| fwereade | rogpeppe, forcing users to check and calculate things in order to deploy machines feels counterproductive to me | 10:06 |
| fwereade | rogpeppe, especially since the effects are not immediately visible in the first place | 10:06 |
| rogpeppe | fwereade: ah, now there's an interesting point | 10:07 |
| fwereade | rogpeppe, the inheritance is the really annoying bit here | 10:07 |
| rogpeppe | fwereade: you don't get any immediate feedback if no machines might satisfy the constraint | 10:07 |
| rogpeppe | fwereade: yeah | 10:07 |
| fwereade | rogpeppe, yeah: changing env constraints could break an existing service (which you might only need to add a unit to in a week) | 10:08 |
| fwereade | rogpeppe, we can't entirely eliminate it but we should really try to guard against it | 10:08 |
| fwereade | where we can | 10:08 |
| rogpeppe | fwereade: or someone might be doing it deliberately - it's always possible | 10:09 |
| rogpeppe | fwereade: here's a possibility: when you specify a set of constraints that can't be satisfied, you get an immediate warning. | 10:09 |
| fwereade | rogpeppe, if I want to make a specific service undeployable I think I'll be eplicitly setting screwed-up settings on that service, not figuring out the right env change to break that and only that | 10:09 |
| rogpeppe | fwereade: and when setting environment constraints, you'll get a warning for all services that have been broken. | 10:10 |
| rogpeppe | fwereade: that seems potentially more useful (and less provider-specific) than saying that instance-type overrides cpu and mem | 10:10 |
| fwereade | rogpeppe, I can get behind that, but IMO it's orthogonal to the is-this-little-bit-of-magic-actually-useful-in-realistic-scenarios | 10:11 |
| fwereade | rogpeppe, I would really prefer to avoid forcing users to think when we don;t have to | 10:11 |
| fwereade | rogpeppe, please give me a plausible scenario in which we have an instance-type that doesn't specify cpu/mem | 10:12 |
| rogpeppe | fwereade: hmm. we're not talking MS-Word users here. these, hopefully are people that *can* think! | 10:12 |
| Aram | btw, I did did yesterday: code.google.com/p/rbits/godefs . It's a go style command that prints only type definitions from go programs. it's very useful with acme rogpeppe. | 10:12 |
| rogpeppe | fwereade: it's not that instance-type won't always specify cpu and mem, but that it often implies more | 10:12 |
| Aram | I plan to sort the output to be more useful. | 10:13 |
| Aram | but it's massively useful even in this stage | 10:13 |
| Aram | to me | 10:13 |
| rogpeppe | Aram: i can better that (but only by sending you a binary - my code won't build any more) | 10:14 |
| Aram | please. | 10:14 |
| fwereade | rogpeppe, my contention is, I guess, that cpu/mem/instance-type constraints are the most obvious and commonly-used ones, and kinder behaviour with those is valuable even if it doesn't include a full DWIM engine for every possible constraint | 10:17 |
| rogpeppe | fwereade: and i think that having the special rule in this case will mean that people will be more confused when they move beyond the bounds of this special case | 10:20 |
| rogpeppe | Aram: i've sent you a tarball. | 10:20 |
| rogpeppe | Aram: assuming you're running ubuntu under amd64, it should work | 10:20 |
| Aram | thanks | 10:20 |
| rogpeppe | Aram: the way it works is like this | 10:20 |
| rogpeppe | Aram: you click on an identifier, and middle-click on "def" in the window tag. | 10:21 |
| rogpeppe | Aram: it prints the file address of the definition of that identifier | 10:21 |
| rogpeppe | Aram: (and also the file address where you are currently, so you can jump back in context) | 10:21 |
| rogpeppe | Aram: it works for almost any identifier | 10:22 |
| rogpeppe | Aram: (it doesn't work if you've imported to ".") | 10:22 |
| rogpeppe | Aram: you can do "def -a" to find out the actual type and all members of the value. | 10:22 |
| rogpeppe | Aram: i really need to port it to Go 1. | 10:22 |
| Aram | hi niemeyer | 10:25 |
| rogpeppe | niemeyer: yo! | 10:25 |
| fwereade | niemeyer, heyhey | 10:25 |
| rogpeppe | niemeyer: how's london? | 10:25 |
| rogpeppe | niemeyer: looks hot to me | 10:26 |
| Aram | rogpeppe: your def thing works great. | 10:29 |
| rogpeppe | Aram: cool | 10:30 |
| rogpeppe | Aram: i think you're probably user number 1. | 10:30 |
| Aram | heh. | 10:30 |
| rogpeppe | Aram: i wrote it years ago | 10:30 |
| rogpeppe | Aram: unfortunately i forked the go parser to do it, and my changes were not acceptable. | 10:31 |
| rogpeppe | Aram: and i've been waiting for something equivalent to appear since then. | 10:31 |
| Aram | can go/parser do the job now? | 10:32 |
| rogpeppe | Aram: nope. | 10:32 |
| Aram | what does it lack? | 10:32 |
| niemeyer | Heya | 10:33 |
| rogpeppe | Aram: a place to put type definitions. my version of go/parser resolves all symbols at read time. | 10:33 |
| Aram | hmm... | 10:33 |
| niemeyer | rogpeppe: Great, sprint running smoothly | 10:33 |
| rogpeppe | niemeyer: cool. what's the sprint BTW? | 10:34 |
| niemeyer | and it's warm, yeah | 10:34 |
| rogpeppe | Aram: i just need to bite the bullet and have an independent symbol table. but i haven't had the incentive since the binary continues to work (the joy of static linking!) | 10:35 |
| Aram | indeed. | 10:35 |
| Aram | rogpeppe: I wonder if I can't do some plumbing tricks so middle clicking on def is not required? | 10:36 |
| rogpeppe | Aram: i don't think you want a right-click to take you to the definition of that identifier. usually you want the next *use* of that identifier | 10:36 |
| rogpeppe | Aram: otherwise i'd've done it. | 10:37 |
| Aram | yes, indeed. | 10:37 |
| rogpeppe | niemeyer: now that go-ec2-use-go-client is in, i thought i'd look towards constraints. i was just having a useful conversation with fwereade about same. | 10:40 |
| Aram | niemeyer: what's the stance on zookeeper? do we want to replace it if we can? | 11:01 |
| niemeyer | rogpeppe: I'd prefer if we focused on getting the basic image actually running first | 11:14 |
| rogpeppe | niemeyer: what do you mean? | 11:15 |
| rogpeppe | niemeyer: you mean get the agents going? | 11:15 |
| niemeyer | rogpeppe: Constraints give us some freedom about which images and so on to pick | 11:15 |
| niemeyer | rogpeppe: But none of us can get a running deployment yet, on any machine | 11:16 |
| niemeyer | rogpeppe: That sounds more important to get sorted first | 11:16 |
| rogpeppe | niemeyer: AFAICS we need the agents to do that | 11:16 |
| niemeyer | rogpeppe: Hmm, yes? :-) | 11:16 |
| rogpeppe | niemeyer: perhaps i should look at doing an agent then | 11:16 |
| rogpeppe | niemeyer: machine agent seems like it's next in line | 11:17 |
| rogpeppe | niemeyer: i'd be happy to start on that | 11:17 |
| niemeyer | rogpeppe: Indeed | 11:17 |
| rogpeppe | niemeyer: great, will do | 11:17 |
| niemeyer | rogpeppe: That'd be brilliant, thank you | 11:17 |
| niemeyer | rogpeppe: fwereade was going to do it, but he ended up focusing on the agent tools and whatnot, which are nice things to spin off too | 11:18 |
| rogpeppe | niemeyer: that's true. and there's quite a lot more in that area | 11:18 |
| niemeyer | Aram: There's a tentative plan to get an implementation of the gozk API on top of MongoDB | 11:19 |
| rogpeppe | niemeyer: we need to think about where there might be an Aram-sized hole somewhere too :-) | 11:19 |
| niemeyer | Aram: I got started on that front, but ended up not moving on much since the flight from UDS :) | 11:20 |
| niemeyer | rogpeppe: Agreed | 11:20 |
| Aram | I don't know anything about mongodb, does it offer the same features and constraints as zookeeper/doozer? | 11:21 |
| niemeyer | Aram: It's a pretty different beast, for better and for worse | 11:21 |
| Aram | I read somewhere that mongo uses "eventual consistency" while the algorithms used by zookeeper/doozer are consistent at all times. | 11:21 |
| Aram | indeed. | 11:21 |
| niemeyer | Aram: Hmm.. that's not *entirely* the case | 11:22 |
| niemeyer | Aram: It has a master | 11:22 |
| niemeyer | Aram: and replication | 11:22 |
| niemeyer | Aram: It has leader election within a replica set with consensus-style logic | 11:23 |
| niemeyer | Aram: So it's not entirely unlike it | 11:23 |
| niemeyer | Aram: Either way, as a first step, I suggest reading through the code base and trying to understand the relationship between things | 11:24 |
| niemeyer | Aram: and covering the usual startup tasks | 11:24 |
| Aram | yes. | 11:25 |
| niemeyer | Aram: Then, let's see.. I'd prefer if you helped pushing us forward towards feature parity | 11:25 |
| niemeyer | Aram: But we need to identify a nicely-sized problem chunk for that | 11:25 |
| niemeyer | Aram: If we can't find an isolated-enough problem, then maybe pushing that gozk alternative might be a good option given your previous experience | 11:26 |
| niemeyer | Aram: Perhaps, as a way to get started, you might start looking at a few of the missing command line entry points | 11:27 |
| niemeyer | Aram: We have pretty minimum support for that, and might be a nice way to push some small branches while you get used to the workflow and whatnot | 11:27 |
| niemeyer | Anyway.. I'll get some food or will miss lunch | 11:28 |
| niemeyer | biab | 11:28 |
| Aram | btw niemeyer, is this expected: http://pastebin.com/hgWd0qpg | 11:28 |
| Aram | people on #go-nuts complained about it as well | 11:28 |
| fwereade | niemeyer, I saw that as well, probably should have mentioned it :/ | 11:34 |
| fwereade | Aram, please keep in touch with me re command line tools; I'm working on stuff that's needed for `juju deploy` at the moment (while relation state doesn't exist yet) | 11:36 |
| Aram | ok. | 11:36 |
| fwereade | Aram, there are certainly commands that are orthogonal though | 11:36 |
| fwereade | Aram, I can't think of any blockers for add-unit and remove-unit | 11:37 |
| fwereade | lunch, bbiab | 11:38 |
| niemeyer | Aram: Yeah, it's a small bug in wait.js | 11:49 |
| niemeyer | Aram: It should say "got tired of waiting" or something | 11:49 |
| Aram | ah, so it's nothing on my side that causes the test to fail, ok. | 11:50 |
| rogpeppe | fwereade, niemeyer: this looks wrong to me. it looks like it's parsing the internal machine id to get the value returned from Machine.Id. am i on crack? http://bazaar.launchpad.net/+branch/juju/go/view/head:/state/machine.go#L64 | 11:50 |
| niemeyer | rogpeppe: Why is it wrong? | 11:51 |
| rogpeppe | niemeyer: because it's parsing the internal id (the zk path) which i don't *think* has a one-to-one correspondence with the external id | 11:52 |
| rogpeppe | niemeyer: but maybe i'm wrong and it does | 11:52 |
| rogpeppe | niemeyer: certainly the python code seems to keep them separate | 11:52 |
| niemeyer | rogpeppe: state/machine.py, line 276 | 11:53 |
| rogpeppe | niemeyer: ah, fair enough | 11:53 |
| rogpeppe | niemeyer: i was sure we had a discussion recently which decided they were different | 11:53 |
| niemeyer | rogpeppe: They are different.. | 11:53 |
| rogpeppe | niemeyer: something to do with failing half way through allocating a new machine id | 11:53 |
| niemeyer | rogpeppe: As the comment says, it's an implementation detail that today we map one to the other | 11:54 |
| niemeyer | We should copy the rest of the comment saying that this is an implementation detail | 11:55 |
| rogpeppe | niemeyer: what's responsible for actually creating the machine directory? | 11:55 |
| rogpeppe | i would assume the provisioning agent, but perhaps not | 11:56 |
| * rogpeppe goes back to the source | 11:56 | |
| niemeyer | rogpeppe: state/machine.py, line 18 | 11:57 |
| niemeyer | rogpeppe: No code outside of state should be manipulating ZooKeeper nodes directly | 11:58 |
| rogpeppe | niemeyer: agreed. | 11:58 |
| rogpeppe | niemeyer: i'm slightly surprised that clients are doing non-atomic actions though. | 11:58 |
| rogpeppe | niemeyer: i guess that means that machine ids may not be allocated in strict sequence. | 11:59 |
| niemeyer | rogpeppe: Yep.. there is/was no multinode atomic operation in zk | 12:00 |
| niemeyer | rogpeppe: It's no big deal, though.. | 12:00 |
| rogpeppe | niemeyer: that's true | 12:00 |
| niemeyer | Crap, we're moving rooms.. | 12:01 |
| niemeyer | Without a battery, that means a restart :-( | 12:01 |
| niemeyer | biab | 12:01 |
| rogpeppe | k | 12:01 |
| rogpeppe | off for some lunch | 12:09 |
| Aram | godep launchpad.net/... | grep '^launchpad' | awk '{ printf("%s\n\n", $0)}' | awk 'BEGIN { RS=" " } { printf("%s\n", $0) }' | egrep '(^launch)|(^os/exec)|^C|(^net)|(^io/ioutil)|(^$)' | 12:51 |
| Aram | oops | 12:51 |
| Aram | wrong window :). | 12:51 |
| Aram | wrong paste, rather. | 12:51 |
| niemeyer | No passwords at least! ;-) | 12:57 |
| Aram | niemeyer: I don't understand what path is used for in Coerce. | 13:46 |
| niemeyer | Aram: Presenting debugging details in case of errors | 13:53 |
| niemeyer | Aram: "a.b.c is wrong" | 13:53 |
| Aram | why is path always initialized like this in tests? path := []string{"<pa", "th>"} | 13:54 |
| niemeyer | Aram: No good reason.. | 14:15 |
| Aram | ok, I though it had some meaning and could find any :). | 14:17 |
| Aram | great. | 14:17 |
| Aram | rogpeppe: I am already in love with your acme def tool, I used codesearch for similar behavior, but that was noisy, yours is 100% accurate, no visual grepping needed. | 14:21 |
| rogpeppe | Aram: cool. yeah, i love it too. | 14:22 |
| rogpeppe | Aram: it was even better when i hacked godoc so that the identifiers in the html source code were links | 14:23 |
| rogpeppe | Aram: but that went by-the-by sadly | 14:23 |
| Aram | yeah, I imagine. | 14:23 |
| rogpeppe | Aram: i find it really useful for working my way around strange code bases. | 14:23 |
| rogpeppe | niemeyer, fwereade: in machine/unit.py, UnitMachineDeployment seems identical to SubordinateContainerDeployment. is there a reason for the separation? is there something that is likely to come later, for example? | 14:25 |
| * fwereade looks | 14:26 | |
| niemeyer | rogpeppe: I haven't seen that code at all, FWIW | 14:26 |
| niemeyer | rogpeppe: Haven't reviewed that code | 14:26 |
| rogpeppe | the init methods are byte-for-byte identical | 14:26 |
| niemeyer | rogpeppe: I hope we can take the hindsight to clean and simplify things up in the agents, as a general comment | 14:27 |
| rogpeppe | niemeyer: i certainly intend to do what i can towards that | 14:27 |
| rogpeppe | niemeyer: first blocker i can see is there's no unit watcher | 14:27 |
| fwereade | rogpeppe, yeah, I don't see the point of Subordinate... | 14:28 |
| rogpeppe | fwereade: thanks | 14:28 |
| fwereade | rogpeppe, and it's new to me | 14:28 |
| Aram | why is CurrentArch in environs runtime.GOARCH (dependant on where the user runs juju) but CurrentSeries is hardcoded to precise (independent of where the user runs juju), why aren't both hardcoded since they are related only to the cloud environment and not to the local environment? | 14:32 |
| Aram | Is it because the default is used in things like testing in a lxc container? Why not set up correct arch in that particular test? | 14:32 |
| rogpeppe | Aram: CurrentSeries is TODO | 14:32 |
| rogpeppe | Aram: it's only just gone in | 14:32 |
| rogpeppe | Aram: and i haven't done that bit yet | 14:33 |
| rogpeppe | in fact, i should probably clean that up before starting on the machine agent | 14:33 |
| Aram | I see. | 14:33 |
| rogpeppe | Aram: CurrentArch needs looking at too, as runtime.GOARCH may not map well to linux arch | 14:33 |
| Aram | well yes, I didn't question CurrentSeries, but CurrentArch seemed peculiar to me :). | 14:35 |
| rogpeppe | fwereade: i was just looking at your upstart package again | 15:07 |
| rogpeppe | fwereade: the %q seems dubious (is upstart quoting exactly the same as Go's?) but i couldn't find a definitive reference... | 15:08 |
| rogpeppe | fwereade: (i'm referring to the %q as used to quote env vars in the upstart script BTW) | 15:11 |
| niemeyer | rogpeppe: Isn't the string well known? | 15:25 |
| rogpeppe | niemeyer: Conf.Env looks like it might contain arbitrary text to me. | 15:26 |
| rogpeppe | niemeyer: but i've probably misinterpreted | 15:26 |
| smaddock | niemeyer: ping | 15:27 |
| niemeyer | rogpeppe: No, I think you got it mostly right.. the package is the thing that is a bit questionable I guess | 15:27 |
| niemeyer | rogpeppe: We're really using it mostly for our own configuration | 15:27 |
| niemeyer | rogpeppe: But it might change in the future | 15:27 |
| niemeyer | smaddock: Heya | 15:27 |
| rogpeppe | niemeyer: that's true. but *if* we're going to do this package, i think it should be correct for any input. | 15:27 |
| * rogpeppe hates it when people define a config format but don't properly define the quoting rules. | 15:28 | |
| fwereade | rogpeppe, hmm, that sounds like a nice catch | 15:28 |
| niemeyer | rogpeppe: Yeah, or add a BUG note | 15:29 |
| rogpeppe | fwereade: am currently looking at libnih trying to work out what the actual quoting rules are... | 15:29 |
| fwereade | rogpeppe, heh :( | 15:29 |
| rogpeppe | niemeyer: that too. | 15:29 |
| niemeyer | Given what's there should for the foreseeable future | 15:29 |
| fwereade | niemeyer, rogpeppe: I'll add a bug note | 15:30 |
| rogpeppe | i thought originally that it was shell script, where using %q would be more problematic | 15:30 |
| rogpeppe | always fun when someone rings up and tells you you've got a virus and they can fix it | 17:11 |
| rogpeppe | i couldn't quite work out what the scam was though. what nasty thing could they do by looking at the eventvwr app? | 17:12 |
| rogpeppe | darn, gotta go, i'm late! | 17:13 |
| fwereade | later all :) | 17:27 |
| davecheney | Aram: did you get the tests to pass ? | 21:21 |
| Aram | davecheney: it's upstream, not in my environment. niemeyer confirmed. | 21:34 |
| davecheney | reitveld is having a seniors moment | 22:37 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!