[01:08] * davecheney starts to salivate [01:08] http://shopap.lenovo.com/au/en/products/laptops/thinkpad/thinkpad-innovation?cid=EDM_20120828_ANZ_AU_CON_X1Carbon_ShortRange_SL&RRID=222186471&esrc=EPI2JANZ [01:23] davecheney: Wow, sweet indeed [03:42] * mramm2 has no love for my ISP today -- but they are sending out the second tech support person in 2 days tomorrow morning [06:45] hello, is there an mstate.Open ? [06:45] i can't find it, i must be dumb [07:04] good morning [07:06] TheMue: good morning [07:06] do you know if there is an mstate.Open method ? [07:06] hiya dave [07:07] davecheney: sorry, don't know if it already exist [07:07] TheMue: that is sad [07:07] is there an mstate.Info ? [07:08] davecheney: dunno too, i've just started with lifecycle and test completion [07:08] davecheney: so you just have to scan the code [07:08] TheMue: cool [07:08] thanks [07:08] i'll ask aram [07:14] davecheney: at least in trunk they don't exist [07:14] TheMue: does mongo have anything like the concept of zookeepers' fallover addresses ? [07:14] davecheney: and aram currently focusses on txn and watchers [07:27] davecheney: afaik the concept is different, see http://www.mongodb.org/display/DOCS/Sharding+and+Failover. but here i'm not deep enough into both systems. [07:31] TheMue: ta === TheMue_ is now known as TheMue [08:09] fwereade, TheMue: morning [08:09] rogpeppe, heyhey [08:10] rogpeppe: hi, had a nice day off (spending your time on wedding photos)? [08:10] fwereade: hello [08:10] fwereade: any chance you could run a very brief live test for me? i can't work out if i've mucked up my amazon stuff or if our code has gone wrong [08:10] TheMue, heyhey :) [08:10] TheMue: yes thanks [08:10] rogpeppe, heh, ok, sure; in 5 mins? [08:10] fwereade: np [08:29] rogpeppe, ok, that wasn't 5 mins, but maybe it actually will be in 5 more mins -- would you let me know what I need to do now? [08:30] fwereade: go test -amazon -gocheck.vv launchpad.net/goaws/ec2 [08:30] fwereade: (assuming you've got valid AWS_ environment variables set up) [08:31] oops [08:32] s/goaws/goamz/ [08:33] oops again [08:33] fwereade: go test launchpad.net/goamz/ec2 -amazon -gocheck.vv [08:33] flags after packages, but only for go test :-) [08:34] rogpeppe, failures: http://paste.ubuntu.com/1171358/ (I managed to figure that bit out at least) [08:34] fwereade: interesting, but not the failures i was looking for :-) [08:35] fwereade: i'm seeing signature failures [08:35] rogpeppe, ha, sorry :( [08:36] fwereade: the weird thing is that the python juju works ok. [08:36] fwereade: time to make a minimal failing example, i think [08:39] rogpeppe, something's scratching at my mind about signed urls, I'll let you know if it turns into anything real [08:39] fwereade: ta. it's really odd - some test example code works. [08:50] moin. [08:58] hi Aram [09:03] fwereade: one last test, just to make doubly sure: could you make sure you've got the latest goamz version (cd $GOPATH/src/launchpad.net/goamz; bzr pull) and run this program, with the auth details i gave you earlier substituted as appropriate.. http://paste.ubuntu.com/1171398/ [09:03] fwereade: i'm finding it all a bit weird [09:04] fwereade: or just say bugger off if you're too busy... :-) [09:04] rogpeppe, huh, sorry, looks like I wasn't up to date :/ [09:05] rogpeppe, and np at all I'm watching other tests atm, did something stupid :) [09:05] rogpeppe, bingo, signature does not match [09:05] phew [09:05] fwereade: and if you use your own credentials? [09:06] rogpeppe, doing that now [09:08] rogpeppe, yep, same failures [09:08] rogpeppe, sorry about that :( [09:08] fwereade: lovely thanks [09:08] fwereade: no, that's good! [09:08] fwereade: and when i reverted to revision 8, it all works [09:08] rogpeppe, nah, just sorry about the dumb version thing to begin with [09:09] fwereade: np, i'd've done the same [09:10] fwereade: the three line change between r8 and r9 is the culprit [09:12] Aram: just to get sure, the agreement has been that all reads on one or all entities with a lifecycle returns them regardless if of their life state, isn't it? [09:13] TheMue: yes. [09:14] Aram: ok, i'm currently doing services and machines and will change that too where needed [09:14] ok. [09:21] fwereade: i think i've found the problem [09:21] rogpeppe, oh yes? [09:21] fwereade: the urls don't end with a slash [09:21] rogpeppe, ha! [09:23] aaaaaaaaaaaaaaaaaaaaand we have a (rudimentary) Uniter merged :D [09:23] fwereade: right, now i can have some breakfast [09:23] fwereade: yay! [09:24] fwereade: i will eat muesli in celebratory mood! [09:24] rogpeppe, heh, I should probably do the same in a mo :) [09:49] sorry guys, need to pop out for a bit, bbs [10:09] lunchtime [10:50] biggup! [10:52] heya davecheney [10:55] has everyone got their UDS invite yet ? [10:55] no [10:55] ffs, i told michelle that there was a problem, but she didn't believe me [10:55] eventbrite has this web bug they put on their email, so they claim it has been 'opened' [10:55] like that could ever be wrong [10:55] hmm [10:55] I do have an invite, now that you made me look [10:56] bwahahah [10:56] 14 days ago, actually [10:56] heh [10:56] sssh [10:57] Ubuntu Developer Summit - R [10:57] what's the 'R'? [10:57] Aram, letter after Q [10:57] it's the next letter after Q [10:57] jynx! [10:58] * fwereade gesticulates wildly but emits no sound [10:58] fwereade: ! [10:58] sorry guys 2 mins [10:58] at an old workplace, the IRC bot had a !jynx command [10:59] one guy spent far to much time teaching it levenshtein distances so it could figure out who to mute [11:02] Yo! [11:02] hiya niemeyer [11:02] hey [11:03] Anyone has the invites out yet [11:03] ? [11:03] nup [11:03] Sending [11:07] rogpeppe? [11:07] niemeyer: yo! [11:07] niemeyer: meeting, i guess [11:08] * rogpeppe goes to fetch the other computer [11:14] https://bugs.launchpad.net/juju-core/+bug/1042604 [11:14] https://bugs.launchpad.net/juju-core/+bug/1042579 [11:15] https://bugs.launchpad.net/juju-core/+bug/1038296/comments/5 [11:17] https://bugs.launchpad.net/juju-core/+bug/1038296/comments/6 [11:23] Whoop whoop! [11:54] davecheney: I'm not sure if you want to talked to talk a bit more about the problem [11:54] davecheney: Do you wanna brainstorm on it for a moment? [11:54] niemeyer: sure [11:54] davecheney: Cool, so I'll pick the code to follow alone [11:54] along [11:55] Wow.. interesting typo [11:55] is the hangout still active ? [11:56] davecheney: Would you rather use G+? Cool.. I've sent it again [11:56] jynx! so have I [11:57] niemeyer: https://plus.google.com/hangouts/_/649cd97bfeb012132fd9f58aeaf998dfd90329b2 [11:57] ^ does that work ? [12:05] niemeyer: svc, err := s.Conn.AddService("test-service", charm) [12:05] c.Assert(err, IsNil) [12:05] err = svc.SetExposed() [12:05] c.Assert(err, IsNil) [12:05] units, err := s.Conn.AddUnits(svc, 1) [12:05] c.Assert(err, IsNil) [12:05] err = units[0].OpenPort("tcp", 999) [12:05] c.Assert(err, IsNil) [12:07] davecheney, TheMue: i think the firewaller probably needs to be changed so that it only adds machines to the machineds map when the machine has an instance id. [12:08] davecheney: or, better perhaps, the firewaller could keep the set of current instances, and change it when a machine's instance id changes [12:09] rog: InstanceId() of machine already returns an error if unset and that is caught in firewaller line 202 [12:10] TheMue: yes, but this is something that happens in the normal course of things. we don't want the provisioner to be restarted each time it happens. [12:10] TheMue: i'm not sure that retry logic is the right fit here either [12:12] TheMue: hmm, mind you perhaps retry is ok here, in a kinda hacky sort of way, because we know that the firewaller is in the same process, so it will be seeing the same changes and be reacting pretty quickly. [12:12] rog: if we don't have an instance id retrying to open them doesn't help ;) so indeed there may be a need for (a) watching for the instance id [12:12] TheMue: that's what i'm thinking [12:12] TheMue: (we already have a watcher that can do that) [12:15] rog: sadly i don't see the according error message in the log. here i'm wondering [12:16] rog: ah, found, looked too high [12:17] TheMue: looking at the code, i don't think it would be too hard. [12:17] ... maybe [12:20] rog: depends on the strategy [12:20] rog: wait there less or more good wrapped but blocking [12:20] TheMue: syntax error :-) [12:20] rog: or start a kind of async port opener in an extra goroutine [12:21] more or less [12:21] TheMue: i didn't understand that first sentence, but i don't think the latter is a good idea [12:21] TheMue: if we let ourselves be led by the model, a Machine's instance id can change at any time, and we should track that. [12:22] rog: 1st sentences is a kind of instanceId, err := machined.machine.WaitInstanceId() [12:22] TheMue: i don't think that's a great idea either [12:22] rog: no, because it blocks the fw [12:22] TheMue: i'm thinking we should maintain another map inside the Firewaller [12:22] TheMue: that maps machine id to instance id [12:23] TheMue: or to environs.Instance, better perhaps. [12:23] TheMue: there's something i'm trying to understand about the current firewaller; maybe you can explain [12:23] rog: i'll try [12:24] TheMue: it never calls Instance.Ports, so if the firewaller is restarted, how can it know what ports to close on an instance? [12:26] rog: afaik we once talked about it. i just pass the instance the ports i want have opened or close, regardless if they are alredy open or closed [12:27] TheMue: that's fine for opening ports (open is idempotent, and when we start, we assume no ports are open), but i think it fails for closing ports. [12:27] sorry lads, wasn't watching what you were writing [12:27] was talking to gustavo [12:28] TheMue: Please leave that to davecheney [12:29] rog: why does it fail? [12:29] niemeyer: ok [12:29] TheMue: because when you start up, you need to close any ports that are currently open but that are not mentioned in the state [12:30] TheMue: but unless you call Instance.Ports, you can't know what those are [12:30] ARGH! [12:30] why does LP not have a 'report a bug' link on the milestone page [12:30] I spend my life on that page and there is no bloody link to create a _new_ bug for this milestone [12:31] davecheney: think of daisies, la la la [12:32] niemeyer: is rog with this fw-startup-port-closing right? if so we should file a bug. [12:33] rog: please file a bug [12:33] rog: also, is there a bug for the AMZ breakage [12:34] davecheney: yes, someone else had already filed one [12:34] I have a shittone of 'doesn't work in XYZ region' bugs that I am working through [12:34] TheMue: interestingly, this was an issue that we didn't have in the code sketch that i originally proposed for the firewaller [12:35] TheMue: (well, perhaps... :-]) [12:36] ;) [12:37] davecheney: i'll write a test that breaks first, then i'll file a bug :-) [12:37] rog, you've done this before :) [12:43] davecheney: yup, test fails as expected [12:44] niemeyer: https://bugs.launchpad.net/juju-core/+bug/1042717 [12:44] does this capture (part) of the discussion we just had [12:45] davecheney: i think AllInstances only returns running instances [12:45] davecheney: (or pending) [12:45] davecheney: As far as terminology goes, I suggest stopped vs. terminated [12:45] davecheney: That's what EC2 uses [12:45] ya'all can edit the ticket, please have at it [12:46] sorry all [12:46] mramm2: did my text make it ? [12:46] yep [12:46] and my phone was right next to me [12:46] mramm2: if you wanna have a quick catchup now while everyone is online, lets do it [12:47] but I slept through it [12:47] davecheney: Looks good [12:47] davecheney: I was wondering a bit about (2) [12:48] davecheney: Do we need it right now? [12:48] davecheney: what do you think we should do with a stopped machine? its agent will appear dead. [12:48] niemeyer: ^ [12:48] rog: discuss with niemeyer, he dug up that corpse [12:49] hmm, yeah, it might be problematic if we find we're running two unit agents for the same unit [12:50] but i suppose that's a problem with a down network connection too [12:50] TheMue: Sorry, I did read the discussion, but it's not clear to me what problem is being fixed, or why we should change something there [12:50] niemeyer: this one is a bit pithier [12:50] niemeyer: the problem is that the firewaller sees a new machine and tries to change its ports, but the machine hasn't yet been allocated an instance. [12:50] davecheney: Which one? [12:51] niemeyer: https://bugs.launchpad.net/juju-core/+bug/1042721 [12:51] paste fail [12:51] niemeyer: so the firewaller dies [12:51] rog: I've discussed this with davecheney already.. there's zero reason to change ports for an instance that doesn't exist [12:51] niemeyer: indeed [12:51] niemeyer: which means the firewaller should watch each machine to see when its instance id changes, i think [12:52] rog: I don't get the leap [12:52] niemeyer: when should the firewaller open the ports for a new instance? [12:52] davecheney: +1 [12:52] niemeyer: my only question is [12:53] which ports are we talking about, the ones in the state, or the ones in the security group of the provider ? [12:53] rog: When it gets an open port watcher firing for an instance within it and its service was exposed, in either order [12:53] davecheney: State [12:53] niemeyer: open port watchers fire for machines, not instances [12:53] davecheney: We've agreed back then that StartInstance never gives back something with ports open [12:53] rog: It fires for units, actually [12:54] niemeyer: sure [12:54] niemeyer: ok, then in the pathological case, all the units' machines have been replaced [12:54] rog: Which live within an instance when they are running [12:54] and the service is now offline [12:54] rog: No instance, no unit [12:54] niemeyer: but when you see that state change, you haven't necessarily got an instance to change the ports on [12:54] rog: Impossible [12:54] niemeyer: really? [12:55] rog: The unit lives within the instance.. if there's no instance, there's no uniter, and thus no open ports [12:55] niemeyer: ahh, and you just answered my question, when the instance is replaced, the new uniter will react to 'exposed' and do what it needs [12:55] niemeyer: ah... [12:55] niemeyer: i'd forgotten that rub [12:55] niemeyer: brilliant! [12:55] the missing piece [12:55] niemeyer: so the test is wrong. [12:56] it's all my fault :-) [12:56] I setup a hangout: tps://plus.google.com/hangouts/_/0fbf3a66c1e6ee955123854516a78f947aa621cb [12:56] davecheney: Yeah, or in install/start whatever.. it's free to run open-port in any hook [12:56] not required, but if you want to chat [12:56] feel free to join [12:57] rog: Well, kind of.. as discussed with davecheney, the test is also right [12:57] rog: It's just a different unit tests [12:57] rog: It shouldn't blow up in such a state [12:57] rog: it's wrong in the way a hat made of bacon is wrong [12:57] davecheney: are you dissing my bacon hat? [12:59] niemeyer: should it just ignore the error? [12:59] niemeyer: the firewaller, that is [13:00] rog: Yeah, if the instance is gone, it can ignore it.. the provisioner will close ports in state, fire a new instance, assign to it, and the new uniter will open it back again [13:01] niemeyer, davecheney: so this should fix that particular test: http://paste.ubuntu.com/1171742/ [13:03] niemeyer: me looks [13:04] niemeyer: that should work _almost_ work [13:04] davecheney: almost? [13:04] there is a race between the call to dummy.StartInstance() returning and the intance id hitting the state [13:05] it is a much smaller window than currently exists [13:05] davecheney: shit yeah [13:05] davecheney: i knew about that before [13:05] davecheney: just forgotten it [13:05] * rog is groggy today [13:05] rog: http://codereview.appspot.com/6482081/ [13:05] your thoughts would be appreciated [13:06] note, setting a value higher than about 10ms will cause jujud tests to hang [13:07] davecheney: Huh.. a good one to inspect later :) [13:07] rog: niemeyer : booohhh https://bugs.launchpad.net/bugs/1042545 [13:10] davecheney: why bother putting the delay time in the environState if it's actually global? [13:10] rog: TheMue requested that we be able to change it [13:10] davecheney: ah [13:10] so in theory, we could change it by type aserting to dummy, then reaching in and changing the value on a per test basis [13:11] davecheney: not really [13:11] davecheney: it's all unexported [13:11] davecheney: i'd prefer a possible entry in the dummy environ configuration attributes. [13:11] davecheney: if we wanted an override [13:13] rog: good point [13:13] rog: Yeah, that sounds sensible [13:13] I've suggested a function in the dummy package, but an attribute setting seems even nicer [13:13] environment setting [13:13] niemeyer, davecheney: i'm not sure about it though (an override that is) [13:14] davecheney: That said, I still think there's value in having a flag to enable the delay globally [13:14] when would it be appropriate to use the override? [13:14] davecheney: May be done with a command line flag, though [13:14] davecheney: as suggested in the review [13:14] niemeyer: i think on balance i prefer the environment variable, as it means it's easy to run all tests with the delay enabled. [13:15] rog: -dummy.delay 10s is just as easy [13:15] I'd prefer to stay out of the business of env variables [13:15] niemeyer: i can't do: go test launchpad.net/juju-core/... -dummy.delay 10s [13:15] For that purpose [13:15] niemeyer: that doesn't work when you do: go test launchpad.net/juju-core/... [13:15] what rog said [13:16] Hmm.. okay [13:16] we'll put it in a SOAP web service that the tests query using CORBA. [13:17] Aram: don't forget perl, you'll need lots of perl [13:18] davecheney: Okay, so why all the fanciness? var delaySecs = os.Getenv("..."); func delay() { if delaySecs != "" { time.Sleep(...) } }? [13:19] niemeyer: TheMue requested that it be changable [13:19] niemeyer: +1 (assuming we don't allow delay overriding with the config) [13:19] davecheney: Ah, we have time.ParseDuration actually [13:19] which I now realise didn't work [13:19] Aram: i'm missing the JEE server in the middle receiving the requests and writem the via mqseries into an oracle where we could fetch them [13:20] TheMue: why do you want the delay overridable? [13:20] Which you're already using [13:20] niemeyer: rog so if in principle you are in favor, I'll resubmit that CL tomorrow with something simpler [13:20] env var -> delay() [13:20] davecheney: Yeah, definitely [13:20] davecheney: we can also improve it in the future as we see necessary [13:20] davecheney: i think it's a good idea. i don't think we want a configurable delay - i can't think when it would ever be appropriate to use it. [13:20] davecheney: Potentially approaching the mgo/txn's Chaos stuff [13:21] niemeyer: +1 [13:21] davecheney: But it's not worth it for now [13:21] rog: just has been a quick idea after dave mentioned the topic to test different ranges [13:21] rog: if the implementation concept now makes it useless it's fine for me [13:21] all: i agree, i just need a way to make dummy slow down to better simulate a real provider [13:21] davecheney: +1 [13:22] davecheney: +1 [13:22] davecheney: +1 [13:22] right, i'll try out your test sugggestion tomorrow niemeyer [13:23] davecheney: Cheers man [13:23] niemeyer: thanks for the discussion, i can see a straightforward way for the PA to implement those two tickets we discussed [13:30] davecheney: Superb, thanks for finding the issue! Glad to see these bugs being fleshed out. [13:35] Okay, so reviews, and then presence [13:40] TheMue: ping [13:40] niemeyer: pong [13:40] TheMue: We'll need to fix the Kill methods.. they're changing the cached state to Dying irrespective of previous state [13:40] TheMue: We shouldn't do Dying > Dead [13:41] niemeyer: ouch [13:41] TheMue: Adding a comment [13:41] niemeyer: oh, yes, now i see it [13:41] TheMue: It's fine to move on as they are for the moment, and fix all of them at once in a follow up [13:42] niemeyer: both new ones for service and machine are like the ones for unit and relation [13:42] TheMue: Yeah, it's all good [13:42] TheMue: We can have a new CL that follows up on that and fixes all three at once [13:42] TheMue: (with a test!) [13:42] fwereade: could you just quickly give a high level overview of how constraints work? in particular, who solves the constraints? the PA or the client? I'm presuming the former, but just need to check. [13:43] niemeyer: when i've got your ok i'll change it i'll do the followup [13:43] niemeyer: +1 [13:43] TheMue: Cool [13:43] rog, well, "solve" is frankly a bit of a strong word -- we basically match against a list of instance types, sort by cost, and pick the first [13:44] fwereade: yeah, but until that's done we don't know the architecture that's going to be used for the new unit, right? [13:44] fwereade: or series [13:44] fwereade: but that "solving" is done by the PA? [13:44] rog, ok, the series is known before we even start on constraints [13:44] rog, that's defined by the charm [13:45] rog, in every existing case, arch defaults to amd64 but can be set to i386 if desired [13:45] rog, but remember this is pretty ec2-specific [13:45] fwereade: yeah, i don't want to do anything provider-specific here [13:46] fwereade: so am i right that the PA does the matching? [13:46] rog, and -- well, *probably* it should be up to the PA, but at present no it is not [13:46] fwereade: ah, so the add-unit command works out the architecture etc then adds the new unit with those set? [13:46] rog, but I know niemeyer is -1 on this, and we hashed it out last UDS, so... yes, it is up to the PA [13:46] rog, er, IYSWIM [13:47] rog, yeah, that was what it did [13:47] fwereade: thanks, that's useful [13:47] rog, the reasons for it doing so are not especially interesting, and the actual choice procedure shouldn't really be any different [13:48] fwereade: it makes a difference for upgrading, interestingly [13:48] fwereade: i'm writing a little description of my current problem [13:49] TheMue, Aram: ping [13:49] pong [13:49] rog, ah, ok :) [13:49] Aram, TheMue: Can we quickly talk about the last point here: https://codereview.appspot.com/6495043/ [13:49] niemeyer: pong, sorry, have been afk for a moment [13:49] niemeyer: yes [13:50] niemeyer: what point, this? mstate/service.go:45: s.doc.Life = Dying [13:51] Aram: The last one in the review [13:51] niemeyer, fwereade: here's a description of my current upgrading difficulty: http://paste.ubuntu.com/1171829/ [13:51] niemeyer: currently RemoveUnit() would run on an error if the units are not dead [13:51] niemeyer: but it has to be put into a complete txn later [13:51] rog: Sorry, I'm covering a different issue right now [13:51] niemeyer: that's fine. but when you have a moment, i'm blocked on this. [13:51] TheMue: That's unrelated to transactions [13:52] TheMue: This is about the lifecycle behavior [13:52] niemeyer: otherwise we start to remove relations, remove unitts and may beak before deleting the service [13:52] niemeyer: I thought we had agreed that units should listen for their service and delete/die themselves. [13:52] TheMue: RemoveService is abruptly *removing units from state*, despite whatever state they're in [13:53] niemeyer: ok, reducing it to lifecycle the solution should be to let all units die if Die() is called on a service [13:53] niemeyer: today it's not working this way [13:53] TheMue: Of course it's not.. that's why we're doing the lifecycle stuff in the first place :) [13:53] TheMue: Which is why I'm asking what's the plan [13:54] Aram: Yes [13:54] Aram: Not delete [13:54] yes, only die. [13:54] Aram: Kill themselves, actually [13:54] Aram: and then die, you're right [13:54] Aram: The deletion is the bit that is done outside [13:54] (by the machine agent) [13:55] niemeyer: cool, we're on the same page. it hasn't been done yet because we were lacking watchers, I haven't forgoten about it. [13:55] Aram: Still, there's a problem in that RemoveService implementation.. doesn't look like that's what we want [13:55] well no, now it isn't. [13:55] but the plan is to change it after we have watchers. [13:56] Aram: Okay, I'm wondering because we're changing it now to claim lifecycle integration, [13:56] Aram: and it feels quite bogus from a lifecycle perspective [13:56] nah, the claim is wrong. it's a step, but not the final step. [13:56] there's more work to be done. [13:57] Aram: Cheers [13:58] niemeyer: how should "kill themselves" happen? [13:58]