=== slank is now known as slank_away [07:49] davecheney: ping [07:49] mornin' all! [08:19] wallyworld_: you around? [10:18] hello. [10:31] aram, dimitern: hiya [10:32] rogpeppe, aram: morning :) [10:32] dimitern: openstack tests are broken in trunk BTW, if you don't have OS_TENANT_NAME set [10:32] rogpeppe: hmm [10:32] rogpeppe: that's now or the branch I proposed yesterday? [10:33] dimitern: currently [10:34] dimitern: it would also be nice if the goose "required environment variable" error message printed the name of the required environment variable rather than the struct field name... [10:34] "required environment variable not set for credentials attribute: TenantName" makes it look as if the required env var is $TenantName [10:35] rogpeppe: tell me about it :) I did it like that originally, but.. anyway I'll take a look [10:51] rogpeppe: fixed https://codereview.appspot.com/7196043 - this is trivial, if you think it's ok, I'll land it now [10:51] mgz: poke [10:52] dimitern: LGTM. agreed trivial. [10:52] rogpeppe: 10x, submitting then [10:55] rogpeppe: how about looking at my bootstrap branch - https://codereview.appspot.com/7181046/ [10:55] dimitern: will do [10:56] jam: hey [10:56] mgz: I had some questions about how to setup tarmac, and thought you might be good to bounce off for feedback. [10:56] I think I have it up and running, and a small patch to handle some stuff I didn't realize. [10:56] However, what branch do we want it to actually manage? [10:57] (do we want to add it to ~gophers, and give it write access there, do we want a different team, a branch only accessble by goose-bot, ... ?) [10:59] (I guess the next direct step is to add support for running juju-core tests before landing trunk code, but I'd like to answer the above first) [10:59] hm, the bzr-pqm way is to have the branch owned by the bot [10:59] mgz: right [10:59] the major downside is the "manual escape hatch" [10:59] but being able to get around it and commit anyway will probably be useful for now [10:59] especially because of wanting to be able to land goose code and juju-core code concurrently [11:00] yup. [11:01] so, maybe we want a new team that the bot can be a member of to own the goose code [11:04] I tend to be cautious, so giving it full '~gophers' access seemed a bit much, but it is also a fair amount of bookkeeping just to do goose. [11:04] yeah. [11:05] jam: but isn't that like the other go projects? all owned by ~gophers? [11:07] dimitern: sure, and all of us have the ability to push whatever we want to them. I'm hesitant to allow a bot the same access. [11:08] it's easy enough to make ~gophers a member of the group that owns goose trunk, we just don't want the bot to have too much power :) [11:08] dimitern: it also stems from a group that doesn't have a habit of using a bot to control their trunks, vs the launchpad group using that method for a long time. [11:08] jam: wouldn't it need to merge and push changes? [11:08] dimitern: goose bot needs to be able to write to goose's trunk [11:08] I'm not sure that it needs rights to everything else owned by ~gophers [11:08] (juju-core, goamz, etc, etc.) [11:09] you don't trust the bot? c'mon :D [11:09] it's a nice guy [11:09] dimitern: the inverse is actually just as important (to me). I would rather the equivalent of having to run 'sudo' to be able to alter trunk. [11:10] so that: yes anyone on our team can fix trunk if we need to, but we have to do so very explicitly instead of accidentally [11:10] jam, makes sense - preventing accidental mistakes, provided it doesn't impede normal workflow [11:12] dimitern: reviewed [11:12] so mostly I'm trying to find the tasteful point in balancing these desires, adding goose-bot to ~gophers is the most expedient solution, though not one I personally control. [11:12] as is having the branch owned explicitly by goose-bot [11:13] I suppose goose-bot the launchpad account could just have us all know the password, and all have our ssh-keys attached to the account. [11:13] mgz: ^^? [11:13] that would give us "do X to become the goose-bot superuser" [11:13] rogpeppe: tyvm [11:14] mgz: ah, but the problem ends up that only goose-bot can approve changes to ~goose-bot/ branches. I'll have to look into the review side of things to figure out how to fix that [11:14] that would probably be okay [11:15] mgz: looks like you can set a "reviewers" on a branch [11:15] which I can just make ~gophers [11:15] yeah [11:15] so they can approve branches even though they can't commit directly to it. [11:35] jam: poke? [11:35] rogpeppe: i am around now, am in a stand up [11:35] wallyworld_: np, the issue's been fixed now. [11:36] ok, sorry i was at soccer training [11:58] wallyworld_: i only pinged you 'cos you were the only one that showed up on the IRC user list :-) [11:58] ah ok [11:58] i didn't log out at my EOD [11:59] a couple of CLs if anyone wants. the first one is pretty trivial, the second a bit bigger: https://codereview.appspot.com/7197043/ https://codereview.appspot.com/7133063/ [11:59] rogpeppe: I'm on the first one [11:59] dimitern: thanks === jtv1 is now known as jtv [12:04] rogpeppe: reviewed [12:04] dimitern: thanks [12:04] anyone else wanna take it to the magic two LGTMs? :-) https://codereview.appspot.com/7197043/ [12:05] rogpeppe: it's weird though, unlike the usual case the first comment wasn't "Please take a look" from you [12:06] dimitern: ha, i'd forgotten to lbox propose it without -wip [12:07] dimitern: it's got that message now :-) [12:07] rogpeppe: ok :) [12:24] niemeyer: yo! [12:25] rogpeppe: Heya! [12:28] niemeyer: hey! [12:28] dimitern: Hey there [12:28] niemeyer: would you like to take a look at this - https://codereview.appspot.com/7181046/ [12:34] mgz, wallyworld_, dimitern: I just sent you guys an email with the security keys, etc you'll need. I used your public GPG keys, but if you need help figuring out how to decode them, just ask. [12:34] jam: cheers [12:37] jam: yep, I have to replace that PGP key for which I forgot the passphrase a long time ago [12:38] jam: I'll do it today and you can resend it with my new key perhaps? [12:42] niemeyer: small CL to make add multiple addrs to the API: https://codereview.appspot.com/7197043/ [12:43] dimitern: CHecking [12:43] thanks [12:43] rogpeppe: Super, thanks [12:45] rogpeppe: Btw, have you had a chance to check that bug Dave filed? [12:45] niemeyer: which one? [12:45] rogpeppe: The one saying bootstrap is broken [12:46] niemeyer: i couldn't reproduce it. i suspect he was running without --upload-tools. [12:47] rogpeppe: Have you tried in the regions he suggested? [12:47] rogpeppe: Or at least one of them? [12:48] niemeyer: i'll do that. the error in the bug report "error: API entity name not found in configuration" should not be possible to happen in the branch he was talking about. [12:48] niemeyer: but there may well be another bug [12:48] rogpeppe: I'm poking because he asked there, so it'd be good to at least interact to clarify the bug [12:49] Since Dave isn't on the same timezone, if we don't get back to him on async locations, we won't talk [12:51] dimitern: Any significant change to the ec2 logic on that bootstrap function? [12:52] niemeyer: how do you mean? [12:52] niemeyer: no, it's essentially the same [12:52] niemeyer: the Bootstrap() code, that is [12:54] niemeyer: i'm trying it now [12:55] rogpeppe: Thanks a lot [12:55] dimitern: Cool, I was really just curious if there was anything that had to be done differently there [12:55] dimitern: It's great that there isn't (except for Roger's point in the CL, which I agree with) [12:56] dimitern: Have you seen rogpeppe's comment? [12:56] dimitern: Are there consistency issues there too? [12:56] niemeyer: well, it's seems not - both tests (svc doubles and live) pass - although on canonistack the lack of floating IPs fails the test, but everything else works [12:57] niemeyer: about consistency - not sure how to test that, but I've run into other consistency issues with OS on some API calls [12:57] dimitern: Oh, consistency issues and "seems" don't go well together :-) [12:58] dimitern: The idea, in S3, is that the answer is totally arbitrary for an undefined period of time [12:58] dimitern: i suggest starting without any of the eventual-consistency loops, and seeing if things still work. [12:58] dimitern: You create, and it's not there.. you delete, and it's still there [12:58] rogpeppe: Please no [12:58] niemeyer: I've had the same issues with swift while doing live tests [12:58] rogpeppe: We already have the logic for that.. let's not introduce the bug to see if we can hit it [12:59] niemeyer: creating/deleting/modifying containers in quick succession [12:59] rogpeppe: I agree with you cleaner would be better, but let's ask someone that actually knows that stuff if OpenStack is consistent or not [12:59] niemeyer: i suppose so. but if there are no such issues in openstack, then that code is more complex for no good reason [12:59] niemeyer: yes, that's a good plan [13:00] rogpeppe: Sure, that's the part I agree.. I just don't think rolling back something we know works well to wait for the bug is great [13:00] dimitern: Oh, then it has the same issues [13:00] rogpeppe: well, we can ask yes, but how can it be consistent, when the arch is the same - distributed, multi-tier, API disconnected from the actual service, etc. [13:00] dimitern, rogpeppe: I suggest just leaving the same logic there then [13:01] niemeyer: yeah, I agree [13:01] dimitern, rogpeppe: If we start to have a few other backends with the exact same logic of Bootstrap, we should factor it out.. [13:01] niemeyer: agreed [13:01] dimitern, rogpeppe: That said, I think OpenStack is perhaps too similar to EC2.. I suggest seeing how the others look like before going over the trouble of generalizing [13:02] niemeyer: +1 [13:02] niemeyer: +1 as well - let's not refactor/optimize prematurely :) [13:06] dimitern: LGTM [13:07] dimitern: Does it work for real already? [13:07] niemeyer: great, 10x [13:07] niemeyer: yes it does on canonistack, partially [13:08] niemeyer: starting the instance works, putting the tools, all ok, except for the floating iP shortage, which fails to add an address to the machine after building [13:14] rogpeppe: can I consider your review as LGTM as well, with the points mentioned addressed? [13:15] dimitern: i'll have a quick once-over, if that's ok [13:16] niemeyer: i have a suspicion that eventual-consistency issues are fouling upload-tools. [13:16] rogpeppe: you mean repropose it after changes before submitting? [13:16] dimitern: yeah [13:16] rogpeppe: sure [13:18] niemeyer: ah, no that's not it. [13:20] rogpeppe: reproposed - https://codereview.appspot.com/7181046/ [13:21] dimitern: looking [13:22] dimitern: LGTM [13:22] rogpeppe: cheers [13:23] niemeyer: bootstrap against sa-east-1 works fine for me, although it failed for me first time because i used a different juju to the one it was uploading [13:24] niemeyer: i suspect that was dfc's issue too [13:24] rogpeppe: Curious.. was the symptom the same? [13:24] niemeyer: similar [13:25] niemeyer: the problem (for me) was that it was making the cloudinit script with an old version of the agent config data structure. [13:27] rogpeppe: How did that change again? [13:27] rogpeppe: This kind of thing is interesting to observe and keep in mind [13:27] rogpeppe: THis is a major breakage [13:27] niemeyer: in my case, i was using a version that produced JSON, not YAML [13:27] rogpeppe: Ah, that [13:27] niemeyer: the problem is that we're not incrementing the version numbers when we make incompatible changes [13:28] niemeyer: which is of course convenient when we're breaking things all the time, but will lead to these kinds of issue [13:30] niemeyer: actually... i don't think we check that the tools we're uploading are compatible with the version that's running. that should be fixed, if true. [13:31] niemeyer: (not that that would've helped here though [13:31] rogpeppe: What I mean is that breaking isn't okay.. the problem is breaking [13:32] rogpeppe: and breaking without even being aware that you're breaking [13:32] niemeyer: i was very aware that i was making breaking changes [13:32] rogpeppe: You should mention that, and coordinate with Dave an update to the tools [13:33] rogpeppe: Since whatever is up there doesn't work anymore [13:33] rogpeppe: Just being aware isn't quite enough :) [13:33] niemeyer: i was presuming dave was updating the tools on every release, given that every release is breaking backward compatibility, pretty much [13:34] rogpeppe: Sorry, I'm talking past you [13:34] niemeyer: the next one is going to too - how big a priority for us is preserving backward compatibility currently? [13:34] rogpeppe: Please coordinate with Dave when you introduce breaking changes. [13:35] rogpeppe: Thanks. [13:35] niemeyer: will do [13:43] lunch [14:20] rogpeppe: Please ping once you're back [14:20] niemeyer: ping [14:20] rogpeppe: Yo :) [14:20] rogpeppe: Do you have some time for a call in.. 10 mins? [14:20] niemeyer: sure [14:20] rogpeppe: COol thanks [14:32] rogpeppe: Will send the invite [14:48] I won't be able to join today's kanban meeting, have to run to some place. mramm, I think this applies to our 1:1 as well (though not 100% yet, I might be back soon enough). [14:48] I may not be able to make the 1 on 1 either [14:49] there are sabdfl meetings all day here in Austin === slank_away is now known as slank [15:02] is anyone gonna make the kanban meeting? [15:02] i'll be there if anyone else will [15:08] fwereade, mramm: hiya [15:08] fwereade: how's tricks? [15:09] rogpeppe, heyhey, not bad thanks [15:09] rogpeppe, and yourself? [15:09] fwereade: not bad at all thanks, pushing on. [15:10] fwereade: missing your reviews :-) [15:10] rogpeppe, awesome [15:10] rogpeppe, I came here with the best of intentions but it's been hard to find the energy ;) [15:10] fwereade: no, i wouldn't expect you to while you're there tbh [15:11] rogpeppe, jolly good :) [15:32] fwereade: hey :) how's texas? === otubo1 is now known as otubo [15:52] Lunch.. biab [15:55] dimitern, heyhey [15:55] dimitern, texas is... texan :) [15:56] dimitern, although that's an unfair thing to say really, this is austin after all [15:56] dimitern, and I've barely left the hotel actually [15:56] dimitern, when not working, sleeping has been high priority [15:56] dimitern, I *am* very happy that my bag was delivered last night [15:56] dimitern, that's 3 timesin a row now [15:56] dimitern, tyvm for reviews btw [15:57] fwereade: wow, lost again? [15:57] dimitern, yep :/ [15:57] dimitern, I'm getting used to it now though [15:57] fwereade: np, my pleasure [15:57] fwereade: hehe, all in there? [15:57] dimitern, yeah, nothing lost [15:58] dimitern, all I really suffered was a brief blood-pressure spike at houston [15:58] fwereade: hmm, that doesn't sound good [15:58] fwereade: how was the flight? [15:59] dimitern, only metaphorical -- a brief internal GAAAAAAAAAAAAAAAAH [15:59] dimitern, flight was way smoother than I expected given the weather at heathrow [16:00] dimitern, greenland was covered in clouds, which was a shame, that place is *beautiful* if you have a nice view from 40000ft [16:00] fwereade: yeah, the cancelled like 250 flights that day [16:00] dimitern, oof, luckier than I thought then :) [16:00] luckily you managed to sneak through [16:01] dimitern, was able to be pretty relaxed about the prospect though, I could always have stayed the night with my sister, she's pretty close [16:01] fwereade: she always seems around when you're somewhere on a trip :) [16:01] dimitern, heh, maybe :) [16:03] hazmat, ping [16:04] fwereade: we have bootstrap working with openstack now, landed today [16:04] dimitern, w00t! [16:04] :) [16:04] yeah, it's taking shape faster now I think [16:05] the provider [16:05] fwereade, pong [16:05] hazmat, I think we have enough people to discuss identities if you can tear yourelf away [16:05] hazmat, mramm has been collecting interested parties [16:08] i'd love a review of this branch, if anyone can spare a mo. i'm stacking up stuff on top of it. it shouldn't be too bad, despite the number of files changed: https://codereview.appspot.com/7133063/ [16:55] dimitern: unfortunately the branch you reviewed earlier was WIP for a good reason - i hadn't got tests to pass yet! [16:55] minor details :) [16:56] dimitern: i've now updated it so they do, but somehow it was necessary to add some more stuff along the way, and it's acquired a dependency. if you could have another look (and at the dependency too), that'd be great. https://codereview.appspot.com/7197049/ [17:19] rogpeppe: sorry, just saw this [17:19] dimitern: np [17:19] rogpeppe: I'll take a look now [17:19] dimitern: thanks [17:20] rogpeppe: while I'm on it, would like to look at https://codereview.appspot.com/7195046 [17:21] dimitern: looking [17:23] rogpeppe: Would be interesting to have your opinion on https://codereview.appspot.com/7149043/ [17:23] rogpeppe: as you're working right around it [17:24] niemeyer: i've pinged dfc for a conversation about this [17:24] niemeyer: hopefully we'll have one tonight [17:24] niemeyer: i think it's the wrong thing to do, but he might have better plans than me, so i want a chat before commenting too much [17:25] rogpeppe: What seems wrong, specifically? [17:25] rogpeppe: Looks quite straightforward to me [17:25] niemeyer: i think it should be a machiner job, not a new subcommand [17:26] rogpeppe: Ah, indeed [17:26] niemeyer: then it works at the same level as the API server, and in the future we'll be able to dynamically adjust servers etc [17:26] rogpeppe: Agreed [17:26] rogpeppe: you've got a review [17:26] dimitern: thanks a lot [17:26] rogpeppe: Would you mind to point that out in the CL? [17:26] rogpeppe: Even if you include "Let's talk live about it."? [17:26] niemeyer: ok, i will do [17:27] rogpeppe: Thanks! [17:27] rogpeppe: The stater itself looked nice.. I've skimmed over too quickly thinking it was a job as well [17:28] niemeyer: yes, the stater itself is ok, but i'm not sure it works very well as a task [17:28] niemeyer: most of the logic is fine though [17:29] rogpeppe: Maybe it needs additional stuff, but what's there seems well structured, and necessary [17:30] niemeyer: yes. i'm not entirely sure that worker/stater is the right place, but all the unpacking and running logic is good, which makes up the bulk. [17:30] rogpeppe: Seems right as a worker too [17:30] mgz: ping [17:31] niemeyer: i was wondering if the tar unpacking code might live nicely in a standalone package, but it's probably ok where it is until we need it elsewhere. [17:31] rogpeppe: Indeed [17:33] niemeyer: i'm going to try to catch dave tonight - about when he starts work. [17:33] rogpeppe: Super, cheers [17:44] niemeyer: if you have time, take a look at https://codereview.appspot.com/7195046/ [17:52] * rogpeppe just started running first live test that connects to the API server... i wonder if it'll work. [18:00] rogpeppe: did it? :) [18:00] mgz: still running... [18:01] mgz: oh, darn [18:01] mgz: failed [18:01] ;_; [18:01] mgz: ah, you're here :) can you look at this https://codereview.appspot.com/7195046/ - especially the way private and public addresses are discovered (ported the logic from py-juju) [18:01] dimitern: sure [18:02] mgz: thanks [18:02] you ported the tests too, right? :) [18:03] mgz: well, not really, but I test for all paths I think [18:03] the tests are the bit with value, because they cover the sort of json I actually saw in the wild [18:04] mgz: I see, then I could port their essence as well [18:07] hm... and GetServerAddresses really shouldn't be needed, it's included in servers/{}/detail which juju is likely using in all cases [18:08] mgz: unfortunately, it's not the same always [18:09] that's an ongoing issue though, the ip/name bindings aren't static [18:09] unless you mean something else? [18:09] mgz: sometimes (at startup, but it happened to me multiple times after some uptime) the address fields are empty there, while the /ips returns them correctly [18:09] right [18:10] well, ips will only work once a binding exists [18:10] but that's pretty much the same thing [18:10] what binding? [18:10] i'm not talking about floating ips [18:11] when you first start a machine, you get details back before the networking setup is done [18:11] the automatic ones nova assigns at boot [18:11] there's a hack you might notice in the python code that polls for contents of addresses in launch in one case (where you need to assign a floating ip) [18:11] mgz: yes, but even after that /servers/detail won't get me the addresses - they're always empty [18:12] they're not always empty, they're just initially empty [18:12] mgz: well, /ips works even at BUILD, while /servers/detail doesn't [18:12] I think because it reflects current reality rather than intent or something [18:12] it works at BUILD, just not straight away, only when it's actually bound [18:13] anyway, this might be a neat trick for the moment [18:14] anyway, it's not reliable to use /servers/detail for addresses in my experience [18:14] and having it this way simplifies the tests / doubles a bit [18:15] really juju needs to learn that addresses aren't reliable [18:15] :) [18:15] especially if they float [18:15] but if this hacks around that broken assumption for now, it's not the end of the world [18:17] yeah [18:21] mgz: i was listening on localhost! [18:24] time to stop [18:24] g'night all [18:25] rogpeppe: c u [18:30] dimitern: commented, fine overall, but I think much of that code should live in juju-core [18:30] mgz: thanks, I'll think about it some more tomorrow [20:14] davecheney, ping [20:38] davecheney, I need to head to meetings in a few minutes but I should be on again in a couple of hours, let me know if I can clarify anything... [22:01] davecheney: ping [22:18] rogpeppe: ack [22:18] davecheney: yo! [22:18] davecheney: G+? [22:19] sure [22:23] ringing ... [22:23] this is fucking annoying [22:23] can we use Skype like grownups ?