[02:06] axw or wallyworld: Rietveld: https://codereview.appspot.com/13717044 [02:06] wallyworld: how's baldrick? [02:06] thumper: otp [02:06] ok. stupid tail [02:06] axw: ack [02:07] stupid tail or stupid dog? [02:07] both [02:07] thumper: did you talk to william? [02:08] wallyworld: yeah, but he was packing and moving yesterday [02:08] wallyworld: landed the first part where it doesn't write to the agent conf [02:08] and I don't need to any more [02:08] ok [02:08] so now we have just one place where the logging conf is saved [02:08] and that is in state [02:08] great [02:09] so we start the machine and unit agents with --debug [02:09] always [02:09] and the logger worker updates as necessary [02:09] as soon as it starts [02:09] sounds good [02:11] thumper: i'm not 100% across it - will unilaterally removing the file writer stuff up new environments? don't we want to only remove it for old envs? [02:12] wallyworld: the new environments won't ever set it [02:13] ok [02:13] wallyworld: and unfortunately we have no way to know which version it is from [02:13] no point asking if it is there [02:13] ok [02:13] the method returns an error if it isn't there [02:13] but we just ignore it [02:45] sorry thumper was having my one-on-one with robbie ... reading back now [02:45] np, wallyworld handled it [02:45] was just a review [02:45] ah cool [02:45] I have another coming [03:10] wallyworld: re: an integration style test, no we don't have one, and I think the effort in all the wiring up is out of proportion to result [03:10] given that all the parts are tested [03:10] famous last words :-P [03:10] ok though [03:10] perhaps it is something that we want to add to the more "live tests" that we do daily with jenkins [03:10] yeah [03:11] wallyworld: well, we have tests for all the parts [03:11] so I'm pretty confident there [03:11] ok [03:20] 24 files changed, 31 insertions(+), 158 deletions(-) [03:20] that is a good metric I reckon [03:21] * thumper waits for lbox to calculate the diff that LP has already done [03:22] wallyworld or axw: https://code.launchpad.net/~thumper/juju-core/add-cleanup/+merge/185964 [03:22] lbox finally caught up [03:22] the description has the link now [03:23] just fixing some stuff, will look in a sec [03:24] kk [03:31] thumper: just one question on rietveld [03:31] shoot [03:32] no I mean, there's a question waiting for you on your MP [03:32] oh [03:32] ok [03:34] replied [03:38] thumper: thanks, LGTM [03:38] ta [03:58] axw: whenever you have a moment, i've updated https://codereview.appspot.com/13725043/ to move the httpstorage as requested [03:59] thanks wallyworld, I'll take a look [03:59] ta, no real urgency [04:00] wallyworld: I thought it was ok being in environs, since it's used by things other than the provider. doesn't really matter I suppose. [04:00] axw: in environs, i'd prefer to keep stuff there generic. if it's ec2 specific, it belongs under the ec2 package imo [04:01] i'd also prefer that stuff not use it outside the provider i guess [04:01] since it is ec2 specific [04:01] and general code should not depend on ec2 isms [04:02] there's been way too much of that which has caused grief dealing with other providers [04:02] doesn't tools lookup still use this to fall back on? [04:02] tools uses a http data source [04:02] which is not ec2 specific [04:03] i really wish the ec2 specific one would go away [04:04] ok. for some reason I thought FindTools still depended on it [04:05] wallyworld: ah, right. it's the sync code that I'm thinking of. [04:06] yeah, that will be migrated to use http://juju.canonical.com as soon as that is ready [04:06] so it's short term === thumper is now known as thumper-afk [04:42] axw: thanks for the review, i'll address the issues [04:43] i really wish Go had better debugging facilities [04:43] feels like a bad 1970's timewarp sometimes. everything is sooooo primitive [04:48] wallyworld: heh :) there is gdb support... [04:48] but not good IDe support [04:48] I believe there's even some IDE integration with gdb/go [04:48] liteide? [04:48] i use intellij [04:48] liteide may work, i will need to take another look i guess [04:49] wasn't much last time i looked from memory [04:49] wallyworld: it certainly isn't up to par with Java, but I'll take it over C++ any day for debugging [04:49] really? [04:50] then again, I'm not much of an IDE person :) [04:50] apart from java, which is first class, python is also great [04:50] yeah [04:50] so how to you refactor etc with good ide support? [04:50] to be fair, they're much older [04:50] or navigate code, or find semantic usages etc [04:50] well go is 4 or 5 years old now? [04:51] yeah, 4 [04:51] erm, the hard way :) [04:51] 4 years is plenty long enough for good tools to have emerged [04:52] not sure if you follow the mailing lists, but there was an announcement last week of a new tool called "The Go Oracle" [04:52] this will drive IDE developments [04:52] for doing callgraph analyses and whatnot [05:01] yay, about time :-P [05:05] wallyworld: also, there's basic renaming/refactoring capabilities in gofmt [05:05] godoc gofmt [05:05] if you're not aware of it.. [05:05] i was not :-) [05:06] although in 2013 "basic" isn't enough anymore :-) [05:06] yeah, it's certainly not all there [05:46] why the fuck is the juju-core repo 22mb ?!?! [05:46] who has been storing guttenberg in there [05:55] help [05:55] % bzr push bzr+ssh://bazaar.launchpad.net/~go-bot/juju-core/1.14 [05:55] bzr: ERROR: Cannot lock LockDir(chroot-80212880:///~go-bot/juju-core/1.14/.bzr/branch/lock): Transport operation not possible: readonly transport [05:55] cannot tag 1.14 [05:55] cannot tag 1.14.0 === tasdomas_afk is now known as tasdomas [06:51] axw_: is there an issue for juju debug-log screwing up ? [06:52] davechen1y: not that I'm aware of? [06:52] ok [06:52] i guess that one went through to th keeper [06:52] but it's fixed ? right [06:52] on 1.14.0 ? [06:52] oh sorry [06:52] yes there was one [06:53] and yes it was fixed a while ago [06:53] I thought you meant one outstanding [06:53] I'll have a look for it [06:54] davechen1y: here's one https://bugs.launchpad.net/juju-core/+bug/1211147 [06:54] <_mup_> Bug #1211147: Deploying service to bootstrap node causes debug-log to spew messages [06:55] I can't remember what the other issue was... [06:59] davechen1y: https://bugs.launchpad.net/juju-core/+bug/1212148 [06:59] <_mup_> Bug #1212148: rsyslog accumulator strips newlines [06:59] both merged in 1.13.2 [07:01] mornin' all [07:02] i'll be out this morning for a doctor's appointment for a couple of hours (hopefully less), leaving in an hour. [07:03] hey rogpeppe. okey dokey. [07:03] axw_: hiya === axw_ is now known as axw [08:02] dimitern, jam: i've got a doctor's appointment now. back before the standup i hope. [09:03] jam: if you are interested - https://codereview.appspot.com/13632056 is part 1, part 2 to follow. part 2 will adjust tools fetching to use retry logic when necessary. this current work always does not use retry logic for tools fetching. bootstrap is fast again :-) [09:04] well, upload tools is still slow [09:04] but tools seeking is fast === tasdomas is now known as tasdomas_afk === tasdomas_afk is now known as tasdomas [09:42] back [09:59] jam: mumble? [10:58] jam, ping [11:12] jam: help [11:12] i can't tag the 1.14 branch [11:13] davecheney: what's up sepcifically? [11:14] you should just be able to set your user to the bot and run tag against the remote branch as usual [11:14] mgz: hmm, maybe that was it [11:14] my history from the last tag is missing [11:15] davecheney, arosales: ? [11:15] mgz: bzr push bzr+ssh://bazaar.launchpad.net/~go-bot/juju-core/1.14 [11:15] what am I missing ? [11:15] jam, davecheney needs some assistance with the tagging of 1.4 [11:15] ah, that's not creating the tag, you want to make the branch? [11:15] 1.14, blocking the release needed today [11:15] davecheney: "bzr tag -d bzr+ssh://go-bot@bazaar.launchpad.net/~go-bot/juju-core/1.14" [11:15] you need ://go-bot@ [11:15] well, you need to specify the tag as well :0 [11:15] otherwise you're probably running it as dave [11:16] bzr tag -d bzr+ssh://go-bot@bazaar.launchpad.net/~go-bot/juju-core/1.14 -r -1 juju-core-1.14 [11:16] natefinch: https://wiki.canonical.com/CDO/UbuntuServer/IOM-Lab is the "garage maas" that we have set up (I believe) [11:17] * TheMue => lunch [11:17] jam: mgz thank you [11:18] jam: thanks [11:21] jam: mgz https://code.launchpad.net/~go-bot/gwacl/test [11:21] ^ is this lp:gwacl ? [11:22] davecheney: nope.though it's probably a branch of it [11:23] lp:gwacl is ~gwacl-hackers/gwacl/trunk [11:26] mgz: i'm confused the bot doesn't own the branch [11:26] the bot owns the test branch [11:28] davecheney: the bot only owns branches that it manages [11:28] meh [11:28] no time for scenary today [11:29] need to get mr page a release [11:31] mgz, jam, davecheney, once we have the source tar ball any way the ubuntu package can be built without directly (without going through saucy first)? [11:32] sorry, let me restate [11:32] mgz, jam, davecheney, once we have the source tar ball any way the ubuntu package can be built directly (without going through saucy first)? [11:32] arosales: no, but I can build from the source of the tarball to test [11:32] ...I'm still not sure I understand [11:32] might need some jiggery pokery to get the tools to work [11:32] you can pull the tarball into the packaging branch and build that with the normal debian tools [11:33] mgz: good idea [11:33] that will work [11:33] but probably not necessary [11:33] i think I can fake the tools [11:33] mgz, I think davecheney dputs the built packages once they are available in saucy [11:33] arosales: the build-release-tarball also does a smoke test compile of the source after building it anyway [11:34] I was asking if we could just build directly, and avoid the dput step [11:34] arosales: no, i _want_ to do that, but we don't currently [11:35] davecheney, ah so you currently don't wait for it to be available in saucy. [11:35] arosales: we need to wait for saucy _AND_ the ppa backports before we can announce the release [11:35] but we do not need to wait for either of those to test it [11:35] there will be ways of working around that [11:36] technically I don't see why we need to wait for saucy. That was my questions about building directly [11:37] arosales: my mistake [11:37] i was not clear in understanding your goal [11:37] * arosales not wording well either mgz didn't understand either. [11:38] arosales: what do you want, not what do you want us to do :) [11:39] once you have the source tar ball building the packages should be identical to what gets built in saucy. [11:39] arosales: correct [11:39] not should [11:39] _is_ [11:39] the tarball contains all the source code [11:39] there are no external build depdencies [11:40] Thus to help get the release in the ppa today would it be possible to build juju 1.14 outside of the saucy archive so it is available in juju/stable sooner? [11:40] arosales: no, that is not possible [11:41] we do not have a build recipe for that [11:41] i would *like* the juju source to contain the debian/ build recipe [11:41] but at the moment that is not done [11:41] ok 1/2 branches waiting on the bot to merge [11:42] the other can be submitte after the release [11:42] davecheney, ok . . . [11:42] arosales: i have not been clear [11:42] the build release tarball contains the upstream source [11:43] the magical ubuntu debian parts are missing [11:43] so while you can build juju from the srouce [11:43] you cannot replicate the packaging [11:43] that magic is not something i have access too [11:43] however, I do not think this is a major hurdle for smoke testing [11:44] smoke no, maybe for getting something out the door today. [11:44] afik anyone can build a ppa if they have the upstream source and the debian source package [11:44] arosales: yes, it is possible [11:44] no, we do not have the magic to do so [11:45] arosales: i am working as fast as possible [11:45] davecheney, I know and its appreciated. I know your up late too. I was just inquiring about the getting the release into juju/stable step. [11:45] arosales: in theory it is possible to skip that step [11:46] at the 11th hour, without ever having tried it [11:46] i don't know how to do it [11:46] nor recommend it [11:46] arosales: so, I've not found the ppa useful or a good recommendation [11:46] arosales: what do you need, and what time do you need it by ? [11:46] when is the announcement ? [11:46] mgz, juju/stable? [11:47] will a binary .zip be sufficient ? [11:47] but, it would be pretty trivial to change it to do basically what the release into saucy does, by just making it use a packaging branch straight up and merging the tarball into that [11:47] jamespage: hello [11:47] davecheney, we'll need a ubuntu package. [11:48] davecheney, time wise it was never set to an actual hour. If it is not available in the next 4 hours then we'll probably miss today as the UK folks go offline and they are the web team that will do the announce. [11:48] tarball is ~30 mins away [11:48] less [11:51] rogpeppe, ping [11:51] dimitern: pong [11:52] rogpeppe, I have a weird panic in a test when I run the full suite, but it's ok if I run each test on its own [11:52] dimitern: which suite? [11:52] so.. right now the most stable version of juju till the next tarball is ppa/devel [11:52] jam: FYI logging in to garage maas is not working. Seems likely I don't have permission. [11:52] dimitern: and what's the panic? [11:52] arg, fucking build falied [11:52] rogpeppe, http://paste.ubuntu.com/6119111/ [11:52] trying again [11:53] rogpeppe, provisioner - this is the code http://paste.ubuntu.com/6119112/ [11:53] arosales, davecheney re new stable version.. afaics we should have 1.13.3 in stable ppa [11:53] hazmat: NO [11:53] rogpeppe, it doesn't happen with the deployer, which also uses PasswordChanger the same way [11:53] no, no no, a thousand nos [11:53] davecheney, why not.. forget the undocumented even/odd stuff.. [11:54] hazmat: even if I could [11:54] the tools owuldn't work [11:54] really [11:54] no [11:54] hmm [11:54] this is not a good idea to be trying [11:54] trunk has a bunch of tools issues [11:54] hazmat: sure, we're not releasing trunk [11:54] davecheney, didn't you tell me to use 1.13.3 earlier today for a customer setup? [11:54] we are relasing 1.14.0 which is 1.13.3 + 4 small patches [11:54] ah.. ic [11:54] dimitern: does it happen consistently? [11:54] so, the best version of juju we have [11:54] rogpeppe, so far on each run [11:54] plus some azure fixes from brisbane [11:55] so it's even betterer [11:55] rogpeppe, tried updating mgo to r240, as the dependencies.tsv says - still the same error [11:55] davecheney, so what's the transition process from dev to stable? last dev gets incremental fixes.. till its marked stable? [11:55] rogpeppe, would've tried with the latest mgo tip, but that just fails on its own tests [11:56] hazmat: as it stands, we develop on trunk under an unstable dev version [11:56] then a decision is made that a particular unstable tag looks good [11:56] so we branch from there [11:56] then start a new unstable series [11:56] so [11:56] 1.13.3 -> 1.14.0 [11:56] dimitern: could you put a log print just before line 114? [11:56] -> 1.15.0 -> 1.15.1 [11:58] dimitern: the weird thing is line 35 [11:58] arosales: 15 minute delay, the bot failed to run the tests correctly [11:58] trynig again [11:58] davecheney, so basically we tag a version of trunk or dev tag as release series, and create a release series branch against that and then do bug fixes against till its stable? [11:58] if it fails, we'll go around th bot [11:58] rogpeppe, the log doesn't show at all [11:58] rogpeppe, what's weird about line 35? [11:58] hazmat: no [11:59] hazmat: docs/juju-release-process.txt [11:59] dimitern: provisioner.SetPasswords should make an API call, right? [11:59] hazmat: also https://docs.google.com/a/canonical.com/document/d/1bxekJclEzXT3b8fEmP2wHfZ2hiiYI-ojzBnr_2mVfpo/edit [11:59] rogpeppe, no [11:59] and about a million other google docs [11:59] rogpeppe, that's the server-side test [11:59] davecheney, if only they could be searched ;-) thanks [12:00] hazmat: i believe you can ask your govt. for assistance there [12:00] dimitern: oh, of course, sorry. [12:00] davecheney, nice. citizens only get guns.. info requires using it [12:00] rogpeppe, why is the session getting closed anyway? and why only on the last test [12:00] dimitern: but you're not seeing a log message that's printed just before line 114? [12:01] davecheney, thanks for the continued work on it. [12:01] rogpeppe, ah, sorry I see it yes [12:01] rogpeppe, just after that the log continues with reset dummy environment [12:02] dimitern: so you see the reset *after* the log message? [12:02] rogpeppe, yes [12:02] arosales: no problem, it's what i;m here for [12:02] arosales: we're really close [12:03] i'd like to not cowboy the packaging [12:03] it is one of the few things that works well [12:03] dimitern: could you print out the callers of dummy.Reset? [12:03] and we'll only cock it up if we rush [12:03] dimitern: (you can use "code.google.com/p/rog-go/exp/runtime/debug".Callers(0, 20)) [12:04] dimitern: e.g. log.Infof("dummy resetting; callers: %s", debug.Callers(0, 20)) [12:04] davecheney, ack we should follow the process [12:04] rogpeppe, just a sec [12:05] arosales: merged, tag updated, building tarball [12:05] getting closer :-) [12:06] davecheney, that doc seem quite right.. there is only one dev branch and tags.. so my previous description seems accurate. but the doc states.. "For stable releases they are branched from a known working devel branch" [12:06] er.. doesn't seem quite right [12:08] hazmat: for the 2 stable releases ive done [12:08] one of which I am doing right now [12:08] I branched from a devel tag [12:08] that is all I can tell you [12:09] davecheney, k [12:10] hazmat: the pirate code is more, um, a guideline [12:10] davecheney: nice [12:10] rogpeppe, http://paste.ubuntu.com/6119176/ [12:11] arrr [12:11] arosales, davecheney: I'm building the 1.14 juju client windows installer. I have a windows build of the juju client that I made off of the 1.14 branch. I believe that's all that is required. is there anything I'm missing? The windows installer code didn't actually make it into 1.14, but it's completely separate from the client code, so it shouldn't matter. [12:12] natefinch: just make sure you get the right revisions of GWACL et al [12:12] rogpeppe, it seems somehow TearDownTest gets called [12:12] the revisions are in the build-release-tools script [12:12] dependencies.txt in the root of the 1.14 branch is incorrect [12:12] in fact [12:12] i'm going to delet eit [12:12] arosales: release tarball: /home/dfc/devel/juju-core/1.14/scripts/build-release-tarball/juju-core_1.14.0.tar.gz [12:12] DONE! [12:12] dimitern: could you paste the log from running with -gocheck.vv [12:12] natefinch, I defer to davecheney [12:13] jamespage: relase incoming in ~ 5 mins [12:14] rogpeppe, http://paste.ubuntu.com/6119185/ [12:14] jamespage, do you have a rough estimate on when saucy could have 1.14 available ? [12:14] arosales, I'll pick it up tomorrow AM [12:14] need to checkin with the release team as well [12:15] dimitern: that is weird [12:15] but as its 'stable 1.13' that should not be to much of a problem [12:15] rogpeppe, it seems it calls both SetUpTest and TearDownTest before trying the actual test [12:16] dimitern: could you push the branch - there are a couple of things i want to check [12:16] davecheney, arosales, mramm: speaking of which I really need to get what we want to ship in Saucy nailed down here - https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1219879 [12:16] <_mup_> Bug #1219879: [FFe] juju-core to 1.14.0 stable release for Ubuntu 13.10 [12:17] jamespage: arosales i would like to assign that bug to mark ramm [12:17] does he have the bandwidth to deal with it ATM ? [12:19] rogpeppe, https://codereview.appspot.com/13720045 [12:19] jamespage, ack. I will follow up on that bug. I think thumper-afk and jam had some ideas on bits needed for Saucy. [12:20] arosales: will you please hold the bug for the moment [12:20] jamespage: arosales https://launchpad.net/juju-core/1.14/1.14.0 [12:20] sinzui: https://launchpad.net/juju-core/1.14/1.14.0 [12:20] ^ sorry mate, stole your thunder [12:20] davecheney, will do. I was mostly going to sync up with thumper-afk and jam on saucy and update bug comments. [12:21] jamespage: is it possible to build this release directly into the juju:ppa/stable PPA ? [12:21] for P, Q, R and S ? [12:21] dimitern: ok, i can reproduce the problem at least [12:22] davecheney, I'm just prepping for saucy now then I can backport (have a script) [12:22] davecheney, no [12:22] rogpeppe, good start :) [12:22] davecheney, thanks for getting it built [12:22] davecheney, jam controls gobot? did you ask jam to add the tags to goose and 1.14 branch? [12:22] jamespage: BOJAH [12:22] sinzui: looks lie he did gwacl [12:22] see https://docs.google.com/a/canonical.com/document/d/1DuQfYq3SB1MJigrnpcuKBkfJfMJaTstQc4bRzN5Q0Uo/edit# [12:22] i did the rest [12:22] I did gwacl because I had permission [12:23] ahhh [12:23] cool [12:23] * davecheney table flips over our stupid permission model === tasdomas is now known as tasdomas_afk [12:31] dimitern: this is very odd - somehow the call to mgo.Query.One is triggering the TearDownTest call [12:32] rogpeppe, hmmm [12:36] https://code.launchpad.net/~dave-cheney/juju-core/9001-update-release-tarball-for-1.14.0/+merge/186024 === tasdomas_afk is now known as tasdomas [12:40] arosales, davecheney: my windows fixes didn't make it into 1.14... which means when you run juju on windows, what you get is "error: cannot determine juju home, neither $JUJU_HOME nor $HOME are set" [12:41] natefinch: why didn't those get landed? [12:41] they were landed the day after 1.14 was cut [12:41] ... after 3 landing tries failed [12:42] (from conflicts and then some bot problems IIRC) [12:42] natefinch: are you sure [12:42] oh [12:42] so they only landed on trunk [12:42] and they were not backported to 1.14 series [12:42] well, fuck [12:42] correct [12:42] indeed [12:42] arosales: what do you want to do ? [12:42] i recommend not cutting a new release [12:42] 1.14.1 with some more things from trunk is fine [12:43] but comitting nates fix to 1.14 [12:43] and he can do a dodgy .zip release [12:43] right. [12:43] it'll be fine [12:43] natefinch: can you please propse the merge anyhoo [12:43] davecheney: sure [12:43] we'll figure out the policy reguardless [12:44] rogpeppe, any more progress? [12:44] can I get a F'yeah for https://code.launchpad.net/~dave-cheney/juju-core/9001-update-release-tarball-for-1.14.0/+merge/186024 [12:44] dimitern: just more weirdness [12:44] davecheney: I don't like it much, because just fixing the release script is easy... but it seems okay for now [12:44] dimitern: i'm starting to suspect a compiler bug - i'll try against go tip [12:45] davecheney: I find it very inverted to have juju-1.14 as a label on gwacl/goose, but I can understand it from a pragmatic perspective. [12:45] davecheney, yes we'll need to get nate fixes in. Sounds like a 1.14.1 is needed. [12:45] rogpeppe, wow.. these always come to me [12:45] dimitern: yeah [12:45] dimitern: i think somehow a stack frame may be being mashed [12:46] jam: it was something that happened without much thought [12:46] jam, mgz: how do I cherry pick revisions to merge into 1.14? I have 1.14 checked out... but I don't want to waste people's time figuring out the bzr syntax [12:46] davecheney, so is it correct to state that a 1.14.1 is needed? [12:46] a while back wallyworld___ and I started tagging goose with juju-x.y.z tags [12:46] and it snowballed form there [12:46] from there [12:46] natefinch: similar to your revert the other day [12:46] arosales: i'd like to avoid it [12:46] natefinch: cd $1.14 branch [12:46] dimitern: because i can put a defer func(){sleep(some time); log.Printf("returning")}() into mgo.Query.One and it sleeps and i see the print [12:46] natefinch: bzr merge -c REVINTRUNKITLANDED lp:juju-core [12:47] and cut a special .zip file for windows from the tip of 1.14 series [12:47] dimitern: but a log statement immediately after the return i never see [12:47] rogpeppe, right [12:47] rogpeppe, well, if it works with tip, we have something at least [12:47] davecheney, will there be a tools mismatch if natefinch pull from 1.14 tip? [12:48] arosales: no, they won't notice [12:48] rogpeppe, any ideas how to work around it? [12:48] dimitern: i need to characterise it first [12:48] they = juju, correct? [12:48] because nate's build will still think it is 1.14.0 [12:48] davecheney, akc [12:48] ack [12:48] to be sure, this is a BAD idea [12:48] but something we can live with [12:48] to avoid resetting the clock with the work jamespage is doing [12:48] davecheney, ok [12:49] natefinch: with the caveat that you have to then backout your logrotaet changes again [12:49] davecheney, don't like the sound of that [12:49] I just uploaded to saucy [12:49] natefinch: "bzr merge -c 1788 lp:juju-core" should be about right [12:49] jam: yep, had already taken that into account [12:49] jamespage: thank you [12:49] arosales: this is more importnat than a small bodge [12:49] dimitern: thank goodness the bug doesn't seem to be too sensitive to code changes [12:50] jamespage: i can cut 1.14.1 as soon as natefinch 's change is ready [12:50] can you upload that to sauce and backport today ? [12:50] jam: should I be merging directly into the 1.14 branch or a branch off of that? [12:50] rogpeppe, good [12:51] natefinch: checkout 1.14 branch [12:51] merge the change there [12:51] davecheney: cool [12:51] bzr push lp:~natefinch/juju-core/1000-fix-windows-build [12:51] dimitern: just building go tip [12:51] davecheney, later tonight yes - you caught me on a baby break :-) [12:51] then go to lp [12:51] jamespage: ok, we'll do it properly [12:52] sinzui: do you want to drive 1.14.1 ? [12:53] rogpeppe, did you manage to isolate it to a minimal code snippet? [12:53] davecheney, I suppose so. [12:53] sinzui: "no" is a perfectly acceptable answer [12:53] dimitern: ha ha [12:53] rogpeppe, what? [12:53] natefinch: is there a bug for the JUJU_HOME thing ? [12:53] i need to assign it to a 1.14.1 milestone [12:54] davecheney: I think so, lemme look [12:54] dimitern: no chance! [12:54] davecheney, Is dependencies.tsv really wrong in the 1.14 branch? [12:54] dimitern: (not yet anyway) [12:54] davecheney: at least indirectly [12:54] sinzui: yes [12:54] and we dont' use it [12:54] * rogpeppe now *actually* builds go tip [12:54] and we can always add back a correct one in the future [12:54] but an incorrect file is a hand grenade with the pin loose [12:56] sinzui: in the future the goal is for the release-tarball script (and everything else) to consume dependencies.tsv [12:56] but it doesn't today [12:56] so on this stable branch [12:56] i have voted to remove it [12:56] as a 2nd pair of underpands [12:56] pants [12:57] natefinch: https://launchpad.net/juju-core/+milestone/1.14.1 [12:57] please adjust the JUJU_HOME bug to include this series [12:57] or give me the original bug number and I'll do it [12:57] davecheney, backports for p q and r uploaded to stable ppa as well [12:57] jamespage: wonderful [12:57] sorry, you're going tohave to do it all again in 40 minutes [12:57] dimitern: ok, the issue's still there in go tip [12:58] davecheney, understood. I see you removed dependencies.tsv from your branch to avoid confusion [12:58] davecheney: should I not be using lbox propose? It keeps trying to propose against trunk [12:59] natefinch: nah, that'll fuck up [12:59] just propose the branch by hand [12:59] sinzui: correct, this is just to avoid confusion [12:59] we can always add a correct one tomorrow [12:59] rogpeppe, bugger.. [13:00] ... for small values of tomorrow [13:00] davecheney: https://code.launchpad.net/~natefinch/juju-core/014-fix-windows-build/+merge/186028 [13:00] dimitern: now trying to isolate [13:00] davecheney, I only hesitate because I have just gotten up and finishing my first cup of coffee. I try not to commit when I don't have my wits about me [13:00] rogpeppe, great [13:00] sinzui: https://code.launchpad.net/~dave-cheney/juju-core/9002-set-stable-release-version-to-1.14.1/+merge/186029 [13:00] sets the version to 1.14.1 [13:00] sinzui: like i said, "no" is a perfectly acceptable answer [13:01] there will be many more releases [13:01] some less shotgun [13:01] natefinch: bloody hell, this isn't a small change [13:02] davecheney: 99% is mechanical, changing $HOME to a small function call [13:02] ok, jam rogpeppe dimitern etc, https://code.launchpad.net/~natefinch/juju-core/014-fix-windows-build/+merge/186028 [13:02] davecheney: the comment is unfortunate, since it was written when the change was just a single line :/ [13:02] can we get a _1 [13:02] +1 [13:03] natefinch: found that bug ? [13:03] sinzui: i'm going to reuse the 1.14.1 release notes [13:03] understood [13:03] davecheney, LGTM [13:03] tradition dictates that i mention that 1.14.0 was replaced with 1.14.1 [13:03] nobody seems to care [13:04] natefinch: please can you use lbox propose (you can use lbox propose -for lp:juju-core/1.14) so the changes can be seen in context [13:04] sinzui: [13:04] 1.11.4 is a small bugfix release for 1.11.3, please refer to the [13:04] release notes for 1.11.3 for full details. [13:04] rogpeppe: I tried that and it still tried to do it into trunk [13:04] ^ for example, this was one previous release note [13:04] natefinch: it worked ok for me recently [13:04] davecheney: I think I was hallucinating... no bug for that, but we do have: https://bugs.launchpad.net/juju-core/+bug/1195757 [13:04] <_mup_> Bug #1195757: Package Juju for Windows [13:05] natefinch: that's a paddling [13:05] natefinch: it might work now if you just do lbox propose [13:05] I have to AFK for 10 mins to meet my partner at the train station (it is late, and she's coming home on her own) [13:05] i will be back v shortly [13:06] rogpeppe: trying [13:10] rogpeppe: no luck [13:10] rogpeppe: it completes, but nothing is in rietveld [13:11] natefinch: did it print a codereview link? [13:11] rogpeppe: no :/ [13:11] natefinch: try "lbox propose -cr -for lp:juju-core/1.14" [13:12] rogpeppe: right, oops [13:12] * natefinch forgot the -cr [13:12] natefinch: the -cr *should* be automatic, but... [13:12] rogpeppe: still nothing printed out [13:13] rogpeppe: and nothing in rietveld.... [13:13] natefinch: odd. what does it print if you use the -verbose (or -debug) flag? [13:15] rogpeppe: it's really spammy, but looks like it's bailing because it was already proposed.... http://pastebin.ubuntu.com/6119401/ [13:15] rogpeppe: I gotta run for a few to help my wife with the kids, sorry [13:16] rogpeppe: already put her off for a half hour [13:16] natefinch: ok, np - i'll pull it locally and have a look in a bit [13:16] rogpeppe: thanks [13:16] natefinch: currently trying to track down dimiter's bug [13:18] bak [13:19] dimitern: there's definitely *something* wrong going on. i've got it down to this so far: http://paste.ubuntu.com/6119414/ [13:19] rogpeppe: natefinch i think it would be better not to overcomplicate things nad just propose this via lp [13:19] dimitern: if i change line 42 to use s.machine not s.machines[0], the test passes [13:19] natefinch: rogpeppe please be mindful of jamespage 's time [13:19] the more screwing around we do [13:19] the more time he has to wait [13:19] davecheney: ok, i'm on it [13:20] rogpeppe: thanks [13:20] for fsck's sake...my computer just rebooted and mir blocked me from logging back in [13:22] you can take raring from my cold dead hands [13:22] i'm not falling for the unstabl eupgrade trap again [13:22] davecheney, I'm out for the next ~8 hours or so [13:22] rogpeppe, interesting [13:22] if you need more time I can do first thing UK time tomorrow [13:22] rogpeppe, so it might be a stack overflow kind of thing perhaps? with the slice present [13:22] jamespage: understood [13:23] natefinch: in config.expandTilde, shouldn't path.Join in expandTilde be filepath.Join ? [13:23] dimitern: no [13:23] saucy has been quite a naught series. My keyboard is often unmapped and the audio drivers work on the roll of the dice [13:23] dimitern: it really seems like a compiler bug [13:23] dimitern: ha ha! [13:23] dimitern: i thought you meant something suitable for a stackoverflow.com question! [13:23] dimitern: i don't *think* it looks like stack overflow [13:24] dimitern: after i've done this review, i'll have a look at the assembly [13:24] davecheney, so jamespage uploaded 1.14.0 to the ppa but the client still needed a 1.14.1 release, which is in progress, correct? [13:25] arosales: yes [13:25] * arosales thought natefinch was just going to use 1.14 tip . . . [13:25] arosales: we changed our minds about 2 pages ago [13:25] ok [13:25] jamespage prefered that we did it correctly and said there was no blocker in getting 1.14.1 done 1st thing UK time [13:26] aside from to companies holding their PR announces [13:26] s/to/two/ [13:26] arosales: i need you to give me clear directoin [13:26] I guess I let folks know we'll shoot for tomorrow [13:26] rogpeppe, ok, thanks for the help [13:27] i can sto the 1.14.1 process and as soon as nates' windows fiz lands we can build a dodgy .zip version from trunk [13:27] /s/trunk/tip [13:27] davecheney, jamespage: do you think we'll have a 1.14.1 ready UK morning? [13:27] davecheney, nah is already in progress [13:30] arosales: ok, understood, the plan is to continue to release a 1.14.1 as soon as possible [13:30] davecheney, to confirm does 1.14.1 have natefinch fixes? Or is that still under review by rogpeppe ? [13:30] still in review [13:31] when it lands 1.14.1 will contain nates fixes [13:31] davecheney, who is adding the tags to goose and 1.14.x branch? You? [13:31] arosales: still in review, sorry - it's quite a substantial change [13:31] bot is batting 2 for 0 [13:31] sinzui: I will do it [13:32] * arosales not sure how davecheney is cutting a 1.14.1 release with code still under review [13:33] * arosales goes to break the news that we are not releasing 1.14 today . . . [13:35] arosales: stop [13:35] we _CAN_ release a 1.14.x today [13:35] we can bodge up a windows client release without going through the release process [13:36] arosales: i can cut the release as soon as the banch land [13:36] the commands are queued up in my buffer [13:38] arosales: i am waiting on your word to try for a proper 1.14.1 or keep 1.14.0 that we have now and bodge in the windows zip fix [13:39] if we go for the latter option [13:39] we can fix it in short order with a proper 1.14.1 release on the next day [13:40] it would be nice to have something that works today [13:40] and then we can fix the bits behind the scenes by tomorrow [13:41] davecheney, what if we go with the former option [13:41] arosales: both options need natefinch rogpeppe to land that branch [13:41] going with a proper 1.14.1 probably adds another 8 hours delay [13:41] at minimum [13:41] davecheney: i've just finished the review [13:41] rogpeppe: sweet [13:41] davecheney: it LGTM with one minor issue [13:41] mark that sukka as approved [13:41] oh [13:42] davecheney: one place is using path.Join instead of filepath.Join [13:42] davecheney: as nate's not around, i think i might just fix it, and approve it myself [13:43] davecheney: could you confirm my comment on https://code.launchpad.net/~natefinch/juju-core/014-fix-windows-build/+merge/186028 please? [13:44] rogpeppe: sorry... wrangling two kids right now... yes, you're probably right [13:44] natefinch: are you able to fix it and approve soon, or i could do it if you can't [13:44] rogpeppe: probably better for you [13:46] rogpeppe: sorry and thanks [13:46] davecheney, lets proceed with the following: [13:46] for today we use the 1.14.0 with a hacked 1.14.0 msft client installer [13:47] tomorrow we update the links to use the 1.14.1 msft installer [13:47] davecheney, ack? [13:47] arosales: ack [13:47] arosales: hold [13:48] i am locating the 1.14.0 debs to produce tools [13:48] natefinch, I'll need a 1.14.0 tools that uses 1.14.0 tip download link [13:50] arosales: um, can you say again please [13:50] davecheney: approved [13:50] because we havne't bumped the version string in 1.14 trunk [13:50] any release produced from there thinks it is 1.14.0 [13:51] rogpeppe: thanks [13:51] davecheney, I was saying for today we can use the 1.14.0 release jamespage already uploaded with natefinch msft client that pulls from tip. [13:52] after natefinch has the 1.14.0 build from tip we can proceede with a 1.14.1 proper release and I can adjust download links tomorrow. [13:52] arosales: yes, that is is correct [13:52] arosales: yes, that is also correct [13:53] ok so now I just need natefinch to kick a 1.14.0 msft client build that pulls from tip. [13:54] ok, please hold [13:54] https://code.launchpad.net/~dave-cheney/juju-core/9002-set-stable-release-version-to-1.14.1/+merge/186029 [13:54] waiting on this merge to fail [13:55] if it _succeeds_ i'll have to back it out [14:00] arosales, by adjust the download links, I think you mean remove the 1.4.0 tarballs from Lp after the 1.4.1 files re uploaded. Lp will show the new links because they are the latest uploads and since they are the only uploads on series 1.14, that is the only links available for that series [14:01] sinzui, no I mean downloads links of the msft installer from juju.u.c/install [14:01] arosales: sinzui which we haven't yet procued [14:01] so we can bodge that [14:02] sinzui: arosales ignore [14:02] what i just said was wrong [14:02] ignore it [14:02] i will _not_ be changing the tarball on the milestone [14:03] as jamespage has alreday submitted it [14:03] ah! [14:03] nate, do this to make yorself a new tarball [14:03] cd scripts/build-release-tarball/ [14:04] change line 41 to read [14:04] bzr-checkout lp:juju-core/1.14 $TAG launchpad.net/juju-core [14:04] bzr-checkout lp:juju-core/1.14 -1 launchpad.net/juju-core [14:04] then run [14:04] bash build-release-tarball.bash juju-1.14.0 [14:04] this will produce a tarball that is identical to what we uploaded to the milestone, but follows 1.14 trunk [14:04] then you should be able to feed that to whatever prodeuces your windows .zip build [14:08] natefinch, note I'll need a msft download link to give the web team asap. [14:09] natefinch: ping me if my instructions are unclear [14:10] dimitern: i still don't know the underlying cause, but i've discovered a quick workaround for your problem (which also fixes a bug in your code) [14:10] dimitern: set s.machines to nil at the start of provisionerSuite.SetUpTest [14:11] rogpeppe, ah, ok will do, tyvm [14:12] rogpeppe, if you manage to isolate and file a golang bug report, please give me a link to put it in a comment there [14:13] dimitern: that's something you want to do anyway - otherwise you add 3 new machines in each test, ending up with 9 in total by the start of the last test [14:13] arosales, rogpeppe, davecheney: back, sorry for the delay [14:13] dimitern: tbh, you could probably just rename to SetUpSuite [14:14] rogpeppe, no, I need a SetUpTest with the machines created before each case [14:15] dimitern: ah, ok [14:15] dimitern: in which case you want to reset the slice [14:15] dimitern: aaaargh! [14:15] dimitern: i see the problem! [14:16] dimitern: it's not a compiler bug at all [14:16] dimitern: the above issue *is* your problem [14:16] dimitern: and that's it [14:16] natefinch, no worries I understand daddy duty [14:16] dimitern: you're using Machines with stale state [14:16] natefinch, I was just needing a msft client installer that uses 1.14.0 tip [14:17] dimitern: hmm, though... [14:17] arosales: no problem, working on it [14:18] dimitern: i still don't see why TearDownTest is being called [14:18] natefinch, thanks, ping me when you have it. [14:18] dimitern: ah, yes, i understand that too now [14:18] dimitern: it all becomes crystal clear, phew [14:19] arosales: i have located the ppa's for 1.14.0 [14:19] proceeding to make tools [14:20] lets hope this release bloody works [14:20] davecheney, lol [14:20] arosales: http://ubuntuone.com/7aKY92pbRTXGe6IiSVzcMl [14:20] * rogpeppe crosses his fingers for the release [14:20] robbiew, oh really? [14:20] rogpeppe: ^ [14:20] lol [14:20] lol [14:20] oops :) sorry [14:21] I did that yesterday [14:21] dimitern: yeah [14:21] it's funny every time it happens... :) [14:21] dang auto complete [14:21] quassel had better tab completion than xchat it seems [14:21] until it stopped working at all [14:21] dimitern: it would have helped the diagnosis quite a bit if mongo didn't panic in Session.cluster, but returned an error instead [14:22] rogpeppe, so you replaced the panic and caught it? [14:22] dimitern: no [14:22] dimitern: the panic was just a symptom of the bug in your code [14:23] rogpeppe, what's the bug? [14:23] dimitern: the bug is the one i pointed out above - you don't reset the slice in SetUpTest [14:24] dimitern: which means that the first machines that you call Refresh on are from a previous test, and so have a stale mongo connection [14:24] rogpeppe, oh god, yes! [14:24] dimitern: innit [14:25] rogpeppe, so it's no compiler bug then [14:25] dimitern: no indeed. [14:25] robbiew, whew :) [14:25] robbiew, sorry [14:25] dimitern: and my ba [14:26] rogpeppe, I really ought to start typing 2 chars at least before pressing tab [14:26] 3 actually [14:26] dimitern: and my length of time to find the problem was exacerbated by the fact that i was seeing Query.One return - which it was, but only because a panic was happening [14:26] dimitern: your IRC client should be better behaved [14:27] dimitern: and choose the last-addressed username by default [14:27] rogpeppe, absolutely - quassel does that [14:27] dimitern: (mine seems to do that) [14:27] dimitern: Konversation works pretty well for me [14:28] rogpeppe, ah, there it is - just a option I need to tweak, and now it completes last spoken to nick [14:30] davecheney: is there a script or something that sets the release number in the code? I need to get the windows installer script hooked into something like that, so we don't have to hand-update the version in the script [14:30] natefinch: there is a little logic in the package recipe that extracts the detail from version/version.go [14:30] * davecheney finds link [14:31] okay, davecheney , natefinch I don't understand how we are avoiding a tools mismatch...oh, ^ are we tinkering with the version? [14:32] natefinch: http://bazaar.launchpad.net/~dave-cheney/juju-core/package/view/head:/debian/rules#L10 [14:32] sinzui: no, no tinkering [14:32] the version/version.go file continues to report 1.14.0 [14:32] sinzui: tinkering would be required for it to know it's a different version [14:32] it sounds like nates biuld script doesn't extract the version automagically like the packageing does [14:34] davecheney: right now my build script is cd cmd/juju && go-windows-386 build [14:34] natefinch: that'l work fine [14:35] seeya mup [14:35] Are we incrementing to 1.14.1 *after* natefinch cuts the window's installer. Thus the windows release is different from the *nix release? [14:35] davecheney: good :) ...and then open up a windows VM and build the installer from the stuff under scripts/win-installer ...my intent is to try to get it done via wine and an actual script at some point [14:36] sgtm, lets boil one ocean at at eim [14:36] time [14:36] davecheney: agreed [14:36] sinzui: http://paste.ubuntu.com/6119698/ [14:37] ^ how to bodge the tools uploader for saucy universe packages [14:38] ! thank you davecheney [14:38] im sorry for the hack [14:39] folling the various naming gyrations is complicated [14:40] following [14:40] arosales: et al, tools coming online very shortly [14:40] saucy is there [14:40] raring is coming [14:40] if anyone is running those series [14:40] and wants to test [14:40] now is the time [14:41] sudo apt-add-repository -r ppa:juju/devel [14:41] sudo apt-add-repository ppa:juju/stable [14:41] sudo apt-get update && sudo apt-get install juju-core [14:41] make sure $(which juju) points to /usr/bin/juju [14:42] davecheney, for aws only correct? [14:42] arosales: at this stage yes [14:42] davecheney, ok [14:43] we can sync-tools in a sec [14:43] but lets make sure nothing is cocked up [14:43] davecheney, ack [14:44] jam, mgz: are you able to sync-tools to hp cloud and canonistack ? [14:45] davecheney: I should have the instructions and creds in my inbox somewhere [14:46] mgz: ta [14:47] mgz hold for a few mins [14:47] still getting Q and P tools [14:47] sure. === benji__ is now known as benji [14:49] natefinch: i'm getting the wind up here [14:50] how is your confidence on makig the 1.14.0 windows release ? [14:50] davecheney: 100%. I already gave the installer to arosales (at least, I gave him a link) [14:50] mgz: sync tools [14:50] arosales: 1.13.3 to 1.14.0 upgrade worked fine [14:51] natefinch: cool [14:51] i need to sign off very shortly [14:51] otherwise there won't be much of me left to sign on tomorrow [14:51] davecheney: thanks for all the hard work so late at night [14:52] natefinch: not a problem, it is what i am here for [14:52] mgz will confirm this is nothing like the hilarity of the 1.10.0 release [14:52] that was indeed fun. [14:53] http://paste.ubuntu.com/6119756/ [14:53] in case I get hit by a bus, here's the build process for the windows installer: https://docs.google.com/document/d/1WMm6lcUTDA4wZnmA8YzfSXyLnQRz6Fqrr5g2lDtZZes [14:53] OH SHIT [14:53] this is not good [14:53] well [14:53] it's not terrible [14:53] but fucking auto sync tools is extending the bootstrap time [14:54] mgz: is this because the 'waves hands' magic data in simple streams needs to be updated ? [14:55] hmm, only appears to affect ap-southeast-1 [14:55] too bad [14:55] we'll live with it [14:55] bootstrap works fine on ap-southeast-2 [14:55] davecheney: maybe, I'm not up to date on what error messages are what with simplestreams [14:56] yup, just a problem with ap-southeast-1 [14:56] we'll survive [14:56] ap-southeast-2 and us-west-2 work fine [14:56] this is another hilarous eventually consisent problem on aws [14:56] :( [14:56] arosales: i think, to the extent that I can test at 1 am [14:56] we're good [14:57] davecheney, thank you sir for the late night hard push [14:57] mgz, let me know when aws, hp cloud tools are sync'ed [14:57] if anyone wants to test in hp cloud [14:57] I can work on Azure [14:58] it will work right now without tools synced [14:58] it will just take longer [14:58] to bootstrap [14:58] -v, always -v [14:58] arosales: I'm assuming aws is done, because I'm using that as a source [15:02] natefinch: don't forget to port your patches for the windows fixes, path => filepath back to trunk [15:02] davecheney: thanks [15:07] hm, sync-tools needs a -v [15:07] having it just sitting there quietly makes me nervous [15:07] okay, canonistack done [15:08] arosales: http://ec2-54-202-26-90.us-west-2.compute.amazonaws.com/?p=4 [15:08] done [15:08] and cute, 1.14 includes the armhf binaries, if canonistack grows an arm region :) [15:08] mgz: but only for saucy [15:08] atm [15:08] all: i have to sign off [15:08] be online in (insert short amount of time) [15:08] thank you davecheney [15:08] we'll do a proper 1.14.1 tomorrow [15:08] hp... I need to check creds with arosales I think [15:08] davecheney, woot [15:08] davecheney: thanks, have a good night [15:08] night ya'll [15:09] davecheney, have a good night [15:09] thanks for your work [15:09] i'll sync with sinzui on 1.14.1 [15:09] mgz, do you need a password reset for hp? [15:14] arosales: I'm going to check now [15:14] can't remember where we left it last time [15:14] ok, let me know. [15:15] I think you said you were able to login, but we had to update your info. [15:17] arosales: what user name do you have for me on that account? [15:17] arosales, Lp will not promote the 1.14x releases because they are on an older series: https://launchpad.net/juju-core lists the latest release from trunk [15:18] the issue is my own account uses my canonical email address, so not sure how I log into the tools one instead [15:18] https://launchpad.net/juju-core/+download lists the release after it lists all the releases from trunk [15:31] ok, wrong channel [15:31] dimitern: lovely, looking [15:31] rogpeppe, https://codereview.appspot.com/13720045/ I have a review for you :) [15:31] rogpeppe, thanks [15:32] dimitern, we like having you over there though :-) [15:33] gary_poster, I'm there all the time :) [15:33] cool :-) [15:55] dimitern: ping [15:56] rogpeppe, here [15:57] dimitern: just wondering why the provisioner auth function allows access to any machine [15:57] rogpeppe, how else would you do it? [15:57] dimitern: doesn't that depend on whether the machine is an environmanager or not? [15:57] rogpeppe, hmm it should yeah [15:58] dimitern: as it is, any machine can remove any other machine... [15:58] rogpeppe, how about the lxc provisioner? [15:59] rogpeppe, it has to be able to change any machine, which has parentId == currently auth'ed machine's id [15:59] dimitern: that sounds reasonable [15:59] rogpeppe, ok, so I'll make the getAuthFunc smarter [16:00] dimitern: sgtm [16:00] rogpeppe, it has to be either an environ manager, hence - access to all machines, or a machine agent, and access only that machine's children [16:01] dimitern: seems reasonable. although... [16:01] dimitern: should the environ manager be able to remove machines several levels deep? [16:01] dimitern: i guess it probably should, as i suppose we should be able to force-remove a machine [16:02] rogpeppe, yeah, leave a fallback option [16:02] dimitern: i wonder if that should be the case for machine agents too [16:02] rogpeppe, what? [16:02] dimitern: so you're allowed access to all machines below the current machine [16:03] rogpeppe, I don't think this is easier [16:03] dimitern: i wasn't suggesting it because it was easier [16:03] rogpeppe, it's easy enough to check the parent of a machine, but all parents up? [16:03] dimitern: but because i think it might be more correct [16:04] rogpeppe, don't think so [16:04] dimitern: why should the global provisioner be able to force-remove a machine, but a machine-local provisioner not be able to force-remove a container? [16:04] rogpeppe, we shouldn't jump levels of agents like that [16:04] rogpeppe, it will be, but not more than 1 level deep [16:05] rogpeppe, same goes for the environ provisioner [16:05] dimitern: that means if you've got a totally broken instance/container, you won't ever be able to remove it, no? [16:05] rogpeppe, can you do that now? [16:06] dimitern: i don't know, but it's a definite issue [16:06] rogpeppe, if it is, it's not an agent api problem [16:06] dimitern: things do break, and we need to be able to clean up when they do [16:06] rogpeppe, it's more like a management cli/api thing [16:06] dimitern: hmm, you may well be right [16:07] rogpeppe, yeah, let's not overcomplicate things just now [16:07] dimitern: so the client does the remova [16:07] l [16:07] dimitern: sgtm [16:07] rogpeppe, well, the cli does that now - and it's the only way [16:07] dimitern: so i guess the environ manager *doesn't* get access to all machines [16:08] rogpeppe, only the ones without a parent? [16:08] dimitern: yeah [16:08] rogpeppe, ok, looks better to me this way, nicely symmetrical [16:09] dimitern: yeah [16:09] * dimitern has to step out for 10m [16:26] * dimitern is back [16:30] rogpeppe, so, about that review? [16:30] dimitern: still on it [16:31] rogpeppe, ah, ok, sorry [16:37] dimitern: reviewed [16:37] rogpeppe, cheers [17:49] natefinch, juju-tools uploaded to Azure, when you have a spare moment could you test msft client 1.14.0 with 1.14.0-juju-core in Azure? [17:49] * rogpeppe finishes for the day [17:50] mgz, did you get the hp tools uploaded? [17:50] g'night all [17:50] arosales: will do [17:50] rogpeppe, night [17:50] natefinch, thanks. [17:51] arosales: I presume normally, we'd do that *before* the announcement, right? :) [17:53] yes, normally it would be part of the release process [17:54] mgz, for aws I see all the tools at /tools [17:54] http://juju-dist.s3.amazonaws.com/ [17:55] mgz, however for hpcloud I see the 1.14.0 juju-tools @ tools/releases [17:55] for azure I followed the aws convention [18:07] arosales: bootstrapped, waiting for machine 0 to get out of pending state [18:15] *phew* all tests work [18:29] so, time to leave. good night all. [18:35] arosales: hmm didn't notice this before, but juju ssh doesn't work, because it expects to run the ssh command (which doesn't exist on windows) I had always assumed we just used an ssh implementation in Go, but evidently not. [19:03] natefinch, hmm [19:04] natefinch, btw did your bootstrap come back with the default juju-tools bucket? [19:04] on the precise release stream? [19:04] natefinch, not sure how to handle ssh on windows but be good to log a bug for tracking/triage purposes [19:07] arosales: how can I tell what bucket it uses? [19:07] arosales: juju status gives me [19:07] environment: azure [19:07] machines: [19:07] "0": [19:07] agent-state: pending [19:07] dns-name: juju-azure-x4upm0herf.cloudapp.net [19:07] instance-id: juju-azure-x4upm0herf [19:07] instance-state: Created [19:07] series: precise [19:07] services: {} [19:07] hmm, not sure post bootstarp [19:07] arosales: it hasn't ever gotten off pending [19:07] during bootstrap you can see which tools are choosen during a --debug [19:08] * arosales will fire up an instance from ubuntu [19:08] ahh ok. I'll rebootstrap after I try deploynig some stuff [19:10] natefinch, without ssh on windows your nice page, https://juju.ubuntu.com/docs/getting-started-keygen-win.html, is invalid, correct? [19:10] natefinch, well not invalid just doensn't function atm [19:11] arosales: no no. That's still valid and needed to connect to the clouds... it's just that trying to do "juju ssh 0" won't work [19:12] natefinch, I bootstrap with the -v which tells you what was used. [19:12] * sinzui looks for hidden log [19:12] yeah, seems like everyone uses -debug or -v ... seems like we should have some better default than no output [19:13] natefinch, As a developer withing with unstable tools, it becomes common. Canonistack has good months and bad weeks. I like to know when it is a bad week [19:13] natefinch, ah you are saying for msft clients connecting to clouds other than Azure [19:14] arosales: yep, exactly [19:15] gotcha, just 'juju ssh 0' won't work [19:27] arosales: I think there's a major bug with the windows client [19:30] arosales: major is probably an understatement. Critical? Catastrophic? Will not function. [19:31] :-( [19:31] I don't think the juju tools got up thier correctly [19:31] I am re-uploading [19:31] not sure if that is what you are hitting. [19:32] arosales: unlikely. This looks like the client is telling the server to create directories with windows-style backslash path separators [19:32] natefinch, ok, I'll let you debug then [19:33] arosales: I'll switch to EC2 to double check that it's not azure [19:33] natefinch, ack [19:38] well at least you hit that issue too [19:39] * arosales types into wrong window [19:46] natefinch, you should be able to ssh into and have a look at the cloud init log [19:49] natefinch, you can't use juju ssh .. but you should be able to ssh into the dns name with an ssh client.. juju ssh uses the bootstrap node [19:50] hazmat: yeah, ssh is working, and ec2 shows the same symptoms when bootstrapped from the windows client [19:50] sigh [19:50] sinzui, natefinch, incidentally on the topic of using -v --debug all the time.. bug 1226786 [19:50] <_mup_> Bug #1226786: mechanism to set default --params to commands [19:50] natefinch, can you pastebin /var/log/cloud-init-output.log [19:51] haz [19:51] hazmat: yep [19:55] hazmat: https://pastebin.canonical.com/97613/ [19:57] hazmat: an example of the problem is mkdir -p varlibjujudb/journal [19:57] jam, incidentally all the places your doing for ssl cert support are all the same places we'll need to hit up again for proxy afaics [20:00] natefinch, that does look suspicious, client path separator perhaps.. is juju running on the machine?.. the end state there looks sane (mongo running, juju state initialized) [20:02] hazmat: yeah, the path separator was my first thought. I also see \varlogjujumachine-0.log and \var\lib\juju\server.pem as files in the root directory (note, those are the filenames... not a path) [20:04] hazmat: mongo is running, but jujud is not [20:04] natefinch, we've got tons of filepath.Join which is going to be problematic [20:04] hazmat: yeah.... ironically, by using the "right" function... we're shooting ourselves in the foot [20:04] it would be nice to see the raw cloudinit, i'm curious how yaml is treating the \ separation [20:04] indeed [20:05] hazmat: I'm kinda surprised that the client is the one that defines the paths that get set up on the server [20:05] natefinch, its only for bootstrap's cloudinit [20:05] natefinch, although possible we'll see other issues with charm deploys and bundle assembly [20:05] hazmat: I guess that makes sense [20:05] since those are also client assembled [20:06] natefinch, so the quick fix for this release, is to make a utils.PathJoin which does the unix thing, and replace extant uses of filepath.Join [20:06] although we really need to check for other uses of filepath i guess since there are others uses of os.PathSep [20:06] hazmat: yeah [20:07] natefinch, and then revisit as we get further into windows support [20:09] hazmat: k. I'll start working on that [20:09] hazmat: if it were me, I'd pull the current installer off the website... it's just cause heartache and pain [20:09] natefinch, given we have a press release out.. [20:10] hazmat: yes, but the big announcement was azure, not windows client. I was looking for the windows client announcement, and almost didn't see it [20:10] i'd rather yank it to be honest, and replace with a note for windows then have peopl dl something totally broken [20:10] arosales, ^ [20:11] yeah.. its making its way through the news wires/pundit sphere now.. [20:11] natefinch, i don't actually see the download though [20:12] hazmat: https://juju.ubuntu.com/install/ [20:12] oh. there it is [20:12] jcastro, do you know how to update that till we get this fixed [20:13] the windows download link.. its off of juju.ubuntu.com which we could fix there.. but really we also want to yank the binary from http://assets.ubuntu.com/sites/ubuntu/latest/u/files/juju/juju-setup-1.14.0.exe [20:14] * hazmat escalates via email [20:14] mramm, ^ [20:14] hazmat: thanks, I was going to do the same [20:14] hazmat: also: Searching 784 files for "filepath.Join" 307 matches across 92 files [20:15] arosales: ^^^^ [20:16] * arosales reading backscroll [20:16] arosales: short version: windows client is unusable [20:16] arosales, posted to canonical-juju [20:17] hazmat, mramm, jcastro and I no longer have access to juju.u.c [20:18] I can file an RT and hopefully the IS team have access, but other than than we'll need to await the web team. [20:18] * arosales sure the IS does. [20:18] I would file an RT, IS does have access! [20:18] ;) [20:18] s/IS/IS team/ [20:19] natefinch, is this a 1.14 specific issue you didn't see in your earlier testing? [20:20] mramm, WTF to the rescue ;-) [20:20] arosales: windows-specific ... the version doesn't really matter. Yes, I didn't see it in earlier testing. My testing was a lot lighter than it should have been. I thought it had already been tested, so I just tested the fixes I made. === tasdomas is now known as tasdomas_afk [20:25] natefinch, really we only need to the filepath.Join fix on cloudinit parts [20:26] natefinch, at least thats worth testing to see if that works by itself for bootstrap and deploy [20:26] RT filed [20:26] arosales, cool [20:26] hazmat: yep, definitely the place to start. *at least* those parts need to be fixed [20:27] technically mark baker said in his blog availability tomorrow so natefinch you may have some time to cut a new release if you find the bug. [20:27] * hazmat returns to prepping training [20:27] arosales: the problem is, it could be 100 spots in the code. [20:29] natefinch, understood, just letting you know === tasdomas_afk is now known as tasdomas === tasdomas is now known as tasdomas_afk [21:04] natefinch: could line termination be an issue too? Just wondering, since it's another difference between windows and *nux [21:06] andreas__: could be. The immediate problems I saw were definitely path separators, though. [21:07] yeah, sure, that was very visible [21:07] in a screenshot someone sent to the list [21:07] just wondering if the content is ok [21:07] me :) [21:07] ah :) [21:08] do you still have that instance up? Might want to run cat -vet on one of those files [21:08] should end in $, not ^M$ [21:08] iirc [21:08] it's gone now... but in general, Go is actually really good about always just using \n and not \r\n, even on windows [21:13] andreas__: fixed a couple spots in cloud init where we definitely want unix line endings (or rather, line endings that match the server OS, not the client OS).... seeing what else breaks now === thumper-afk is now known as thumper [21:23] thumper: good morning [21:23] hi natefinch [21:27] * thumper trawls through the emails [21:36] natefinch, arosales do we want to file a bug about the win client path non-sense and target it to the 1.14.1 milestone? [21:37] sinzui, yes as we'll need that but for the milestone and release [21:38] sinzui, I don't have the error but I can file a bug and assign to natefinch [21:39] 1.14 on Azure seems to be working on a sniff test [21:39] http://juju-azure-8jdith6sql.cloudapp.net/ [21:39] sinzui, were you able to do any preliminary testing on HP? [21:39] natefinch: how goes the windows path fun? [21:40] arosales, natefinch I think the bug just needs to state that the windows client needs to make unix paths when working with the bootstrap node. [21:40] arosales, I have not [21:43] thumper: I found at least one spot that needed to be fixed.. but still seem to be having problems [21:44] natefinch: need any help or someone to talk to? [21:44] bug 1226840 opened [21:44] <_mup_> Bug #1226840: Windows client needs to make unix paths when working with the bootstrap node [21:45] thumper: yeah... I don't know the cloud init stuff well. Not to mention, my in-laws came over for the evening and it's about dinner time [21:45] natefinch, arosales: does the bug have all the details? [21:45] natefinch, mind putting your findings in the bug 1226840 and handing off? [21:45] <_mup_> Bug #1226840: Windows client needs to make unix paths when working with the bootstrap node [21:46] yep [21:46] heh, the dreaded slash vs. backslash issue [21:46] using filepath instead of path [21:47] thumper: yep [21:47] I think path.Join always uses / yes? [21:47] thumper: yep [21:47] but filepath.Join will use local [21:47] natefinch: so I may need to go grab a windows laptop? [21:47] at least I have one in the lounge [21:47] natefinch: how hard is the windows setup? [21:48] thumper: you'd want to build on linux... I don't even know if it'll build on windows [21:48] thumper: using Dave's cross compile scripts, at least, that's what I did [21:48] oh... [21:48] thumper, if you want to see the error the installer natefinch is nice for widows [21:48] natefinch: do you have a branch in progress? [21:48] I've poked the cloudinit stuff a lot before [21:48] thumper: I have a couple changes, but they're really small [21:49] natefinch: can you commit and push? [21:49] yep [21:49] and give me the branch [21:49] I could use it as a starting point [21:49] and I'll work with Ian and Andrew and Dave to move forwards [21:49] davecheney: you around? [21:49] thumper, current installer is at http://ubuntuone.com/7aKY92pbRTXGe6IiSVzcMl per natefinch [21:49] arosales: no point me using that if it is broken [21:50] thumper: branch is here: bzr+ssh://bazaar.launchpad.net/~natefinch/juju-core/014-fix-windows-build/ [21:50] thumper, ack. Wasn't sure if you wanted to see the current error [21:50] natefinch: I wonder how go get works on windows with git & hg branches [21:50] arosales: can see it in the bug [21:50] thumper: you have to install git and hg [21:50] thumper: and bzr, obviously [21:51] and also, should be easy enough to find the parts in cloud init [21:51] if you know what you are looking for [21:51] it is the old problem [21:51] $1 to hit the machine with the hammer [21:51] $9999 to know where to hit it [21:51] thumper ok thanks. Any help on moving the bug forward would be a plus so we can move https://juju.ubuntu.com/install/ back to a download button. [21:53] thumper: really appreciate the help. Is there anything I need to do to give you permission to work on my branch? [21:55] natefinch: all I do is branch yours and commit locally [21:55] can then push to my area [21:55] and it has your changes [21:55] DVCS FTW [21:55] thumper: btw, I made a "utils.JoinServerPath" so that we could 1.) find all the spots where we're intentionally not using filepath.Join, and 2.) use it later in case we ever have the client on linux and server on windows (not implemented, but available) [21:55] thumper: ahh, right. awesome [21:56] natefinch: ah intersting point [21:56] although [21:56] thumper: right now it just passes through to path.Join [21:57] thumper: and the couple of spots in cloud init that use it seem to fix the problems I saw with files in root with backslashes... but it seems to never finish bootstrap [21:57] (even with the new fix that is) [21:57] that's where I stopped [22:00] otp [22:02] ok [22:03] natefinch: so you are bootstrapping ec2 from a windows box? [22:03] thumper: correct [22:03] * thumper nods [22:03] thumper: also tried azure, same deal [22:04] ok, I'll take a look and talk with davecheney about his cross comple stuff [22:04] and grab one of the kids laptops to test on [22:05] thumper, note I didn't see this problem on 1.14 with ubuntu on Azure, and davecheney tried aws [22:05] I'll bootstrap hpcloud real quick too [22:05] arosales: you *don't* see the problem? [22:05] thumper: cross compile is pretty easy. basically just need to be running Go from source, run Dave's script to generate the different builders, and then you can run "go-windows-386 build" (or generically go-OS-arch in place of regular go tool to run that tool) [22:06] natefinch: is your branch based on 1.14 or trunk? [22:06] thumper, correct, I do _not_ see the problem on ubuntu clients [22:06] thumper: the one I linked is based on 1.14 [22:06] so who does see a problem? [22:06] thumper: he's not running the client from windows [22:06] ah [22:06] in which case arosales, you don't count [22:07] natefinch: hmm, I'm not running go from source [22:07] I may ask davecheney to do the actual compiles for me :-) [22:07] hmm... [22:07] * thumper starts [22:08] natefinch: we aren't likely to use cloudinit on windows servers [22:08] natefinch: so using a server path there not such a big deal [22:08] natefinch: but useful to have the method for all the agents [22:11] thumper, just passing along a data point. I'll let you get to it. [22:11] :) [22:11] ack [22:20] thumper: thanks again for the help. I'm going AFK for dinner, but will check in after. You can also throw builds at me to run until Dave gets online. he may be in late, he was up until 1am. [22:21] * thumper nods === natefinch is now known as natefinch-afk [22:21] found and fixed one [22:21] thumper: nice [22:23] in a way, this is kind of funny [22:49] natefinch-afk: lp:~thumper/juju-core/014-fix-windows-build, I have just used path.Join in cloudinit and upstart, as those are linux specific as far as I know [22:49] and I think we need more thought about supporting multiple server operating systems [22:50] I also added a trace to write out the cloud init that is generated [22:50] so the commandline can see the bootstrapped cloudinit [22:50] davecheney: can you ping when you arrive? [22:50] davecheney: although if you are starting late due to your late start, you might hit me at the gym, so I'll reply when back [22:51] natefinch-afk: at least writing out the cloudinit will let us eyeball the setup so we can see if there are other places where we are missing slash/backslash tweaks [22:51] simplest way to see it is to do this: [22:52] juju bootstrap --log-config=juju.environs=TRACE --show-log [22:52] thumper: back [22:52] thumper: good thinking about printing out cloudinit ... probably handy to have anyway [22:53] gah, I hit the 1.14 / trunk azure thing trying locally [22:53] natefinch-afk: yeah, I think so [22:53] and since it is at TRACE level [22:53] you have to explicitly ask for it [22:53] --debug won't show it [22:53] natefinch-afk: there was another place in cloudinit you had missed, but also the upstart service scripts [22:54] it seems that go tries to be magic [22:54] if you call filepath.Join('/var/lib/juju', 'blah') on windows it gives you '\var\lib\juju\blah' [22:55] at least that is what it looks like [22:55] not tried it [22:55] thumper: ug... that seems like too much magic. [22:56] I think that is why we see 'varlibjujudb' [22:56] as the \v -> v [22:56] weird that sometimes the slashes get switched completely [22:56] and sometimes, just bits of it [22:56] it's interesting that in certain spots we were getting no separator and certain spots we got backslashes [22:56] yeah === natefinch-afk is now known as natefinch [22:57] if you are able to comple and run that branch, with the logging and pastebin the cloudinit, we can eyeball it while it tries to start [22:57] natefinch: in two weeks, I'll be an hour clouser to you [22:57] as nz hits UTC+13 [22:57] heh [22:58] seems that there is a month of that, then we hit another hour closer as you fall back an hour [22:58] hi davecheney [22:58] and your alter ego davechen1y [22:59] thumper: error: flag provided but not defined: --show-log [22:59] ah... yeah [22:59] use --debug isntead [22:59] but the log-config should still work [22:59] --show-log is obviously just on trunk [23:01] thumper: why the hell does the command line's scrollback default to like 200 lines :/ [23:01] haha [23:01] can you set it to bigger? [23:01] thumper: yeah [23:02] default is 300.... wtf is this, 1990? :/ [23:04] thumper: http://pastebin.ubuntu.com/6121592/ [23:04] thumper: apologies ahead of time for line ending munging [23:05] thumper: actually sort of nice for the certs, though :) [23:05] * thumper reads [23:07] natefinch: ok, that looks like it should work [23:07] lets wait and see [23:08] thumper: looking good.... machine 0 is up [23:09] \o/ [23:09] now the rest of the cloudinit bits are created by machine 0 [23:09] so there shouldn't be any more issues [23:09] lets get this merged into 1.14 [23:09] and then into trunk [23:10] natefinch: I can push and propose mine, and you can review if you like [23:10] sounds good [23:11] oh ffs [23:11] * thumper just uses LP [23:11] haha [23:12] thumper: just in case, here's the contents of cloud-init-output.log: http://pastebin.ubuntu.com/6121614/ [23:14] natefinch: https://code.launchpad.net/~thumper/juju-core/014-fix-windows-build/+merge/186179 [23:15] natefinch: so you were able to get a successful 'juju status' from windows? [23:15] yep [23:15] awesome [23:15] thumper: http://ec2-23-20-77-242.compute-1.amazonaws.com/wp-admin/install.php [23:16] thumper: end to end. makes me happy. [23:16] \o/ [23:19] natefinch: fyi, you don't need to claim the review before you do it, if you are a member of the team that has a pending review, and you provide a review status with the comment, you claim it automagically [23:20] cool [23:23] wallyworld___, hello do you have an RT open for hosting juju-tools simple streams? [23:23] arosales: yes - you were cc'ed on it :-) [23:23] i can look up the number [23:24] 63925 [23:25] wallyworld___, thanks for reminding me. I'll post to that RT [23:25] ok. hope they act on it soon [23:25] thumper: reviewed, approved. I gotta close it up for the night [23:27] thumper: looks like i missed some fun with windows. i skimmed some of the backscroll [23:28] wallyworld___: fun.... yeah. [23:28] I'm at EOD. Thanks for the help thumper. [23:31] sinzui: ping, we should tool up for a 1.14.1 release later this week [23:31] just as a tester [23:32] wallyworld___: yeah [23:33] wallyworld___: there is a clear difference between something running on windows and not crashing, and having the tool actually work [23:33] wallyworld___: magic path issues [23:33] sigh [23:33] unix was there first [23:33] expect more pain like this when we want to have the server be windows [23:33] why didn't windows stick to convention [23:33] * thumper shrugs [23:34] I expect that msdos just followed the earlier dos [23:34] and it flowed from there [23:34] cpm [23:34] thumper: i have a thing at the school today at the same time as the code review meeting [23:34] ok [23:34] so we could reschedule till tomotrrow [23:34] did you want to do it earlier? [23:35] I'm also happy to reschedule [23:35] i'd prefer to do some coding :-) [23:35] haha [23:35] i want to get some bootstrap things done [23:35] ack [23:35] should be finished today though, hence clear my plate wrt that issue [23:36] i just wish my branch had been looked at by the ocr last night :-( [23:36] sigh [23:42] wallyworld___: which branch, I could look [23:42] thumper: that would be neat, but be careful what you wish for :-) https://codereview.appspot.com/13632056/ [23:43] i had a high level pre imp with roger, but the exact implementation was not discussed, just the high level removing of reties in s3 [23:43] wow [23:43] it's a lot of files touched, but only a few lines in each [23:44] I'll look when back from the gym [23:44] * thumper nods [23:44] ok, thanks muchly [23:44] bootstrap is fast again now :-) [23:44] and live tests on ec2 pass [23:44] but they sure do take ages to run [23:46] \o/ [23:46] I know what you mean about wanting to code again [23:46] yeah [23:46] I've now cleared my plate from all previous pending branches [23:47] and was going to start with the kvm stuff this morning [23:47] but saw the windows bootstrap issues [23:47] cool [23:47] so helped out there [23:47] will look at your review this avo, and start poking kvm [23:47] ok, ta