[00:00] jcw4: LGTM, thanks [00:00] marcoceppi: cool [00:12] thumper: dummy.Bootstrap sets a bool: estate.bootstrapped = true. dummy. dummy.StateServerInstances errs after checking that bool: [00:12] if !estate.bootstrapped { [00:12] return nil, environs.ErrNotBootstrapped [00:12] } [00:14] thumper: so, I don't see how writing the bootstrap IP to storage will change estate.bootstrapped? [00:23] perrito666: sorry... it's been a crazy morning. are you still around? === menn0_ is now known as menn0 [00:26] thumper: if you have a moment, https://code.launchpad.net/~wallyworld/golxc/lxc-console-log/+merge/228778 [00:36] waigani: * for i in `seq 0 4`; do juju deploy trusty/ubuntu --to lxc:0 trusty; done; [00:36] sorry [00:36] wallyworld: [00:36] * for i in `seq 0 4`; do juju deploy trusty/ubuntu --to lxc:0 trusty; done; [00:36] * for i in `seq 0 4`; do juju deploy precise/ubuntu --to lxc:0 precise; done; [00:36] ^ from dpb's email [00:36] this will result in a broken juju [00:36] what he means to do is juju deploy trusty/ubuntu -n 4 [00:37] davecheney: haven't seen that email yet, but will look soon [00:37] what will happen is the furst unit of the trusty/ubuntu service will deploy [00:37] the other 7 will fail [00:39] yes, i think you are right [00:39] thanks [00:43] wallyworld: in that case, i have no idea what is going on [00:43] does --to lxc:0 override the duplicate service check ? [00:44] don't think so [00:44] in that case i cannot explain what dpb is doing [00:44] or how it works [00:44] or how it didn't work but he didn't notice [00:45] i'm also loath to comment as it will be a distraction [00:45] ok, i'll look into it [00:45] ta [00:45] thank you too [00:54] thumper: i'm down to three api packages failing, some 10 test cases left to fix [01:30] thumper: wallyworld http://paste.ubuntu.com/7900177/ [01:30] do you agree that this change is correct [01:30] it is the source of a lot of problems [01:30] AuthAlways is curently accepting anything, including invalid data and returning true [01:31] when I change it to only return true for valid tags => dozens of test failures [01:33] waigani: have you found the problem then? [01:34] davecheney: that change looks ok to me [01:35] thumper: not yet [01:36] thumper: if !canWatch("") { [01:36] return result, common.ErrPerm [01:36] } [01:36] this is rife through the codebase [01:36] what does this mean ?!?! [01:36] canWatch(anything?) [01:36] canWatch(nothing?) [01:36] authenticate anyuthing [01:37] even the empty string !?!? [01:41] I'm not sure... [01:43] axw: i tested the endpoints branch on 1.20, seems to work ok, plus has the fix for the list rendering, so off it goes to the bot [01:44] all bollocks, except bot is blocked [01:45] thumper: can you review that lxc fix so i can unblock the bot? [01:45] wallyworld: doing it now [01:45] fanks [01:45] wallyworld: rework needed [01:46] wallyworld: lxc-create doesn't support long opts [01:46] :-( [01:46] lxc-start does [01:46] hazaah [01:46] thumper: but lxc-creat --help says otherwise [01:47] wallyworld: of which version? [01:47] thumper: so what are you basing that on? [01:47] wallyworld: I'm looking on the man page, and output of lxc-create on precise 0.7.5 [01:47] hmm, i have 1.0.4 [01:47] sigh [01:48] heh, the man page for lxc-create says no long opts, lxc-create --help says yes [01:48] for 1.0.4 [01:48] yep [01:48] go 'man lxc-create' [01:48] geez [01:48] confusing [01:48] yep [01:48] thumper: i tested running up a local env [01:48] but that was 1.0.4 [01:48] sure, but you have a modern lxc [01:49] are you sure 0.7.5 doesn't support long args? [01:49] yep [01:49] ok, and it's just create? [01:49] well, not actually tried passing arg [01:49] seems to be [01:49] well [01:49] i only looked at lxc-create and lxc-start [01:50] lxc-stop has long [01:50] lxc-info, lxc-wait? [01:50] lxc-destroy short only [01:50] thumper: fuck, it, maybe we just go back to short [01:50] lxc-info is long [01:50] lxc-wait is long [01:51] ok, so just create, destroy [01:51] * thumper nods [01:51] rightio [01:51] I do wonder if we could just say "your lxc is too old, go away" [01:52] i wish [01:53] thumper: pushed changes [01:53] ah sorry, left in info [01:54] will delete those, that was for debugging [01:55] lxc-clone is short only on old :( [01:55] wallyworld: ^^ [01:56] ffs [01:57] thumper: try now [01:58] done [02:01] thumper: ty [02:03] thumper: it just gets worse [02:03] http://paste.ubuntu.com/7900384/ [02:03] the fake authoriser used in our tests rarely has the tag that it is authenticating set [02:04] davecheney: if an arg can be empty in a test, you can bet your eggs that someone will pass "" through [02:05] any hints why 'juju ssh 0' keeps giving me Permission denied (publickey,password) ? [02:05] jcw4: yes [02:05] jcw4: I bet you are using the local provider [02:05] thumper: mind reader! [02:05] jcw4: and you have no ubuntu user on the host [02:05] thumper: wow, interesting [02:05] and/or your aren't running sshd on the host [02:06] the former [02:06] jcw4: don't do it [02:06] not supported [02:06] oh? juju ssh not supported on local? [02:06] saves you from running "juju run --machine 0 'sudo reboot; " [02:06] lol [02:06] just for machine 0 [02:06] because it is the host [02:06] I'd love to fix the local provider so machine 0 is a container too [02:07] but I don't have enough working hours [02:07] I'm trying to figure out why all my machines stay stuck in pending except for 0 [02:07] * thumper sighs [02:08] * thumper raises his hand again [02:08] * jcw4 feels nervous now [02:08] probably my fault [02:08] hopefully wallyworld is going to fix it to be less shit [02:08] jcw4: please run "sudo lxc-ls --fancy" for me [02:08] yep, rsn [02:09] lxc: Error: juju-trusty-template creation was not completed [02:09] NAME STATE IPV4 IPV6 AUTOSTART [02:09] ---------------------------------------------------- [02:09] juju-trusty-template STOPPED - - NO [02:09] interesting... [02:09] jcw4: ok, do this: [02:09] sudo lxc-destroy -n juju-trusty-template [02:09] jcw4: then... [02:10] * thumper wonders where the hell he put the lock file [02:10] container not defined? [02:11] n/m; at least lxc-ls is empty now [02:11] jcw4: look in /var/lib/juju/locks [02:11] juju-trusty-template/message|held [02:11] jcw4: delete that dir [02:12] done [02:12] now I expect one of your machines will fail to start [02:12] and then the next will be slow to start [02:12] as it is recreating the template [02:12] the rest should be fast [02:13] do a "sudo lxc-ls --fancy" again [02:13] see if the template has been created again [02:13] yes, but no error now [02:13] right [02:13] now wait [02:13] jcw4: just before everything starts to fly [02:14] you'll see the template go from running to stopped [02:14] the second machine may fail if you have slow internet [02:14] oh, it was stopped when I looked [02:14] due to timing [02:14] that wallyworld if fixing right now [02:14] and machines 1, and 2 are still pending [02:14] jcw4: it may still have been creating [02:14] look now [02:14] thumper: i'm fixing the lock around downloading/creating the template [02:15] NAME STATE IPV4 IPV6 AUTOSTART [02:15] ---------------------------------------------------- [02:15] juju-trusty-template STOPPED - - NO [02:15] the fact that the lock can be left aquired [02:15] ah, now it's starting to come up [02:15] wallyworld: I was referring to the --console-log one [02:15] ah, ok [02:15] wallyworld: as that impacts the "template failed to stop" issue [02:15] thanks thumper [02:15] jcw4: np [02:16] the bot is running the merge job now [02:16] * thumper goes to fix our mongo upstart bouncyness [02:16] * thumper picks music [02:16] disturbed I think [02:17] :) [02:17] thumper: that is a nice bug [02:17] thumper: i can't hear, can you turn up the volume [02:17] agent restarts, rewrites upstart config, upstart restarts mongo, agent restarts, rinse, repeat [02:18] davecheney: aye [02:18] whups [02:18] thumper: OMG [02:19] i just found hundreds of places where the tests set fakeauthorizer.tag to somethign [02:19] without setting fakeauthorizer.entity [02:19] !?!? [02:19] i've changed it so you set the entity and that is what defines the tag [02:19] as they are not indepdeant [02:20] fakeBadAuth := apiservertesting.FakeAuthorizer{ [02:20] Tag: s.mysqlUnit.Tag(), [02:20] LoggedIn: true, [02:20] UnitAgent: true, [02:20] Entity: s.wordpressUnit, [02:20] } [02:20] oh crap [02:20] i can't even do that [02:20] althoguht I cannot explain what this is trying to test [02:20] * thumper hides and codes for 20 minutes [02:26] okay, next question [02:26] juju debug-log is getting spammed with modprobe errors [02:27] could not insert 8021q ; operation not permitted [02:27] on machine-2 [02:28] jcw4: sounds unrelated [02:29] davecheney: okay. I'm just concerned because it's drowning other messages [02:29] it's a bug for sure [02:29] but that's something related to wifi [02:29] interesting [02:29] davecheney: thx [02:31] thumper: we need to have a serious chat about apiserver/testing/FakeAuthoriser [02:31] it's wrong [02:31] in many ways [02:31] and the tests have mutated to accept it as the source of truth [02:31] davecheney: would it be better to take it to the list? [02:32] thumper: i'm not at that stage [02:32] it just keeps getting more confusing [02:32] ok, can I put you off for a few minuts to get this bug fixed? [02:32] sometimes things are authenticated againts the GetAuthEntit [02:32] others authenticate against GetAuthTag [02:32] in most cases the tag and the entity don't match, and most cases one is nil [02:33] thumper: which bug ? [02:33] https://bugs.launchpad.net/juju-core/+bug/1349969 [02:33] <_mup_> Bug #1349969: Jujud rewrites /etc/init/juju-db.conf constantly

[02:46] * thumper takes a deep breath [02:48] * thumper silently stabs the person who named a public struct param "Desc" instead of "Description" [02:49] * thumper mutters under his breath [02:51] fucking brain dead test [02:52] * jcw4 quietly does a grep to make sure it wasn't him [02:53] (╯°□°)╯︵ ┻━┻ [02:53] nice [02:54] I saw davecheney use that table flipping thing before [02:54] jcw4: emojicons.com/table-flipping [02:54] oooh fun [02:54] sorry davecheney, giving away trade secrets [02:55] is there a way to update the jujud binary without destroying and bootstrapping? [02:55] can I just kill the process? [02:57] jcw4: juju upgrade-juju --uplaod-tools [02:57] davecheney: cool [02:57] * jcw4 is embarrassed that almost all of his usage of juju has been in unit tests [03:02] wallyworld: you saw CI failed your golxc PR? [03:02] dependencies.tsv update [03:03] no, not yet [03:03] vapour wouldn't load for me [03:03] i hadn't checked back [03:03] huh, working for me [03:03] looks like a CI infrastructure failure [03:04] axw: so you can load the dashboard? [03:04] i can't [03:04] yep [03:04] http://juju-ci.vapour.ws:8080/ [03:04] yep, the browser title changes, the page goes blank, and it gets stuck reading [03:04] le shrug [03:05] axw: what does the error say (as i can't see it) [03:07] un moment [03:07] ah, chrome can load the page [03:07] i-da61a1f6 doesn't exist anymore [03:07] our ec2 instance has been removed [03:08] oh joy [03:09] oh fark... again... [03:13] wallyworld, I see a problem [03:14] sinzui: i was lookingif our image id were are using had changed [03:14] wellm that is indeed my topis [03:14] topic [03:14] script uses ami-e55a648c [03:14] testing with saucy is bogus, it is EOL [03:14] wallyworld, you want a trusty id? [03:15] maybe ami-018c9568 ? [03:15] * sinzui looks at other trusty test [03:15] i though we did have a trusty id [03:15] thought [03:15] surprised we didn't [03:15] i'll follow up with martin to see wtf is going on :-( [03:16] wallyworld, ami-018c9568 is what CI will use when it re-runs unit tests [03:16] I think I can change it now if you like [03:16] sinzui: yep, that's where i got it from :-) [03:16] i'm changing it [03:16] the github-merge-job [03:16] wallyworld: https://github.com/juju/juju/pull/424 [03:18] looking [03:19] sinzui: i was hoping to be able to do a new 1.20.2 build today but we're not ready :-( [03:20] wallyworld, I understand. I am frantically putting a set of certification tests together for Ubuntu. There just aren't enough people to do the work [03:20] and we keep finding more issues to fix :-( [03:23] wallyworld, Most of the issues I saw today weren't essential I placed them in 1.20-alpha1. I am trying to keep the issues in 1.20 to be regressions and detrimental performance [03:24] sinzui: yeah. issues we're working on: template lock, mongo restarts every 3 seconds, backport of api endpoint fix [03:24] I don't disagree with those. The restart is something I see too often myself [03:31] wallyworld: what is the typo you refer to? [03:31] it isn't a well structured sentence, but no typo [03:31] thumper: maybe i can't read. ah, it makes sense a second time [03:32] sorry [03:32] I'll restructure... [03:34] wallyworld: you merged? how did you get the instance back? [03:34] or was it just the temporary instance went away [03:34] axw: i used a new one [03:34] a trusty one [03:34] ok [03:34] i think the other one was saucy [03:35] but i didn't know that - i assumed were had been set up for trusty [03:35] axw: but something is still wrong, the console output shows no tests as being run [03:35] sigh [03:36] all i did was change the image id [03:37] wallyworld: that's a bit of a worry, since it merged... [03:37] yes indeed [03:41] axw, wallyworld don't panic. we know that the AMI is bad [03:41] wallyworld, see http://juju-ci.vapour.ws:8080/job/run-unit-tests-trusty-amd64/1215/console [03:41] oh [03:41] got a good one? [03:42] wallyworld, looks like we need to find another. I can do that now. lets see these results playout before reverting [03:43] sinzui: they should be ok, it was a simple dependencies.tsv change [03:43] wallyworld, I just cannot get everything transitioned to the slaves fast enough [03:43] yep, I think it is safe too [03:43] yeah, i hear you [03:43] it's way past you eod so thank you very uch [03:50] * thumper is sad [03:50] code was broken but no tests failed [03:50] * thumper writes another test [03:52] another quick question... juju upgrade-juju --upload-tools works great... can I update a charm in a similar way without destroying/deploying it? [03:52] fark [03:52] I know why [03:52] stabby stabby [03:54] jcw4: yes [03:54] juju upgrade-charm foo [03:54] calls the upgrade hoook [03:54] thumper: it's like magic [03:54] yep [03:54] sucky tests suck [03:55] * thumper unsucks the tests [03:55] ha [03:57] wallyworld, our new ami for trusty is ami-b027efd8. I will update the jobs that seem to need it [03:57] sinzui: great, ty [04:03] axw: which todo are you referring to? [04:03] axw: nm, found it [04:04] * thumper needs coffee [04:06] wallyworld, menn0: updated the PR, axw - removed old todo [04:06] * thumper goes to make coffee [04:06] 20 minutes guess -> 2 hours seems about right [04:06] thumper: looking now [04:07] suppose I should squish all those commits for easier cherry picking into 1.20 [04:10] thumper: what about the comment I made (+1'ed by axw) about starting the service? [04:10] the other change looks good [04:11] sure, will do [04:12] menn0: that means another damn test [04:16] thumper: sorry :p [04:17] ok, so how do I squick all these again [04:20] * thumper pushes a single commit [04:20] geez I must be good to get that right first time [04:20] just don't look at the review comments [04:21] ah fark [04:21] * thumper found a problem [04:26] https://github.com/juju/juju/pull/415 [04:26] if anyone has a minute to inspect :) [04:26] I still need to add a few comments to things -- will take care of [04:28] wallyworld: I've updated the LP bugs related to this morning's discussion with everything we currently know and what's being done [04:28] wallyworld: should I also email juju-dev asking for help? [04:28] menn0: \o/ thank you [04:28] with the mongo issue? [04:28] yep [04:28] the panic [04:28] yes please [04:29] ok doing that now [04:32] menn0: and another look at my branch? [04:33] ah fark... [04:34] * thumper stabs it again [04:34] thumper: huh? [04:34] old comments in the new tests [04:34] just removed them [04:37] wallyworld: is the bot unblocked? [04:37] le tme check [04:38] you folks have until my coffee is made to complain about that branch, otherwise I'll take the previous four LGTMs and merge it [04:38] changed somewhat [04:39] thumper: appears to be yes [04:44] menn0: changing the commit message, just for you [04:55] thumper: aww shucks [04:57] is trunk unblocked ? [05:01] woot! we have an end to end action [05:01] marcoceppi, :) [05:03] axw: i have a flock implementation which hopefully will be suitable https://github.com/juju/juju/pull/425 [05:04] ＼(＾O＾)／ [05:06] * thumper goes to drop of kid a sports === thumper is now known as thumper-afk [05:19] wtf, why can't i merge things [05:20] I just saw something lang [05:20] land [05:25] we are going to store metadata for backups in the state...so I'm guessing we need a new collection, etc. [05:25] is that right? what's the best way to do that? [05:27] ericsnow: obtain paper bag, breath deeply [05:27] let the carbon dioxyde sooth you [05:28] davecheney: careful, it's quite late here and my brain is already rather fuzzy :) [05:34] davecheney: critical bugs blocking landing [05:41] axw: ping [06:08] hi [06:08] tried to land my PR for juju-core and got this response from the bot: Does not match ['$$fixes-1347715$$', '$$fixes-1342725$$'] [06:08] what does this mean? [06:11] tasdomas, this means that sinzui has put his foot down -- it's explained on juju-dev === uru_ is now known as urulama [06:16] thumper, wallyworld how come this one landed ? https://github.com/juju/juju/pull/419 [06:17] davecheney: 1.20 [06:17] different target branch [06:17] wallyworld: why did it fail the first time ? [06:18] davecheney: the critical bug affected 1.20 also [06:18] now that one has been fixed [06:18] wallyworld: yes, but the magic string was never mentioned [06:18] i just want to make sure the bot is working properly [06:18] Build failed: Does not match ['$$fixes-1350011$$'] [06:18] build url: http://juju-ci.vapour.ws:8080/job/github-merge-juju/157 [06:18] it was wasn't it? [06:19] it was what ? [06:19] mentioned [06:19] you asked if the magic string was mentioned [06:19] i pasted the comment from the failed merge showing it was [06:19] yes, and I cannot see it there [06:19] the bot said that [06:20] so the build fails, the bot then comments with the right string and resubmits itself ? [06:20] no, the developer has to [06:20] wallyworld: i cannot see a comment from a developer with the right string [06:20] i saw the failed pr and resubmitted [06:20] wallyworld: you said $$Merge$ [06:20] that isn't the correct string [06:21]