[03:14] axw: ping [03:14] thumper: PONG! [03:15] axw: do you have a few minutes to talk though the uniter fix you proposed? [03:15] thumper: sure [03:15] axw: https://plus.google.com/hangouts/_/canonical.com/arghh-run-sucks?authuser=1 [03:15] hehe [04:07] axw... hmm... not looking good just now, seems like the tests are hanging [04:11] axw: hmm... [04:11] ran the uniter tests by themselves and all is good [04:12] did hit two other problems with the whole test suite [04:12] one was a weird tear down failures in cmd/jujud where everything paniced because a mongo session was already closed [04:12] and the other is the peergrouper tests failing with Go 1.5 [04:13] * thumper tries to write a failing test [04:19] UniterSuite.TestUniterUpgradeConflicts intermittent failure [04:19] hazaah [04:19] NOT [04:23] axw: confirmed here too that this fixes the bug as presented [04:23] thumper: cool [04:24] axw: now the hard bit, write a failing test with the current code that the code fixes :) [04:35] * thumper afk to collect a child [04:43] thumper: the panic in cmd/jujud due tot he closed session is fixed [04:43] well, i have the fix but m waiting for master to land... [04:44] *for master to un-block that is... [04:52] good to know [05:06] hey git folks, is there a way to git stash some of the changed files? [05:07] or interactive stash, like bzr shelve? [05:12] thumper: git stash [05:12] :D [05:24] Bug #1498746 opened: azure: units fail to attach block storage

[05:27] Bug #1498746 changed: azure: units fail to attach block storage

[05:30] axw: Your change which I approve, and my test: http://reviews.vapour.ws/r/2735/ [05:30] test confirmed failed before the good change [05:30] thumper: looking [05:32] thumper: sorry to be annoying, but can you delete the TODO(fwereade) at the top of Uniter.RunCommands? I forgot to delete it while I was there [05:33] axw: ah yes, was meaning to do that [05:33] Bug #1498746 opened: azure: units fail to attach block storage

[05:34] thumper: LGTM, thanks. I'll take care of updating master later on [05:34] axw: ok, I'll fix 1.25 [05:34] should be same fix [05:37] * thumper done now, [08:31] voidspace, ping - can we catch up sometime today. [08:32] frobware: sure [08:32] frobware: after standup? [08:33] voidspace, possibly. clashes with dooferlad, but we can discuss in standup [08:34] frobware: ah, it's Wednesday [08:34] frobware: feels like a monday [08:34] frobware: ok, at your convenience [08:34] frobware: I'm here all day... [08:34] frobware: now is good for me if you're free, otherwise later [10:49] Bug #1498859 opened: juju.InitJujuHome should not have side effects [11:15] sinzui: I'm stumped [11:16] sinzui: exact revision using quickstart on that bundle .. no issues in half a dozen tries [11:19] Bug #1498869 opened: doc/backup_and_restore.txt describes deprecated backup plugin [11:55] mgz: ping [11:56] sinzui: ping [12:02] anastasiamac perrito666: standup? [12:05] brt [12:28] Bug #1497297 changed: TestFindToolsExactInStorage fails for some archs Again

[12:34] Bug #1497297 opened: TestFindToolsExactInStorage fails for some archs Again

[12:38] rogpeppe, yeah, I tried that :( [12:40] Bug #1497297 changed: TestFindToolsExactInStorage fails for some archs Again

[12:43] Bug #1496997 changed: TestErrorReadingEnvironmentsFile calls chmod on win

[12:43] Bug #1498869 changed: doc/backup_and_restore.txt describes deprecated backup plugin [12:44] mattyw: hrmph [12:45] rogpeppe, master is unblocked thought - you should be able to go for it now? [12:46] mattyw: ah, cool [12:46] Bug #1496997 opened: TestErrorReadingEnvironmentsFile calls chmod on win

[12:46] Bug #1498869 opened: doc/backup_and_restore.txt describes deprecated backup plugin [12:49] Bug #1496997 changed: TestErrorReadingEnvironmentsFile calls chmod on win

[12:49] Bug #1498869 changed: doc/backup_and_restore.txt describes deprecated backup plugin [12:52] Bug #1496997 opened: TestErrorReadingEnvironmentsFile calls chmod on win

[12:52] Bug #1498869 opened: doc/backup_and_restore.txt describes deprecated backup plugin [12:56] sinzui, mgz: can you mark this as closed? https://github.com/juju/juju/issues/3179 [12:57] bogdanteleaga: Done [12:57] thanks [12:58] Bug #1496997 changed: TestErrorReadingEnvironmentsFile calls chmod on win

[12:58] Bug #1498869 changed: doc/backup_and_restore.txt describes deprecated backup plugin [12:59] editor with completion and checking plus compiling mongo, this computer is about to bail on me [13:04] Bug #1498904 opened: outdated lp:juju-core branch [13:10] Bug #1498904 changed: outdated lp:juju-core branch [13:22] Bug #1498904 opened: outdated lp:juju-core branch [13:28] Bug #1498904 changed: outdated lp:juju-core branch [13:30] frobware: I think I am seeing the same problem as 1491592 [13:30] mbruzek, which is good... [13:31] frobware: Are you unable to reproduce? [13:31] frobware: I added a comment to https://bugs.launchpad.net/juju-core/+bug/1491592 [13:31] Bug #1491592: local provider uses the wrong interface

[13:31] Bug #1498904 opened: outdated lp:juju-core branch [13:32] mbruzek, I could not reproduce. were you able to reproduce based on the git repo I cloned/ [13:33] frobware: no I did not clone a repo, I was just working on some stuff and noticed the problem today. [13:33] mbruzek, I noticed in the original description that "There was no reboot of any of his systems." [13:33] frobware: Yes I believe that is correct, unfortunately my reproduction was involving a reboot [13:34] mbruzek, either way there's obviously an issue here. Your repro is a lot quicker/easier [13:35] Bug #1498904 changed: outdated lp:juju-core branch [13:35] frobware: I still have the error environment up. Do you want any additional information before I destroy-environment? [13:36] mbruzek, please could you add your environment info/setup to the bug report [13:36] mbruzek, I see "-e kvm" but don't know what the setup really is. [13:36] frobware: absolutely, you want my environments.yaml for that environment? And what else? Any log files or other information? [13:37] mbruzek, ahh... is this repro on 1.24.5? [13:37] mbruzek, yes to environments.yaml. [13:40] frobware: all-machines.log ? I can not get to the docker system any longer to get that machine log file [13:40] mbruzek, on the KVM node did you add a bridge? https://jujucharms.com/docs/devel/config-KVM#kvm-guest-network-bridge [13:41] what is the process to add a feature flag? [13:43] frobware: I don't remember adding a bridge but I do see lxcbr0 in my interface listing. [13:44] Is it possible that was created by lxc ? [13:44] where kvm != lxc ? [13:45] mbruzek, oh certainly [13:45] mbruzek, when you said "Restart the KVM host" which machine is that? [13:45] my laptop [13:45] I was using these two charms yesterday and restarted, tried to continue to use them today [13:46] but I could not ssh to the docker/0 charm [13:52] frobware: in any case I can still get to the nagios kvm machine, and it has network access [13:52] frobware: do you need anything else before I destroy the environment? [13:56] mbruzek, nope & thanks [13:58] perrito666: you're assuming there's a process === natefinch-afk is now known as natefeinch === natefeinch is now known as natefinch [14:47] Bug #1498942 opened: UniterSuite.TestRunCommand fails on windows

[14:49] hi katco bug 1495591 (the master bug of 1498942) is a regression in 1.24 There is a trivial fix already in master that can fix 1.24 [14:50] Bug #1495591: TestRunCommand fails on windows

[14:53] sinzui: pretty sure she's out for the rest of the week [14:53] ouch [14:53] sinzui: I will see who should take it [14:54] thank you natefinch : thumpr will find work for himself if it isn't resolved in a few hours [14:55] sinzui: I can backport the fix listed in the bug, if we want [14:56] natefinch: that is preferrable to rolling back. I think the fix is trivial to apply [14:56] sinzui: wicked trivial [15:02] Bug #1498942 changed: UniterSuite.TestRunCommand fails on windows

[15:04] does anybody have "extensive" knowledge of how proxies work in juju? [15:05] Bug #1498942 opened: UniterSuite.TestRunCommand fails on windows

[15:05] bogdanteleaga: probably not [15:05] bogdanteleaga: I know we've had customers complaining that we obey the OS's proxy settings too much... which I find amusing [15:08] sinzui: it's working: https://github.com/juju/juju/pull/3357 [15:08] thank you natefinch [15:11] natefinch: any idea if there are proxy settings being obeyed right at the first connection to the state machine? and if so, how? [15:11] Bug #1498942 changed: UniterSuite.TestRunCommand fails on windows

[15:14] Bug #1498942 opened: UniterSuite.TestRunCommand fails on windows

[15:15] bogdanteleaga: I really don't know the details, would have to dig into the code. [15:15] bogdanteleaga: I know we've had customers complain that they can't customize the port that the state server listens on, so they have to open their firewall for the port we've chosen [15:17] natefinch: ping [15:17] Bug #1498942 changed: UniterSuite.TestRunCommand fails on windows

[15:17] voidspace: sup? [15:17] natefinch: got a couple of minutes spare to help me with a problematic transaction? [15:17] natefinch, thanks for stepping up to help w/ https://launchpad.net/bugs/1495591 [15:17] Bug #1495591: TestRunCommand fails on windows

[15:19] alexisb: merging 3 line changes is my specialty ;) [15:19] :) [15:19] voidspace: sure [15:22] natefinch: I sent you a pm with the details [15:23] no reason it should be a pm really though [15:23] this assert: http://pastebin.ubuntu.com/12531747/ [15:23] always causes this failure: &errors.errorString{s:"cannot set addresses of machine 0: state changing too quickly; try again soon"} [15:23] and I'm wondering why... [15:24] omitempty is set, so I wonder if a missing value would be the problem [15:24] but putting an arbitrary assert that should always pass also seems to trigger the same problem [15:24] so I suspect I'm missing something fundamental [15:25] I think it's a problem with the transaction code in mgo [15:26] voidspace: as I said in DM it's probably a copy of this bug: https://bugs.launchpad.net/juju-core/+bug/1334773 [15:26] Bug #1334773: Upgrade from 1.19.3 to 1.19.4 cannot set machineaddress

[15:26] natefinch: :-/ [15:26] natefinch: thanks [15:27] you might ask niemeyer if he's around today. [15:27] natefinch: we have some tests that actually expect that error (not for machine address but contention tests) [15:29] voidspace: this looks like the error: https://github.com/juju/txn/blob/master/txn.go#L41 [15:29] natefinch: right, thanks [15:29] natefinch: so it's an error that we're raising [15:30] natefinch: that makes me think it's likely to be my fault [15:30] more debugging needed [15:30] Bbl [15:32] Bug #1498968 opened: ERROR environment destruction failed: destroying storage: listing volumes: An internal error has occurred (InternalError)

[15:32] voidspace: that error is caused by too many of ErrTransientFailure, which is only ever returned here, AFAICT: https://github.com/juju/juju/blob/6c929a78879c396a2a244caad99851375984e7f8/state/service.go#L129 [15:34] natefinch: I thought it meant that the transaction had failed too many times [15:34] hmmm [15:35] voidspace: well, yes [15:35] there's no service.Destroy involved here, that's for sure [15:37] voidspace: oh yeah, it looks like if you get txn aboirted too many times, it will return that error [15:37] voidspace: misread a loop [15:38] yeah, I just don't see why that transaction would fail - I'm just asserting that the field I'm changing hasn't been changed by anything else [15:41] natefinch: found it [15:41] natefinch: it's specified as "bson:,omitempty" [15:41] natefinch: needed to avoid a migration [15:41] natefinch: but causes the transaction to fail the first time I set the value - comparing a missing value against an empty address fails [15:41] hmmmm [15:42] that's tricky [15:50] natefinch: so I can assert that the field is either unchanged *or* missing, and that works [15:51] voidspace: ahh, there you [15:51] go [16:21] dooferlad, did you come to any conclusion w.r.t. containers and spaces? [16:22] frobware: still testing, but it looks like EC2 containers are broken. [16:23] frobware: have created bug 1498982 [16:23] Bug #1498982: failed configuring a static IP for container "1/lxc/0": cannot allocate addresses: instId not supported [16:29] dooferlad, voidspace: the order of args to `subnet add' seems awkward to me. I want to do something like: juju subnet add dmz $(get-me-some-subnet-ids) ... [16:30] frobware: we wanted space to be an optional argument, not sure if that was a factor [16:30] so subnets default to the default space [16:31] voidspace, I wonder which is the common case though: adding to [default], or something very explicit. [16:33] voidspace, on EC2 If I get "cannot run instances: Request limit exceeded" is there something I can do ... ? [16:35] Bug #1498982 opened: failed configuring a static IP for container "1/lxc/0": cannot allocate addresses: instId not supported [16:38] Bug #1498982 changed: failed configuring a static IP for container "1/lxc/0": cannot allocate addresses: instId not supported [16:41] dooferlad, any idea about working around EC2 "cannot run instances: Request limit exceeded" [16:41] Bug #1498982 opened: failed configuring a static IP for container "1/lxc/0": cannot allocate addresses: instId not supported [16:43] frobware: seems like the canonical account has enough people hammering it that we are hitting the request limit. Unfortunately it seems like the solution is to ask Amazon to up the limit for us. [16:44] frobware: which the account holder can do, but I am not sure who that is. You could try joey on the Canonical IRC server [16:44] http://docs.aws.amazon.com/AWSEC2/latest/APIReference/query-api-troubleshooting.html#api-request-rate [16:44] http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html [16:44] :-( [16:45] frobware: strange thing is that I am not hitting that message and I am using the canonical account on some machines in eu-central-1 [16:47] frobware: sorry, just seen this [16:47]