/srv/irclogs.ubuntu.com/2016/04/29/#juju-dev.txt

davecheneycherylj: looks like CI is still running 1.200:07
davecheneyat least on the 1.25 branch00:07
davecheneydoes sinzui 's fix need to be backported to 1.25 ?00:08
davecheney>was it just that  the go compiler sucks compared to c++?00:08
davecheney^ i literally cannot even00:08
anastasiamac_wallyworld: axw: i think this is ready to go http://reviews.vapour.ws/r/4573/ could someone plz stamp it? :D00:09
sinzuidavecheney: What is happening here http://reports.vapour.ws/releases/3933/job/run-unit-tests-race/attempt/138700:09
davecheneyta00:10
wallyworldsinzui: are all the juju tests now run on go 1.6?00:10
wallyworldfor 2.0?00:10
davecheneythe race detector does not support ppc6400:10
davecheneyhang on00:10
davecheneyis this ppc6400:10
sinzuiwallyworld: no windows and centos are tested on both 1.2 and 1,6 because the 1.6 do not pass00:10
davecheneyno, this is amazon00:10
sinzuidavecheney: That is exactly my wtf moment. a trusty containter that thinks it is ppc00:11
wallyworldsinzui: that's the unit tests right?00:11
sinzuiwallyworld: yes00:11
davecheneyok, so the race deector in the trusty golang1.6 deb is busted00:11
davecheney:900:11
davecheney:(00:11
davecheneypaging mwhudson00:11
wallyworldsinzui: i thought horatio fixed the windows tests on 1.6, i'll need to check00:12
mwhudsondavecheney: ah yes00:12
sinzuiwallyworld: no his last fix was for go 1.2 regressions00:12
sinzuiwallyworld all packages have been built with go 1.6 for several weeks00:12
mwhudsondavecheney: try https://launchpad.net/~mwhudson/+archive/ubuntu/trusty-race-detector/+packages ?00:12
wallyworldsinzui: the issue right now is there's some upstream azure sdk changes we need but those require go 1.6 to build, so we need 1.6 everywhere first00:13
wallyworldso those tests need fixing00:13
sinzuiwallyworld: IU ask every day in meeting for them to be fixed00:13
wallyworldyeah, everyone is busy :-(00:14
sinzuidavecheney: I am inclined to make the race tests non-voting until we sort out what happened00:15
sinzuiwallyworld: in the case of windows, I see bugs we already see, but more often, and some that seem to have been revealed by recent fixes http://reports.vapour.ws/releases/3933/job/run-unit-tests-win2012-amd64-go1_6/attempt/5100:17
wallyworldsinzui: oh the fix for beta6 for ODS - the pinger shutdown fix - may have introduced a new window failure00:19
sinzuiwallyworld: :(00:19
wallyworldwell just just a guess00:19
sinzuiwallyworld: I am on unknowns tomorrow. I will get every bug files for the remaining Go 1.6 tests00:19
wallyworldit seems go 1.6 on windows slowness may be exaserbating some of these failures00:20
sinzuiwallyworld: and as for slowness. I think we need to extend timeouts for centos and windows00:20
wallyworldif that helps short term, that would be nice so we can get the azure fixes in00:20
sinzuiwallyworld: I will also get that sorted out tomorrow00:23
wallyworldok, ty00:23
sinzuidavecheney: I think we need to force the race tests to run with go1.2 or quickly solve the golang 1.6 ld issue00:36
davecheneymwhudson: what say you to that ?00:37
davecheneyafaik the race detector in go 1.6 works fine00:37
davecheneybut something is wrong with how it's pacakged in trusty00:37
mwhudsonah00:37
* redir is eod later #juju-dev00:37
mwhudsonsinzui, davecheney: there is a ppa with the support files needed00:37
mwhudsonhttps://launchpad.net/~mwhudson/+archive/ubuntu/trusty-race-detector/+packages00:37
davecheneysinzui: is using that ppa an option ?00:37
mwhudsonsinzui, davecheney: i should get this into trusty too00:38
mwhudsonsinzui, davecheney: alternatively, can you run the race tests on xenial?00:38
sinzuidavecheney: yes, yes I think it is00:38
davecheneycool00:38
sinzuimwhudson: I just switched the tests to xenial. got the same error though00:39
mwhudsonsinzui: oh that's more surprising00:39
mwhudsonsinzui: do you have apt/apt-get installing recommends by default?00:39
sinzuimwhudson: no, but we support installing a different set of packages instead of using the makefile (tht is how we test s390x and arm64)00:41
mwhudsonshouldn't matter00:41
mwhudsonsinzui: can you link me to the log of it failing on xenial?00:41
sinzuimwhudson: It is happening right now at http://juju-ci.vapour.ws:8080/job/run-unit-tests-race/1388/console. You need to log in as a dveloper to see it00:42
davecheneycan someone send me the developer credentials00:44
davecheneymy browser has forgotten them00:44
mwhudsonsinzui: how does the golang-1.6 package get installed there?00:47
mwhudsonall i see is golang-1.6 is already the newest version (1.6.1-0ubuntu1).00:47
mwhudsonsinzui: anyway, somehow the golang-1.6-race-detector-runtime package needs to get installed00:49
mwhudsonsinzui: usually that gets installed when golang-1.6 does, because golang-1.6 Recommends: it00:49
mwhudsoner golang-1.6-go Recommends it, to be more precise00:49
mwhudsonsinzui: but if golang-1.6-go is somehow installed when recommends are disabled, it won't, so that's my guess as to what's happening here00:50
sinzuimwhudson: understood. The makefile forces no-recommends. the makefile got an update today to ensure golang 1.6 or higher is preffered. I will reconfigure the job to install the required packages.00:53
mwhudsonsinzui: ah makes sense00:57
axwanastasiamac_: LGTM00:59
anastasiamac_axw: \o/01:00
sinzuithans not fair. the rules to install extra packages conflict with the rules run with --race01:01
sinzuioh, all is good. thank you mwhudson and davecheney . I can see golang-1.6-race-detector-runtime is installed and the tests are running with --race on xenial01:05
mwhudsonsinzui: \o/01:06
davecheneywoot01:20
natefinchmenn0, axw, wallyworld:  anyone care for a quick review?  It's fairly trivial, but for a critical bug: http://reviews.vapour.ws/r/4735/02:41
wallyworldsure02:42
natefinchI'd gotten eric to review, but he had some questions, so I wanted another pair of eyes02:43
wallyworldnatefinch: if we have test coverage for proper use of ssl via other means, then we probably can remove that special case in the test02:50
natefinchwallyworld: well, my static analysis tests are stuck in committee... so right now, we don't.02:50
wallyworldnatefinch: ok, so no harm leaving it there for now. i think the pr is lgtm02:51
natefinchwallyworld: ok... i think that's basically where eric landed too, just wanted a second opinion.  Thanks.02:51
wallyworldnp02:51
natefinchwallyworld: do you know if the apt mirrors thing is *only* applicable to MAAS?03:03
wallyworldnatefinch: the general principal is applicable anywhere, but as axw mentioned to me, maas allows apt mirroe to be configured for it so juju could make special use of that to set the juju apt mirror setting off of what maas uses03:04
wallyworldinstead of making the user to specify manuall in 2 places03:05
axwnatefinch wallyworld: I'm not sure if it does now ... it has configuration for proxies for sure03:05
natefinchwallyworld: does juju's apt mirror get propagated to containers?03:06
wallyworldnatefinch: no, that's the whole point of this work :-)03:06
wallyworldit is supposed to03:06
wallyworldso we need to figure out why not and fix03:06
wallyworldthat's my understanding anyway03:07
natefinchwallyworld: ok... just making sure I understand the problem correctly.  Sounds like it's two things: 1.) juju apt mirrors don't get propagated to containers. 2.) (bonus points) juju should pick up maas's apt mirror and pass it into containers juju creates on maas nodes03:07
wallyworldyes, assuming maas has such a setting03:07
axwyup03:07
wallyworldgiven all this, we need to audit what's there in juju already03:07
wallyworldso we understand the current actual behaviour03:08
wallyworldand figure out the gap between that and the to be03:08
wallyworldand document the to be in that spec03:08
axwwallyworld: I meant to ask, when you've got a spare few minutes, can you please have a look over my openstack PR?03:10
wallyworldsure03:11
wallyworldaxw: looks like you already have a +1?03:12
axwwallyworld: yeah, but babbageclunk is not a graduated reviewer AFAIK03:13
wallyworldah, i didn't check who did it03:13
wallyworldjust saw the tick03:13
mupBug #1334481 changed: juju should not record ssh certificates of ephemeral hosts <ci> <cloud-installer> <landscape> <juju-core:In Progress> <https://launchpad.net/bugs/1334481>03:24
natefinchwhere did all the help go?03:27
natefinchdo we not have help on the providers in the CLI anymore?03:27
wallyworldaxw: lgtm with a drive by request03:27
wallyworldnatefinch: i think the help topics are under construction, not sure03:28
axwwallyworld: ta03:28
natefinchwallyworld: ok, that gives me hope.  Right now, juju bootstrap on a new cloud basically just gives me the middle finger with no way to figure out what I'm supposed to do.03:28
wallyworldwhat do you mean?03:28
natefinch$ juju bootstrap google google03:29
natefinchERROR detecting credentials for "google" cloud provider: gce credentials not found03:29
natefinchand as far as I can tell, none of our help actually tells you how to enter those credentials03:30
wallyworldjuju help add-credentials03:30
natefinch$ juju help add-credentials03:30
natefinchERROR unknown command or topic for add-credentials03:30
wallyworldsorry, no s03:31
wallyworldjuju help add-credential03:31
natefinchoh man... bad one to not have a plural alias03:31
wallyworldwell, it is only a single credential03:31
natefinchit is very often used in the plural, though03:32
wallyworldnatefinch: juju help bootstrap tells you what you need anyway03:32
wallyworldbut spot the sypo03:32
wallyworldtypo03:32
natefinchyeah, IWas gonna say, it even says to see "add-credentials"03:33
wallyworldso i'm not sure how the current help is deficient03:33
wallyworldbootstrap failed, juju help bootstrap has the answer03:33
wallyworldalbeit with a typo03:33
natefinchjuju help bootstrap is a wall of text.  it woudl be nice if the error about "credentials not found" also said "see juju help add-credential"03:34
wallyworldhopefully that stuff is being addressed03:35
natefinchI hope so :)03:35
natefinchwe do still need juju help google/gce03:35
wallyworldi'm not 100% sure of the status of all that03:35
natefinchunderstood03:36
natefinchgah03:57
natefinchspend 15 minutes figuring out gce's credentials only to hit another bootstrap error:03:58
natefinchcannot start bootstrap instance: sending new instance request: sending new instance request: googleapi: Error 400: Invalid value for field 'resource.networkInterfaces[0].network': 'global/networks/default'. The referenced network resource cannot be found., invalid03:58
menn0davecheney: small utils/ssh change to allow StrictHostKeyChecking to be turned on: http://reviews.vapour.ws/r/4738/04:00
menn0axw: or you? ^^04:01
axwmenn0: looking04:02
menn0cheers04:02
natefinchwhelp, can't bootstrap gce... guess it's time to file a bug04:02
axwnatefinch: :/  how are we not finding that in CI ...04:06
axwpretty sure I bootstrapped gce last week, must be something new or maybe region or account specific04:07
natefinchaxw: there was some networking stuff landed today... maybe it hasn't made it through CI yet04:07
axwah04:07
natefinchaxw: or at least, I heard them talking about networking stuff... I don't know for sure if anything landed04:07
natefinchhmm.. the commit history on master does not look like anything interesting landed lately04:09
natefinchyeah, weird, even if I do it from master  weeks ago, still fails same way04:10
natefinchI must be special04:10
natefinchor I have to configure my GCE account in some way that I haven't04:11
menn0axw: thanks for the review. I think strict should be the default too but I didn't want to break existing consumers of utils/ssh04:15
menn0axw: FWIW it will soon default to on for juju ssh/scp"04:15
menn0it's working on my machine04:15
menn0pulling the host keys from the API server etc04:16
menn0but needs tidying up and better test coverage04:16
axwnatefinch: did it fail for you immediately? it seems to be working for me, but hasn't finished yet04:16
axwmenn0: fair enough04:16
axwmenn0: cool :)04:16
natefinchaxw: failed in 10 seconds04:17
axwnatefinch: I guess it's something about your account/project04:17
natefinchaxw: yeah, I'll try creating a new project04:17
anastasiamac_yes, my gce is bootstrapping fine too from master tip..04:18
natefinchthis was a fairly old project... I'll see if a new one works better04:19
natefinchstill weird04:19
natefinchthat seems to be working better.04:22
davecheneymenn0: lookin04:25
menn0davecheney: it's all good. axw already reviewed.04:25
mgzaxw: http://reports.vapour.ws/releases/3933 <- note gce04:28
mupBug #1576503 opened: Agents stuck in failed state <juju-core:New> <https://launchpad.net/bugs/1576503>05:00
axwmgz: interesting, seems to be different tho05:01
menn0davecheney: screw you fslock...05:06
mgzaxw: possibly over quota and bad message?05:06
mgzaxw: we had passes since then05:07
axwmgz: hmm dunno, not much to go on apart from failing to get public addresses...05:08
axwsounds plausible tho05:08
axwmgz: ah yes, GCE operation error: (QUOTA_EXCEEDED) Quota 'CPUS' exceeded.  Limit: 96.005:09
mupBug #1576509 opened: Race in macaroon-bakery <ci> <race-condition> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1576509>05:19
mupBug #1576510 opened: Race in macaroon-bakery <ci> <race-condition> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1576510>05:19
=== urulama|____ is now known as urulama
menn0axw: i've just realised that this SSH host key work has some interestsing corner cases with "juju scp"05:39
menn0axw: do we care much about "juju scp 0:some.file 1:"05:39
menn0?05:39
menn0axw: it doesn't work by default because the keys aren't in place but it could05:39
axwmenn0: does scp let you do that? I thought it didn't05:40
menn0axw: it definitely does... I've just been reading up on it05:40
axwso it does05:40
axwmenn0: I must be living in the past05:40
axwuummm05:40
axwmenn0: what do you mean "keys aren't in place"?05:41
axwwhy is it different to making two ssh connections, and writing to known_hosts twice?05:41
menn0axw: when you do "scp host1:some.file host2:" there's a connection from your local machine to host1, and then host1 connects to host205:44
menn0with juju there's no keys in place to allow connections between host1 and host205:44
axwmenn0: ah, I see.05:44
axwmenn0: I would say that's up to the user to manage05:44
menn0axw: that said, with "scp -3 host1:... host2:" the connections are local -> host1 and local -> host205:45
menn0so that's more likely to work05:45
menn0except I can't make it work right now05:45
axwmenn0: I don't really think it's worth worrying about05:46
menn0axw: with a bit of work I could have juju scp generate a known hosts with the keys for both hosts05:46
menn0axw: but it does make things a bit more complex05:46
menn0axw: for now I'll generate a known hosts for the first host05:47
menn0axw: that covers the majority of cases05:47
axwmenn0: that will still cover "juju scp /local/file 0:remote/path" right?05:47
menn0axw: of course. "0:" would be the first (and only host) in that command line05:48
axwmenn0: ok, just making sure I understood you. sounds good05:48
menn0cool05:48
mupBug #1576503 changed: Agents stuck in failed state <juju-core:New> <https://launchpad.net/bugs/1576503>05:49
wallyworldaxw: a small one for friday? http://reviews.vapour.ws/r/4739/05:49
axwwallyworld: ok05:50
axwwallyworld: how does that satisfyPrerequisites function work on ubuntu < trusty?05:52
axwwallyworld: oh, this is only for series where mongo-3.2 is available05:53
axwnever mind05:53
wallyworldyeah05:53
axwseparate command, was thinking it was run by the machine agent05:54
axwwallyworld: LGTM05:54
wallyworldty05:54
anastasiamac_have u seen this before? I have just destroyed a controller (had 2 models: admin and default)...06:15
anastasiamac_when i list-controllers, i get empty list (just the headers) as expected06:15
anastasiamac_but when i list-models, i get "error: controller local.volumes not found"06:15
anastasiamac_i would have expected (and have in the past) to see empty list with just headers06:15
anastasiamac_it seems that maybe we are no longer cleaning current-controller ?.. is it bug-worthy?..06:15
mupBug #1576527 opened: listSuite.TestListJSON got null (showSuite too) <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576527>06:22
mupBug #1576528 opened: current controller not cleared on destroy <juju-core:New> <https://launchpad.net/bugs/1576528>06:22
mupBug #1576534 opened: upgrade-charm refuses to upgrade <juju-core:New> <https://launchpad.net/bugs/1576534>06:31
bdxhello everyone06:49
bdxis 'juju deploy <service> --to lxd:<machine#>' supported by the maas provider in 2.0?06:49
frobwarebdx: are you talking about MAAS 2.0, or Juju 2.0?  Currently supported for Juju 2.0.  For MAAS 2.0 & Juju 2.0 currently WIP and only available behind a feature flag and building from tip....07:19
bdxfrobware: nice, whats the flag?07:22
bdxfrobware: maas2?07:22
frobwarebdx: yep07:23
bdxfrobware: I also must build master?07:23
bdxwill this be dropping into beta7 then?07:24
frobwarebdx: yep, and a moving target... currently very active for maas2 work07:24
frobwarebdx: tbd07:24
bdxfrobware: yea, I've been following along ... entirely. thx07:24
frobwarebdx: we haven't done a CI run yet with this flag, so YMMV07:25
bdxI have a lab up with juju2.0 + MAAS2.0 + the flag07:25
bdxjust not a build of master :-(07:25
bdxI'll give tip a build and see what gives07:27
frobwarebdx: I would initially try the sinlge NIC setup, just a hint. :)07:28
bdxfrobware: thanks, by single nic, you mean enlisted maas nodes only have 1 nic ?07:29
frobwarebdx: yep. though things could have moved on since wed/thu when I was last tracking this.07:30
bdxthat way no one has to decide what interface to put the bridge  on07:30
bdx?07:30
frobwarebdx: we bridge all interfaces now07:31
frobwarebdx: I just recall see a comment from voidspace that he was trying/testing single NIC setup to begin with07:32
bdxnice, ok07:32
frobwarebdx: single NIC, with multiple VLANs, all end up with their own bridge: http://pastebin.ubuntu.com/16123783/07:33
rogpeppe1anyone know why aws-china is a separate cloud rather than just an aws region ?08:00
mattywrogpeppe1, something something firewalls I think08:06
mattywwallyworld, you around mate?08:06
voidspacefrobware: bdx: I was testing containers deployed on machines with a single nic first08:08
voidspacefrobware: bdx: that worked and then we got multiple nics working too08:08
frobwarevoidspace: ooh nice08:09
voidspacefrobware: maas2 work is now feature complete - as soon as we get it passing CI we'll remove the flag08:09
voidspacefrobware: so still *technically* experimental, but hopefully not for much longer08:09
anastasiamac_rogpeppe1: different clouds for aws, aws-china and aws-gov because they require different authentication types or secret information for different regions08:13
rogpeppe1anastasiamac_: ok, so you can't use the same credentials across those clouds? that makes sense.08:14
voidspacefrobware: we're still getting maas schema failures though (json doesn't match what we expect)08:15
voidspacefrobware: this is the latest CI run: http://reports.vapour.ws/releases/3934/job/maas-2_0-deploy-xenial-amd64/attempt/508:15
voidspacefrobware: I'll fix that08:15
anastasiamac_rogpeppe1: essentially, yes... quoting the spec for more details :D..."There can only be one default credential for a given cloud. If that credential does not work on a particular region then you will need to specify the credential used to deploy in that region. There can only be one default region for a cloud."08:17
rogpeppe1anastasiamac_: thanks08:17
anastasiamac_rogpeppe1: i hope this helps \o/08:18
anastasiamac_rogpeppe1: there is a similar separation for azure :D08:18
voidspacefrobware: standup09:01
frobwarevoidspace: arriving... now!09:01
voidspacefrobware: babbageclunk: https://github.com/juju/gomaasapi/pull/5209:06
rogpeppe1fwereade: ping10:20
Mo0Ohello10:43
babbageclunkfrobware: maas is up and running, but my nodes won't pxe boot to enlist. I've turned on DHCP and downloaded a boot image. What have I missed?10:54
frobwarebabbageclunk: boot order on VM nodes?10:55
babbageclunkfrobware: the nodes have their nics first in the boot order.10:55
babbageclunkfrobware: ha ha10:55
frobwarebabbageclunk: images imported?10:56
frobwarebabbageclunk: eth0 configured to be managed (and dns and dchp)?10:56
babbageclunkyup - do I still need a 14.04 image imported?10:56
frobwarebabbageclunk: you have 16.04?10:56
babbageclunkfrobware: yup10:57
frobwarebabbageclunk: what does commisionning node say in your maas settings? (the drop-down in the UI)10:57
babbageclunkfrobware: xenial10:57
frobwarebabbageclunk: HO?10:58
babbageclunkyes10:58
frobwarebabbageclunk: https://plus.google.com/hangouts/_/canonical.com/juju-sapphire10:59
fwereaderogpeppe1, pong11:02
rogpeppe1fwereade: np, IS just saw an occurrence of this bug after upgrading to 1.25. fixed by a manual JS script https://bugs.launchpad.net/juju-core/+bug/1516989 (but not quite sure why the upgrade migration hadn't worked)11:03
mupBug #1516989: juju status <service_name> broken <canonical-bootstack> <sts> <juju-core:Invalid by waigani> <juju-core 1.25:Fix Released by waigani> <juju-core (Ubuntu):Confirmed> <https://launchpad.net/bugs/1516989>11:03
rogpeppe1fwereade: so, incident fixed, but not sure why it happened.11:04
fwereaderogpeppe1, huh11:06
fwereaderogpeppe1, sorry, I have only the vaguest memories of writing that script, and I don't recall tracking the fix in 1.2511:06
rogpeppe1fwereade: np :)11:07
rogpeppe1fwereade: just thought you might be able to provide some insight at the time11:07
fwereaderogpeppe1, PR3860 went into 1.25 on dec 1, it seems -- is it remotely possible this was somehow an upgrade to an unfixed 1.25?11:08
rogpeppe1fwereade: i don't think so; let me check11:08
rogpeppe1fwereade: the version was 1.25.511:09
fwereaderogpeppe1, yeah, that should be fine11:09
fwereaderogpeppe1, no further insights here, I'm afraid :(11:10
rogpeppe1fwereade: thanks11:10
frobwarebabbageclunk: if you do try my scripts you want to invoke as ... $ VIRT_RAM=1024 ./add-node ...11:25
frobwarebabbageclunk: because the default is 4GB, which may cause you a few problems11:25
voidspacebabbageclunk: how's your branch doing?11:39
babbageclunkvoidspace: finally got maas set up again, just setting up the private space node so I can have some constraints.11:40
voidspacebabbageclunk: ok11:40
babbageclunkvoidspace: it occurs to me I could test the change with storage constraints rather than space ones, sorry.11:42
babbageclunkvoidspace: did you try adding a machine without a constraint?11:42
voidspacebabbageclunk: I've added machines, yes11:43
babbageclunkvoidspace: so should I just kick off the merge anyway? Not making anything worse if it doesn't work (although I think it will) and then unblocking you.11:44
voidspacebabbageclunk: go for it11:44
voidspacebabbageclunk: you might still need to JFDI it11:44
babbageclunkvoidspace: oh, also dimiter made some comments that I should address.11:45
voidspacebabbageclunk: cool11:46
voidspacebabbageclunk: let me know when it's landed please11:47
babbageclunkvoidspace: wilco11:50
babbageclunkvoidspace: will $$jfdi$$ still run the tests?11:51
frobwarebabbageclunk: yes11:52
voidspace^^^^11:52
babbageclunkvoidspace, frobware: so it just bypasses blocking? Or it runs them and then merges regardless of the result?11:53
voidspacebabbageclunk: bypasses blocking11:53
babbageclunkvoidspace: thanks11:53
babbageclunkvoidspace: building now, also successfully added a machine with a space constraint.12:00
voidspacebabbageclunk: awesome!12:01
voidspacegreat news12:01
dooferladfrobware: could you take a look at http://reviews.vapour.ws/r/4741/ please? It is a rather important fix for the last bridge script mods.12:24
frobwaredooferlad: looking12:24
* dooferlad goes for lunch12:25
* babbageclunk is lunching12:27
frobwaredooferlad: reviewed12:43
mupBug #1576670 opened: undefined: tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 <build> <centos> <ci> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576670>12:47
mupBug #1576510 changed: Race in macaroon-bakery <ci> <race-condition> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1576510>13:17
mupBug #1576674 opened: 2.0 beta6: only able to access LXD containers (on maas deployed host) from the maas network <oil> <juju-core:New> <https://launchpad.net/bugs/1576674>13:17
mupBug #1576686 opened: bundle repository/bundles-lxd.yaml is invalid <juju-core:Triaged> <https://launchpad.net/bugs/1576686>13:41
babbageclunkvoidspace, frobware: is there any way to get maas-cli 2 installed on wily?13:43
babbageclunkexperimental3 only has xenial.13:43
frobwarebabbageclunk: is this because your laptop is running wily?13:43
babbageclunkfrobware: yeah13:43
babbageclunkfrobware: can't try to use/adapt your scripts without it13:44
mupBug #1576695 opened: Deployer cannot talk to Juju2 (on maas2) because :tlsv1 alert protocol version <ci> <deployer> <maas-provider> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1576695>13:53
dooferladfrobware: updated http://reviews.vapour.ws/r/4741/14:01
mupBug #1576700 opened: Misleading "Waiting for agent initialization to finish" message <ci> <status> <juju-core:New> <https://launchpad.net/bugs/1576700>14:11
mupBug #1576704 opened: MigrationExportSuite.TestUnits unequal results <ci> <go1.6> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576704>14:11
mupBug #1576705 opened: cloudImageMetadataSuite.TestSaveDiffMetadataConcurrentlyAndOrderByDateCreated wrong order <ci> <go1.6> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576705>14:11
alexisbbabbageclunk, voidspace just want to make sure you guys saw this from mgz : https://bugs.launchpad.net/juju-core/+bug/157636814:19
mupBug #1576368: blockdevice 2.0 schema check failed: model: expected string, got nothing <ci> <deploy> <maas-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1576368>14:19
babbageclunkalexisb: yup, voidspace is on it - https://github.com/juju/gomaasapi/pull/5214:21
frobwaredooferlad: it's not clear to me why we don't know all the routing information as and when we create the new stanzas and simply accumulate them. if we accumulate then it also means those bridging functions have no side effects.14:27
dooferladfrobware: the only function that has a side effect is bridge_now.14:28
dooferladfrobware: the most reliable thing to do is save routes for an interface, bridge the interface, restore routes that vanished.14:29
frobwaredooferlad: iff, you first successfully write the new file14:29
frobwaredooferlad: hence my preferred order. transform, successfully write new file, bridge-le-world.14:30
dooferladfrobware: you meen you want to not do anything unelss you have generated a new e/n/i?14:30
dooferladfrobware: I did just update the rev with a comment about that14:30
frobwaredooferlad: it's moot. but if you can't write the file all bets sjould be off.14:30
dooferladfrobware: seems reasonable14:31
dooferladfrobware: so I just suggested not running the bridge_now loop until after the if not args.activate: ... exit(0) code14:32
frobwaredooferlad: that would re-read the (new) eni file?14:32
dooferladfrobware: no14:32
frobwaredooferlad: so what happesn before your exemplary if not activate?14:32
dooferladfrobware: easiest if I just write the code. Will be two minutes.14:33
frobware:)14:33
alexisbbabbageclunk, voidspace awesome14:33
babbageclunkalexisb: just doing the juju diff to update the dependency now.14:34
dooferladfrobware: http://reviews.vapour.ws/r/4741/ updated14:39
mupBug #1576728 opened: ConnectSuite.TestLocalConnectError: windows cannot connect to local lxd server <ci> <go1.6> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576728>14:45
voidspacealexisb: we're playing whackamole with maas schema oddities :-/14:47
voidspacealexisb: we'll get there though14:47
babbageclunkvoidspace, frobware: review please? http://reviews.vapour.ws/r/4742/14:47
babbageclunkvoidspace: figured I'd just do this bit14:47
voidspacebabbageclunk: you beat me to it!14:48
babbageclunkvoidspace: :)14:48
voidspacebabbageclunk: thanks, LGTM14:48
alexisbvoidspace, understood, thank you for the quick turn around it makes it so the CI runs can continue to move forward14:49
babbageclunkvoidspace: I'm going to quickly make a list of the attributes we use and see if roaksoax or anyone else in #maas can point out others that are optional14:49
voidspacebabbageclunk: it's any that appear in the gomaasapi schema - so depending on what you mean by "we use" it's not just the ones we use14:50
voidspacebabbageclunk: but yeah, catching these ahead of time would be good14:51
babbageclunkvoidspace: yeah, that's what I meant - frustratingly, in some of these cases they're not even things we use14:51
babbageclunkvoidspace: we're still good to JFDI things, right?14:52
babbageclunkvoidspace: (optional or nullable, I guess)14:53
voidspacebabbageclunk: yep14:53
voidspacebabbageclunk: and yep14:53
babbageclunkvoidspace: huh. Exact spelling of JFDI please?14:56
mgzI feel like our blocking process has not been working the past few weeks14:57
mgzbabbageclunk: how about we just use fixes-BUGNUM instead14:57
katcoericsnow: redir: hey i'm going to miss this morning's standup... guy who is refinishing our floor just showed up14:57
mgzbabbageclunk: I marked bug 1576368 blocking so don't jfdi, use magic string 'fixes-1576368'14:58
mupBug #1576368: blockdevice 2.0 schema check failed: model: expected string, got nothing <blocker> <ci> <deploy> <maas-provider> <juju-core:Triaged by 2-xtian> <https://launchpad.net/bugs/1576368>14:58
babbageclunkmgz: Ok, thanks14:58
ericsnowkatco: k, good luck14:58
redir_katco: standup?15:10
redir_katco: OK. Missed the scrollback, eric passed it on.15:15
perrito666wallyworld: still working?15:33
mupBug #1576743 opened: juju agent 1.25.5 crashes on ubuntu 14.04 in an private openstack cloud <juju-core:New> <https://launchpad.net/bugs/1576743>15:33
mupBug #1576750 opened: juju2 usability: many options have to be specified for every bootstrap <landscape> <juju-core:New> <https://launchpad.net/bugs/1576750>15:33
mupBug #1576756 opened: cannot shut down controllers running other people's models <juju-core:New> <https://launchpad.net/bugs/1576756>15:33
rogpeppe1anyone know how i can "juju switch" to a different account?15:37
cherylj rogpeppe1 you mean be a different user?15:37
rogpeppe1cherylj: yes15:37
rogpeppe1cherylj: i'm getting "ERROR getting controller environ: getting bootstrap config from API: permission denied (unauthorized access)" when i try to destroy a controller15:38
rogpeppe1cherylj: and i think the problem is probably that i'm using the wrong account15:38
cheryljthere was a switch-user, I thought.   hmm  maybe that was removed15:38
mupBug #1576756 changed: cannot shut down controllers running other people's models <juju-core:New> <https://launchpad.net/bugs/1576756>15:39
rogpeppe1cherylj: i see two users for the controller in accounts.yaml15:39
rogpeppe1cherylj: but the current user is not the admin user15:39
cheryljrogpeppe1: maybe logout / login?15:40
rogpeppe1cherylj: i don't think that's what i want15:41
rogpeppe1cherylj: looks like logout will delete my password15:41
rogpeppe1http://paste.ubuntu.com/16127920/15:41
frobwaredooferlad, voidspace, babbageclunk: PTAL @ http://reviews.vapour.ws/r/4743/15:43
cheryljrogpeppe1: sounds like you want to change your password first :)15:43
cheryljI've never done that before, so I can't tell you what will happen15:43
rogpeppe1cherylj: tbh i don't want to clear the credentials from the client15:44
rogpeppe1cherylj: i ended up just manually editing the accounts.yaml file15:44
cheryljthat works too15:44
rogpeppe1cherylj: thanks for the help15:44
dooferladfrobware: looking15:49
dooferladfrobware: http://reviews.vapour.ws/r/4741/ is updated again15:50
dooferladfrobware: +115:52
babbageclunkvoidspace: this is the schema as we have it now (munged for a bit more clarity): http://paste.ubuntu.com/16127709/15:55
babbageclunkroaksoax is going to get someone to go through it.15:55
frobwaredooferlad: thanks. I raised the question about `ip route' and IPv6 - what happens when the iface has an IPv6 route in addition to 4?16:05
frobwarecherylj: master is blocked... or not?16:29
cheryljfrobware: it is blocked16:29
cheryljthere are new failures16:29
cheryljand they need to be addressed16:30
frobwarecherylj: ok, see you next week. :)16:30
cheryljhaha16:30
cheryljfrobware: it's going to be hot and humid.  Hope you're prepared16:30
frobwarecherylj: it's trying to snow here... ???16:31
cheryljhttps://weather.com/weather/tenday/l/78757:4:US16:31
frobwareooh, some of the numbers start at 816:31
cheryljhigh of 30C Friday16:32
frobwarecherylj: I'll go with old money; 80 sounds way higher16:32
voidspacebabbageclunk: awesome16:32
voidspacebabbageclunk: nice work16:32
cheryljhehe16:33
mupBug #1576778 opened: juju2 tab-completion for sub commands is broken in beta6 <landscape> <usability> <juju-core:New> <juju (Ubuntu):New> <https://launchpad.net/bugs/1576778>16:42
=== urulama is now known as urulama|afk
alexisbkatco, ping16:50
katcoalexisb: pong16:51
alexisbheya katco , we need some help unblocking master and cherylj is currently occupied with sprint tasks for next week16:51
alexisbkatco, can you please rally the troupes around these 1.6 failures:16:52
alexisbhttps://goo.gl/0jYcTy16:52
katcoalexisb: sure thing16:52
alexisbthank you, I am here if you have qs16:52
alexisbthough I will be changing locations here in a minute16:52
katcoalexisb: thanks16:52
katcoalexisb: for bug 1576670. what substrates are still on 1.2? i thought we had 1.6 backported to trusty and we were now fully on 1.6?16:54
mupBug #1576670: undefined: tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 <blocker> <build> <centos> <ci> <go1.2> <go1.6> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576670>16:54
alexisb1.616:54
alexisbno more 1.216:54
cheryljkatco: that bug started the discussion this morning16:54
cheryljof whether or not we still use go 1.2 for our windows / centos unit tests16:55
cheryljbecause those unit tests fail under go 1.616:55
katcocherylj: if we do, that's a bug against CI i think? windows can use 1.6 without any problem. centos i'm less sure of. what do you think?16:55
cheryljso if we can get the windows / centos tests working with go 1.6, that bug will be invalid16:55
mgzkatco: 'without any problem' is the issue16:55
cheryljyeah16:55
katcocherylj: mgz: ah i see... chicken and egg problem? there are some centos/windows tests failing on 1.6?16:56
cheryljkatco: exactly :)16:56
cheryljso we're asking for help to fix those other tests16:56
alexisb1.6 all the way!16:56
katcocherylj: read my mind: i think we should push forward. mark bug 1576670 as invalid and flag failing tests as blockers16:56
mupBug #1576670: undefined: tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 <blocker> <build> <centos> <ci> <go1.2> <go1.6> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576670>16:57
mgz200% compile time slowdowns all the way!16:57
cheryljhuzzah!16:57
katcomgz: that has been addressed in tip :)16:57
katcocherylj: so are you fine with me marking this as invalid? can you raise the other test failures as blocking bugs?16:57
cheryljkatco: already did16:58
cherylj:)16:58
katcocherylj: ty kindly :)16:58
alexisbkatco, what we need from the core team is to fix the test failures16:58
katcoalexisb: agreed16:58
mgzkatco: yeah, it's now just a 100% slowdown16:58
katcomgz: native go compiler > c compiler, and slowdowns will be addressed over time :) it's a good thing overall!16:59
katcoericsnow: redir: redir_: what are you 2 up to?16:59
katcoperrito666: what are you working on?16:59
mgzkatco: I'm not being too serious :)16:59
cheryljokay, I'm going to get some lunch.  bbiab16:59
ericsnowkatco: syslog17:00
katcoericsnow: the 1-pager?17:00
alexisbfolks these blockers are top priority so if you have a nice stopping point17:00
ericsnowkatco: yep17:00
redirkatco: The LTS update fix and ian's feedback.17:00
katcoericsnow: sounds like that's getting trumped17:00
katcoredir: k, carry on17:00
* redir nods17:00
katcoredir: hi btw17:01
redirmorning katco :)17:01
* alexisb changes locations brb17:01
katcoericsnow: you have your pick:17:01
katcohttps://bugs.launchpad.net/juju-core/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.st17:01
* katco now understands why alexisb put that in a url shortener17:01
ericsnowkatco: k17:02
katcoericsnow: lmk which one you pick up so i can pick something else17:02
ericsnowkatco: it'll be a few minutes before I can get to a stopping point17:04
katcoericsnow: np at all, same here17:04
mupBug #1576670 changed: undefined: tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 <blocker> <build> <centos> <ci> <go1.2> <go1.6> <regression> <test-failure> <unit-tests> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576670>17:12
mupBug #1576778 changed: juju2 tab-completion for sub commands is broken in beta6 <landscape> <usability> <juju-core:New> <juju (Ubuntu):New> <https://launchpad.net/bugs/1576778>18:03
mupBug #1576318 opened: juju register not clear that you're creating a new password <apport-bug> <i386> <usability> <xenial> <juju-core:Triaged> <juju-core (Ubuntu):New> <https://launchpad.net/bugs/1576318>18:03
mupBug #1576805 opened: Juju controllers set the rsyslog NetstreamDriver affecting all subsequent rsyslog configuration <juju-core:New> <https://launchpad.net/bugs/1576805>18:03
katcocherylj: hey, build bot is wrong? https://github.com/juju/juju/pull/531818:35
* cherylj looks18:36
mgzkatco: it's being fussy about high vs critical18:36
katcomgz: ah ok. jfdi?18:36
mgzgo again, I edited the bug18:36
katcomgz: ta18:36
mgzuse $$fixes-NNN$$18:36
katcomgz: sanity check: for bug 1576728, i'm just not going to run any tests for lxd on windows19:22
mupBug #1576728: ConnectSuite.TestLocalConnectError: windows cannot connect to local lxd server <blocker> <ci> <go1.6> <regression> <test-failure> <unit-tests> <windows> <juju-core:In Progress by cox-katherine-e> <https://launchpad.net/bugs/1576728>19:22
katcocherylj: ^^19:22
cheryljkatco: sounds good.  You may also need to include a skip for centos too19:23
cheryljso, I guess if ! ubuntu19:23
katcomgz: cherylj: when/if lxd is supported on windows, this code will have to be modified to be platform independent anyway. the error instructions don't make sense for anything but ubuntu19:23
katcocherylj: yeah good point19:24
mgzkatco: I'm not sure on that,19:24
mgzwe have some general isolation test failures with lxd19:24
cheryljbut yeah, we'd want to move to a mocked out lxd client19:24
katcomgz: i don't follow19:24
katcocherylj: not only that, but these tests would be testing code that would never be exercised on those platforms19:25
cheryljthat's true19:25
mgzkatco: I've not looked in detail, but we have a bunch of tests that aren't very unity and actually try to talk to the local lxd19:26
mgzwhich should really just not do that at all, rather than be skipped on some platforms19:26
katcomgz: oh, yes. i completely agree there. just trying to remove the blocker pedantically19:26
katcomgz: not really prepared to do major surgery atm :)19:26
mgzkatco: eg bug 1564511 - which is a different suite19:27
mupBug #1564511: cmd/jujud/reboot tests fail with lxd container running <ci> <regression> <s390x> <tech-debt> <test-failure> <unit-tests> <xenial> <juju-core:Triaged> <https://launchpad.net/bugs/1564511>19:27
katcomgz: yeah. that's just so awful.19:27
mgzso, I'm not anti skipping on windows where the test otherwise makes sense but isn't code we ever run on windows19:27
katcomgz: i'll lay down robustness in layers. i'll address platform robustness in this patch19:28
redirkatco: ericsnow got a minute to help me understan PatchValue?20:27
ericsnowredir: sure20:28
katcoredir: sure, moonstone?20:28
redirsure20:28
ericsnowmgz: do you know of any problems we have had (testing or otherwise) due to Windows's long clock tick (15.6 ms by default)?20:33
mupBug #1571783 changed: Windows unit tests cannot setup under go 1.6 <ci> <go1.6> <jujuqa> <regression> <test-failure> <unit-tests> <windows> <juju-core:Fix Released by hduran-8> <https://launchpad.net/bugs/1571783>20:33
mgzericsnow: not with juju I think20:36
ericsnowmgz: k20:36
ericsnowmgz: I'm thinking that the bug I'm working on may be due to that long clock tick20:37
mgzericsnow: it's possible, given we have a number of rather timing dependent tests20:40
mupBug #1576851 opened: juju debug-log -i unit-rabbitmq-server-0 is unfriendly <juju-core:New> <https://launchpad.net/bugs/1576851>20:43
alexisbanastasiamac, are you there21:32
anastasiamacalexisb: i am21:32
alexisbheya21:33
anastasiamacalexisb: do u have a sec?21:33
alexisbI do21:33
anastasiamaccould we go to our 1:1?21:33
alexisbyep21:33
mupBug #1576873 opened: Juju2 cannot deploy centos or windows workloads on maas 1.9 <blocker> <centos> <ci> <maas-provider> <regression> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1576873>22:19
mupBug #1576874 opened: restore-backup never completes <backup-restore> <blocker> <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1576874>22:19
perrito666alexisb: still here?22:28
redirthanks katco ericsnow that was very useful22:29
alexisbperrito666, yes22:29
alexisbwuz up?22:29
perrito666alexisb: priv22:29
katcoredir: np22:29
ericsnowkatco: you have a sec for a quick review?  http://reviews.vapour.ws/r/4746/22:45
katcoericsnow: sure tal22:45
katcoericsnow: do we have a way to test this empirically?22:46
ericsnowkatco: test what exactly?22:46
katcoericsnow: that the fix for your bug works22:46
ericsnowkatco: I suppose I could spin up my KVM, update it, etc. and then run that test and hope my local timing isn't slow anyway...22:47
ericsnowkatco: but running in KVM might mask the issue22:48
ericsnowprobably22:48
katcoericsnow: i think it'd be enough to land the fix and coordinate with ci folks22:48
katcoericsnow: to ensure it's really solved22:48
ericsnowkatco: that's what I was thinking22:49
katcoericsnow: shipit22:49
ericsnowkatco: thanks22:49
ericsnowmgz: are all the go-1.6, windows blocks supposed to be critical?22:52
ericsnowmgz: trying to merge a fix22:52
katcoericsnow: flag your bug as critical to get it through22:52
ericsnowkatco: done22:53
perrito666:95023:23
perrito666meh, wrong window23:23

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!