/srv/irclogs.ubuntu.com/2016/04/07/#juju-dev.txt

cheryljmenn0: I can take a look later (after I put the kid to bed)00:12
cheryljtych0 - running lxd init; systemctl restart lxd.service worked to get the lxd provider to bootstrap00:13
menn0cherylj: thank you (no rush)00:13
cheryljtych0: will juju need to do that when provisioning lxd containers (using deploy --to lxd)?00:14
redircherylj: is there a kanban board with the helpdocs updates on it?00:46
anastasiamacredir: i think bug squad is still it..00:47
anastasiamacredir: i'll pm u :D00:47
cheryljredir: I haven't been keeping up to date on all the helpdocs bugs and their comments01:03
cheryljredir: might be a good idea to change the titles of them once they've been acked and are ready to merge....  makes it easy to quickly search for them in lp01:03
redircherylj: anastasiamac got me to the board. So I can move cards appropriately.01:04
cheryljredir: they're not all there, btw01:04
cheryljfeel free to add if you'd like01:04
redircherylj: will do when I get to one in lp that isn't on the board.01:05
cheryljredir: if you're interested, bug 1566237 is an easy non-help docs bug01:06
mupBug #1566237: juju ssh doesn't work with multiple models <juju-core:Triaged> <https://launchpad.net/bugs/1566237>01:06
redircherylj: k01:09
mupBug #1567161 opened: juju2 beta3, cannot download charm, failed to download, 400 <landscape> <juju-core:New> <https://launchpad.net/bugs/1567161>01:21
mupBug #1566843 changed: juju add-credential should set default when there is only one credential (when using interactive mode) <docteam> <juju-release-support> <usability> <juju-core:Invalid by axwalk> <https://launchpad.net/bugs/1566843>01:33
mupBug #1567169 opened: juju deploy bundle does not honor existing machine definitions <conjure> <juju-core:New> <https://launchpad.net/bugs/1567169>01:33
redirso if I want to run one test a la `go test . -name TestThatIWantToRun` do I need to do something special for gocheck to honor that?01:57
anastasiamacaxw: I wonder if TerminateInstances call should be re-written to use retry too instead of shortAttempt...01:57
axwanastasiamac: probably01:59
anastasiamacredir: go test -gocheck.v -gocheck.f TestYouWant01:59
anastasiamacredir: which I think is a regular expression, so u can have both 'myTestSuite' or 'individualTestName' or whatever matching...02:00
redirthank you02:00
redirif it is a pass through then it should be a regexp02:00
redirand thank you02:02
anastasiamacu r welcome \o/02:02
mupBug #1567170 opened: Disallow upgrading with --upload-tools for hosted models <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1567170>02:03
mupBug #1567175 opened: Service start is not reliable with any network initialization issues <juju-core:New> <https://launchpad.net/bugs/1567175>02:15
=== redir is now known as redir_eod
redir_eodsee you all tomorrow sometime02:22
thumpero/ redir_eod02:23
=== redir_eod is now known as redir_afk
mupBug #1567179 opened: ec2 terminate instances should retry with exponential delay on err <juju-core:New> <https://launchpad.net/bugs/1567179>02:36
mupBug #1567182 opened: Fallback to installing mongo 2.4 if no 3.2 doesn't work <mongodb> <juju-core:Triaged by hduran-8> <https://launchpad.net/bugs/1567182>02:36
davecheneymenn0: while i'm waiting for other thigns02:39
davecheneyi started a set of PR's about testing that we talked about02:39
davecheneyhttps://github.com/juju/juju/pull/502302:39
=== natefinch-afk is now known as natefinch
menn0davecheney: taking a look02:40
menn0davecheney: very nice. land it!02:43
cheryljmenn0: that lxd failure has been hit on other branches, so it's not unique to model migrations02:45
menn0cherylj: ok good02:46
davecheneymenn0: thanks03:00
davecheneylots more where that came from03:00
cheryljoh god03:04
cheryljI hacked distro-info to return xenial as the lts03:04
cheryljto see what fails in unit tests03:04
cheryljso many failures03:04
natefinchwas gonna say, maybe you should catalog what passes.  Might be faster.03:05
cheryljlol03:05
mupBug #1567169 changed: juju deploy bundle does not honor existing machine definitions <conjure> <juju-core:Invalid> <https://launchpad.net/bugs/1567169>03:06
* cherylj cries03:08
cheryljwe're going to be screwed in 2 weeks03:08
cheryljbetter to know now?03:08
rick_h_cherylj: more time for wine between now and then?03:08
cheryljrick_h_: I wish.  We gotta fix these or else we will not be able to release anything once xenial is our official lts03:09
rick_h_cherylj: understand, it's 11pm have to go light hearted03:10
natefinchcherylj: really really smart to check that.... sorry the results weren't more in our favor.03:10
mupBug #1567169 opened: juju deploy bundle does not honor existing machine definitions <conjure> <juju-core:Invalid> <https://launchpad.net/bugs/1567169>03:12
cherylj92 test failures so far03:13
cheryljwell, to be fair, there are some lxd failures because I don't have lxd on my laptop03:13
cheryljso maybe 8003:13
natefinchman, I really hate that our code cares what series exist and relies on that data never to change and never to be out of date.03:14
cherylj2 years is forever, man03:15
cheryljtrusty 4 lyfe03:15
natefinchcherylj: 2 years my butt. This happens every 6 months when a new series comes out and also as a bonus, whenever we get a new windows or OSX release... I suppose probably centos as well.03:15
mupBug #1567169 changed: juju deploy bundle does not honor existing machine definitions <conjure> <juju-core:Invalid> <https://launchpad.net/bugs/1567169>03:16
cheryljhaha03:16
cheryljand what unit test run would be complete without a test timeout!!03:17
cheryljI wonder how many would pass if we set testing.FakeDefaultSeries = xenial03:19
cheryljor, what we should do is testing.FakeDefaultSeries = actualDefaultSeries03:19
cheryljand people can patch03:19
cheryljif they need to03:19
cheryljanyway that's a problem for tomorrow Cheryl03:20
rick_h_night cherylj03:20
natefinchpoor tomorrow Cheryl03:20
cheryljyeah, yesterday Cheryl is terrible03:20
cherylj:)03:20
natefinchlol... yeah, I hate yesterday Nate. That guy's an idiot.03:20
davecheneyfunc (s *BaseRepoSuite) SetUpSuite(c *gc.C)    {}03:44
davecheneyfunc (s *BaseRepoSuite) TearDownSuite(c *gc.C) {}03:44
davecheneygreat03:44
davecheneythat's just great03:44
anastasiamacdavecheney: we should have this on the t-shirt "juju, just great"03:46
menn0thumper: easy review pls http://reviews.vapour.ws/r/4466/03:51
menn0davecheney: http://reviews.vapour.ws/r/4466/ pls04:03
davecheneylookin'04:07
davecheneyLGTM, mostly harmless04:07
menn0davecheney: ta04:12
davecheneymenn0: https://github.com/juju/juju/pull/502504:13
davecheneymore of the same04:13
natefinchahhh... damn macaroons04:16
=== thumper is now known as thumper-afk
mupBug #1567228 opened: "juju destroy-controller" can leak hosted models <juju-core:Triaged> <https://launchpad.net/bugs/1567228>05:25
frobwaredimitern, if you need an immediate fix for the broken /e/n/i try this http://pastebin.ubuntu.com/15653945/07:09
dimiternfrobware, cheers, I'll try this on the next test I'm doing shortly07:10
frobwaredimitern, it's obviously not the complete story... but is in essence the fix07:11
frobwaredimitern, any joy with the case where the first add-machine lxd:0 fails?07:12
dimiternfrobware, sorry, I had to step afk07:40
dimiternfrobware, well, add-machine lxd:0 only works (post-bootstrap), if you first juju switch admin, is that what you mean?07:41
dimiternfrobware, otherwise it says no such machine 007:41
dimiternbut apart from that, yes - it still fails, still digging..07:41
frobwaredimitern, nope. if you switch to :admin and add-machine lxd:0 the first one fails to work until the agent is restarted on the node hosting the container.07:42
frobwaredimitern, I think we're saying the same things (give or take the model switch)07:44
dimiternfrobware, yeah07:44
dimiternfrobware, :) I was about to write the same thing07:44
=== thumper-afk is now known as thumper
thumperfrobware, dimitern: why is there no standup listed for today?08:58
thumpervoidspace, babbageclunk: care to hangout to discuss maas2?08:59
thumperI was awaiting a standup that doesn't appear to be scheduled09:00
voidspacethumper: team meeting today at 11am instead09:03
voidspacethumper: but sure09:03
thumperI'm not going to hang around for that09:03
voidspacethumper: that's our loss09:03
thumperow, ow, ow.... cramp09:04
* thumper stretches09:04
voidspacedimitern: we could use you to talk to babbageclunk and thumper about the link layer model and what information we need from maas09:04
* thumper is in the sapphire normal standup hangout09:04
voidspacedimitern: currently the maas 1 code uses stuff like link.Parent() and IPAddresses that don't seem to exist in MAAS 209:04
thumperwell it does09:05
voidspacethumper: omw09:05
thumper..09:05
thumperhmm09:05
dimiternoops sorry guys, omw09:09
dimiternvoidspace, babbageclunk, thumper, standup HO?09:10
voidspacedimitern: we're there09:10
voidspacedimitern: hello09:10
dimiternbabbageclunk, dooferlad, frobware, join us guys for standup?09:21
dooferladdimitern: sure. URL?09:21
babbageclunkComing now09:21
mupBug #1567296 opened: Plugin API fails with multiple juju binaries <juju-core:New> <https://launchpad.net/bugs/1567296>09:34
rogpeppeanyone know what the best way to shut down a juju lxd provider controller is when juju won't let me (probably because i started it with an old juju version) ?10:02
davecheneywhoopse, that was rather aburpt10:12
davecheneysorry about that10:12
mgzrogpeppe: the best way has been build that old env and kill-controller with that10:17
rogpeppemgz: unfortunately i've no idea what version i used to bootstrap the old env10:17
rogpeppemgz: i think i need to go behind the scenes10:17
rogpeppemgz: but my naive approach ("lxd shutdown") doesn't seem to have got me to a good place10:18
rogpeppemgz: i just removed $HOME/.local/share/juju and now i'm getting "ERROR invalid config: no addresses match"10:19
rogpeppemgz: when i try "juju bootstrap local lxd"10:19
rogpeppemgz: :-(10:19
rogpeppemgz: AFAIK i haven't *got* a config10:19
mgzrogpeppe: yeah, it's a joy10:22
rogpeppemgz: i've just proposed https://github.com/juju/cmd/pull/35 to make it easier to debug that kind of error message10:23
rogpeppemgz: that makes the error somewhat more informative... 2016-04-07 10:25:39 DEBUG cmd supercommand.go:449 error details: [{github.com/juju/juju/cmd/juju/commands/bootstrap.go:406: } {github.com/juju/juju/environs/open.go:104: } {github.com/juju/juju/environs/open.go:175: } {github.com/juju/juju/provider/lxd/provider.go:44: } {github.com/juju/juju/provider/lxd/environ.go:43: invalid config} {github.com/juju/juju/provider/lxd/config.go:10:26
rogpeppe182: } {github.com/juju/juju/provider/lxd/config.go:255: } {github.com/juju/juju/tools/lxdclient/config.go:65: } {github.com/juju/juju/tools/lxdclient/remote.go:199: } {no addresses match}]10:26
mgzdidn't that used to have prettier formatting?10:31
voidspacefrobware: dimitern: babbageclunk: doing a merge of master onto maas210:36
voidspacefrobware: dimitern: babbageclunk: some fixing to do as anastasiamac fixed a maas bug that changes the maasInstance interface10:36
babbageclunkvoidspace: nothing too nasty I hope?10:37
rogpeppemgz: possible. i don't care much :)10:37
voidspacebabbageclunk: maasInstance.zone() now returns an error - so the interface definition and our maas2Instance implementation need updating10:38
rogpeppemgz: FWIW the error is because juju's looking for netif lxdbr0 but my machine only has lxcbr010:38
voidspacebabbageclunk: plus there are a bunch of new tests that do a type cast to maasInstance that should now be maas1Instance10:38
voidspacebabbageclunk: plus another bugfix by davecheney in maas storage10:38
voidspacebabbageclunk: nothing too nasty, no - easy enough10:38
perrito666morning10:52
rogpeppemgz: ha, i upgraded lxd and now there *is* an lxdbr0 interface but it doesn't have any ipv4 addresses so i get the same failure10:55
rogpeppeanyone used the lxd provider with the latest juju, by any chance?10:56
TheMuemorning perrito66610:56
rogpeppevoidspace: do you know anything about this issue, by any chance?10:56
rick_h_rogpeppe: there's an email thread.on the juju list around this i think11:04
rick_h_rogpeppe: from yesterday11:04
rogpepperick_h_: yeah, just looking for it11:04
rogpepperick_h_: found11:05
rogpepperick_h_: i don't think it helps me though11:07
rogpepperick_h_: there's no "lxd-bridge" service on my machine AFAICS.11:08
voidspacerogpeppe: no, afraid not11:08
voidspacerogpeppe: I just saw an upgrade do the same thing (well - change the bridge name) on my server too though11:09
voidspacerogpeppe: jam might know if he's still around11:09
rogpeppevoidspace: yeah, it seems to break everything11:09
voidspace:-(11:09
voidspaceor maybe frobware I know he's done a lot of work with lxd11:09
rogpeppevoidspace: i guess i need to create an ipv4 address on the bridge somehow11:09
voidspacerogpeppe: it should *get* an ipv4 address11:09
voidspaceso it's odd11:10
rogpeppevoidspace: it really seems that it shouldn't *need* an ipv4 address anyway11:10
rogpeppevoidspace: aren't we supposed to work with ipv6?11:10
voidspacemaybe look in /etc/network/interfaces and see how the bridge is defined11:10
rogpeppevoidspace: good idea11:10
voidspacerogpeppe: I'm not convinced I think we've *tried* to be compatible but not been thorough11:10
rogpeppevoidspace: FWIW ifconfig shows this: http://paste.ubuntu.com/15667206/11:10
voidspacerogpeppe: and ifdown/ifup iit11:10
rogpeppevoidspace: well, utils.GetAddressForInterface discards all ipv6 addresses...11:11
voidspacerogpeppe: no packets dropped and some traffic11:11
voidspacerogpeppe: we have a preferIPv6 setting somewhere - does that code not honour it11:12
rogpeppevoidspace: almost nothing in /etc/network/interfaces11:12
rogpeppevoidspace: just two lines:11:12
rogpeppeauto lo11:12
rogpeppeiface lo inet loopback11:12
rogpeppevoidspace: nothing in interfaces.d either11:12
voidspaceoh, odd11:12
voidspaceI wonder where it is - this is xenial?11:12
rogpeppevoidspace: nope, trusty11:13
voidspaceah11:13
voidspaceI've not worked with trusty networking - I can commission a trusty box though11:13
* rogpeppe has been meaning to find the time to reinstall from scratch for *ages*11:13
dimiternrogpeppe, for trusty you need to add "trusty-backports" to /e/apt/sources.list && sudo apt-get update && sudo apt-get install -t trusty-backports lxd11:14
rogpeppedimitern: like: add-apt-repository trusty-backports ?11:15
dimiternrogpeppe, voidspace, prefer-ipv6 is no longer respected (always false), but the setting itself is not gone yet (will be soon though)11:15
rogpeppedimitern: hiya BTW :)11:15
dimiternrogpeppe, it's not a repo, it's a cloud pocket or something like that11:15
dimiternrogpeppe, ;) hey11:15
rogpeppedimitern: what exact line(s) do i need to add for /etc/apt/sources.list ?11:15
dimiternrogpeppe, if you paste yours, I can show you - can remember off-hand11:16
rogpeppedimitern: paste what?11:16
dimiternrogpeppe, /etc/apt/sources.list from the trusty machine in question11:16
rogpeppedimitern: the current contents of my sources.list11:16
rogpeppedimitern: https://pastebin.canonical.com/153655/11:17
dimiternrogpeppe, actually - here's a wiki page: https://help.ubuntu.com/community/UbuntuBackports11:18
dimiternrogpeppe, so you need a line like this somewhere: deb http://archive.ubuntu.com/ubuntu trusty-backports main restricted universe multiverse11:18
dimiternrogpeppe, but looking at your paste you seem to have it enabled already11:19
rogpeppedimitern: yeah11:22
dimiternrogpeppe, so just apt-get update && apt-get install lxd/trusty-backports should work now11:23
rogpeppedimitern: still same problem after doing that11:24
dimiternrogpeppe, the container still did not start?11:24
rogpeppedimitern: juju refuses to do anything because the interface has no ipv4 addresses11:25
dimiternrogpeppe, lxdbr0 ?11:25
rogpeppedimitern: yeah11:25
dimiternrogpeppe, hmm I'm not that familiar with lxdbr0 specifics, but you should be able to use "lxc-bridge: lxcbr0" in your bootstrap config11:28
dimiternrogpeppe, provided lxc1 package is also installed and lxcbr0 exists11:28
voidspacedimitern: frobware: babbageclunk: dooferlad: merge master onto maas2 http://reviews.vapour.ws/r/4473/11:28
rogpeppedimitern: i'd prefer to be able to bootstrap without using that tbh11:28
rogpeppedimitern: but i'll try it11:29
dimiternrogpeppe, well, on trusty lxd is still kinda flaky; btw you might instead need to change /etc/default/lxd to enable IPv4 addresses11:30
dimiternrogpeppe, check this, it might help - https://github.com/lxc/lxd/pull/971/files11:30
rogpeppedimitern: unknown config field "lxc-bridge"11:30
dimiternrogpeppe, ah, sorry - that's no longer supported yeah :/ - I guess the second option then - changing /e/d/lxd (or lxd-bridge) there to provide IPv4 addresses11:42
voidspacedimitern: frobware: dooferlad: babbageclunk: branch adding a feature flag for the MAAS 2 work http://reviews.vapour.ws/r/4474/11:42
rogpeppedimitern: /etc/default/lxd doesn't exist for me11:42
rogpeppedimitern: ah, but lxd-bridge does11:43
dimiternrogpeppe, right - there11:43
dimiternvoidspace, looking11:43
rogpeppedimitern: hmm, i added a bunch of ipv4 stuff in lxd-bridge, restarted lxd (sudo initctl restart lxd), but no ipv4 addresses seem to be arriving11:46
rogpeppedimitern: FWIW my lxd-bridge file now looks like this: http://paste.ubuntu.com/15667827/11:46
dimiternrogpeppe, try adding a new container11:48
rogpeppedimitern: is that easy? i've not used lxd directly before.11:48
anastasiamacvoidspace: in fact, if possible, it'd be awesome if any property accessors from maas instance also returned errors... it was very sad that we were swallowing that one :/11:48
rogpeppedimitern: is the lxd command the way to do it?11:49
rogpeppedimitern: (i don't see anything that talks about creating a new container in its help)11:49
* rogpeppe googles11:49
dimiternrogpeppe, lxc launch ...11:50
rogpeppedimitern: not lxd?11:50
dimiternrogpeppe, lxc is the cli client, lxd is the server daemon11:51
dimiternvoidspace, LGTM11:51
rogpeppedimitern: ok, now i have to work out how to fetch an image...11:51
dimiternhttps://linuxcontainers.org/lxd/getting-started-cli/ << nice guide11:52
dimiternrogpeppe ^^11:52
rogpeppedimitern: ah, launch ubuntu failed but launch ubuntu:14.04 worked11:52
rogpeppedimitern: still no ipv4 address11:53
dimiternrogpeppe, I think juju adds an alias "ubuntu" for the current series image11:53
dimiternrogpeppe, ah :/ bad luck - folks in #server @c might be able to help11:54
rogpeppedimitern: i'll just give up and use ec2 i think. i've already spent too much time chasing this.11:54
rogpeppedimitern: thanks for your help.11:54
voidspacedimitern: thanks11:55
voidspaceanastasiamac: the way we interact with maas has completely changed11:55
voidspaceanastasiamac: isn't it horrifically late for you now?11:55
dimiternrogpeppe, np11:56
voidspacedimitern: shall I just land the master merge?11:56
=== urulama|eod is now known as urulama
voidspacedimitern: http://reviews.vapour.ws/r/4473/11:56
anastasiamacvoidspace: i had a preview of new maas abstraction (peeked at thumper's branch)! it's awesome \o/ thank you for doing it!11:58
voidspaceanastasiamac: yeah, it's very cool - all thumper's work, we're just using11:58
anastasiamacvoidspace: it's 10pm... for normal ppl probably not a work hour... but who here respects that? :D11:58
voidspaceanastasiamac: ah, I thought it was later11:59
voidspaceheh11:59
anastasiamacvoidspace: r u re-writting maas19 interactions in similar way too?12:00
anastasiamacor it stays the same?12:00
dimiternvoidspace, does it build and pass make check?12:01
babbageclunkvoidspace: sorry, been talking to frobware, not doing much typing12:37
mupBug #1567396 opened: tools/lxdclient should not be using a global variable for several functions <lxd> <tech-debt> <testing> <juju-core:Triaged> <https://launchpad.net/bugs/1567396>12:41
frobwaredooferlad, ping12:41
dooferladfrobware: pong12:42
frobwaredooferlad, can we HO about the bond issue - was interested in your repro case12:43
voidspaceanastasiamac: probably not, too much work :-(12:43
dooferladfrobware: sure, but I really need lunch12:43
voidspaceanastasiamac: MAAS 2 is new code so worth doing right12:43
dooferladfrobware: can it wait 30 minutes?12:43
frobwaredooferlad, whenever works12:43
dooferladfrobware: thanks!12:43
voidspacedimitern: yep, builds and passes12:43
voidspacedimitern: I had some conflict resolution to do - nothing too difficult though12:43
anastasiamacvoidspace: sounds sensible12:44
voidspacebabbageclunk: my feature flag change has landed on MAAS212:45
babbageclunkvoidspace: Sweet12:46
voidspacebabbageclunk: so after you update you'll need to set the MAAS2 feature flag to bootstrap against MAAS 2.012:46
dimiternvoidspace, lgtm12:46
voidspaceanastasiamac: it would certainly be possible, and nicer... so if anyone has any spare time :-)12:46
voidspacedimitern: ta12:46
anastasiamacvoidspace: well, i don't think it's worth it... but the decision should b based on maas.19 life expectancy ... if it's to be short-lived, I'd say that what we have now will do...12:48
voidspaceanastasiamac: yeah12:48
voidspacenobody ever has any spare time anyway12:48
anastasiamacso true12:49
rogpeppecould i have a review on this almost-entirely-trivial one line change, please? it should make some things easier to debug in the future. https://github.com/juju/cmd/pull/3513:28
dooferladfrobware: back13:29
natefinchrogpeppe: would prefer not to log the error twice13:40
natefinchrogpeppe: can you do an if debug enabled...13:40
frobwaredooferlad, was your repro case with h/w?13:40
dooferladfrobware: yes13:41
frobwaredooferlad, as opposed to KVM13:41
rogpeppenatefinch: what would you do by preference? it's *really* useful to be able to see the error details.13:41
rogpeppenatefinch: you'd print the error differently if debug enabled?13:41
rogpeppenatefinch: i think it's kinda reasonable to have a separate debug line exposing details of the error without changing the usual error printing13:42
natefinchrogpeppe: yes... details if debug enabled, otherwise just the string..... otherwise you're printing the error twice, which could be confusing, make it look like we did something twice13:42
rogpeppenatefinch: tbh i think we'd want the error details on a separate line for readability anyway13:43
rogpeppenatefinch: perhaps a better error message string would felp?13:43
rogpeppehelp13:43
natefinchrogpeppe: I just worry that two log entries would be confusing. Maybe if debug is enabled, append the details to the single errorf log message?13:44
rogpeppenatefinch: if you do that, that line becomes very unwieldy13:45
rogpeppenatefinch: i think it should be reasonably obvious - one's at debug level and they both contain the same text.13:45
rogpeppenatefinch: perhaps surround it all in brackets? "(error details: xxxx)" and print after the main error message?13:46
natefinchrogpeppe: then put a comment in the code that states you're doing it on purpose, otherwise someone's going to come along and "clean up" the "duplicate" log output.13:46
natefinchrogpeppe: I do think after is better than before13:46
natefinchrogpeppe: and brackets will make it more obvious that it's not just another instance of the error.13:47
natefinchrogpeppe: and maybe use a sans-serif font, since that'll make it stand out more.13:47
rogpeppenatefinch: :)13:47
mupBug #1567458 opened: destroy-controller message and failure is not user friendly <juju-core:New> <https://launchpad.net/bugs/1567458>13:56
rogpeppenatefinch: better now? https://github.com/juju/cmd/pull/35/files14:10
natefinchrogpeppe: lgtm14:15
rogpeppenatefinch: thanks14:15
alexisbmorning all14:15
dooferladmorning alexisb14:18
perrito666hi alexisb14:18
kwmonroehey tych0, does your lxdbr0 fix always return 'lxdbr0', or do we check elsewhere for whatever the user configured?  (https://github.com/juju/juju/pull/5004/files#diff-d0829a0ea5ffc060343eba40ecd40aa8R66)15:20
tych0kwmonroe: yes, that only gets called when the user hasn't configured anything15:24
tych0kwmonroe: expand the last hunk of the diff and you can see15:24
voidspacebabbageclunk: StartInstance can allocate and start a machine15:25
voidspacebabbageclunk: and I've done as much of StartInstance as is possible without further work in gomaasapi15:25
voidspacebabbageclunk: now to test it all...15:26
voidspacebabbageclunk: don't forget to write up what you've done in the doc15:26
kwmonroecool tych0, thx15:27
frobwarebabbageclunk, http://juju-sapphire.github.io/15:56
frobwarebabbageclunk, https://github.com/dimitern/testcharms16:02
perrito666cherylj: ping16:04
babbageclunkvoidspace: nice! I've got the storage stuff done, although not tested yet. Need to merge your master merge.16:04
cheryljhey perrito666, what up16:04
voidspacebabbageclunk: cool16:04
babbageclunkvoidspace: will write up now16:05
voidspacebabbageclunk: I found another undocumented parameter to AllocateMachine16:05
voidspacebabbageclunk: storage....16:05
babbageclunkvoidspace: yay16:05
perrito666hey, I am trying to test my fix using lxd provider but I get a permission denied on my instances :( when juju tries to ssh, did you enconter this issue testing with xenial?16:05
voidspacebabbageclunk: so storage and spaces are not yet supported, but trivially easy to add them16:05
babbageclunkvoidspace: why not spaces?16:06
voidspacebabbageclunk: because interfaces parameter is missing on AllocateMachineArgs16:07
voidspacethat's the first undocumented parameter, storage is the second16:07
perrito666cherylj: that was for you16:07
voidspacethey haven't changed since 1.9 - but they weren't documented there either...16:07
babbageclunkvoidspace: ah, right - thought you meant because storage isn't done yet16:07
cheryljperrito666: I'm thinking of what might be causing that....16:08
voidspacebabbageclunk: no, you specify storage to AllocateMachine (as well as the storage specific code)16:08
voidspacebabbageclunk: I'm off to do some exercise - bbiab16:08
cheryljperrito666: when are you running into this?  during bootstrap?16:08
cheryljperrito666: do you have a --debug output you could paste?16:09
voidspacebabbageclunk: what happened with StopInstances - is it done?16:09
voidspacebabbageclunk: I need that too really, it's used by StartInstance16:09
perrito666cherylj: during bootstrap everything seems fine until the moment where it has to try logging int16:09
perrito666which is odd because the ubuntu user has the right credentials in authorized keys16:09
cheryljperrito666: what's your host?  xenial?16:09
perrito666wily16:10
babbageclunkvoidspace: It's done but not tested - it needed storage so I parked it while I was doing that.16:10
babbageclunkvoidspace: Once storage is merged and pushed I'll merge storage into stop instances and push that too.16:11
babbageclunkThen you can use both/16:11
mupBug #1567518 opened: Payload commands don't work <juju-core:New> <https://launchpad.net/bugs/1567518>16:11
cheryljperrito666: could you humor me and paste the --debug output?16:13
cheryljredir_afk: your PR: https://github.com/juju/juju/pull/5022 is going to conflict with one of mine because I also removed that lower case test :)16:15
cheryljit's a race to merge!!16:15
perrito666cherylj: sure I was waiting for it to finish16:16
perrito666cherylj: https://pastebin.canonical.com/153710/16:17
cheryljoh oh oh I think I may know16:17
cheryljperrito666: are you in the lxd group?16:17
perrito666yes16:17
cheryljI thought I saw something similar16:17
cheryljdamn16:17
cherylj not that then16:18
cheryljwhy is it connecting to 10.0.3.1??16:18
cheryljperrito666: can you run lxd list and see what the IP is for juju-bf76fca1-e52a-4c5a-877e-650f42945363-machine-0?16:19
cheryljsorry lxc list16:19
perrito666well it was destroyed by juju now but it was16:19
perrito666juju-bf76fca1-e52a-4c5a-877e-650f42945363-machine-0 | RUNNING | 10.0.3.1 (lxcbr0) |      | PERSISTENT | 016:19
cheryljperrito666: did you update lxd and do all that reconfiguring?16:19
perrito666cherylj: yup16:19
* perrito666 tries trusty16:20
cheryljand did you configure it to use lxdbr0?16:20
cheryljI think this is the same issue that they're running into in CI16:20
perrito666I changed my lxd to use lxcbr016:20
perrito666cherylj: I found that issue yesterday and fixed it right away16:20
cheryljperrito666: I think this is a lxd or config issue - I don't think any instance should be assigned the bridge IP (10.0.3.1)16:22
perrito666the problem happens only when using xenial, odd16:24
cheryljperrito666: I'm going to play with the machine in CI that's also running into this and see what I can do16:35
perrito666tx16:38
perrito666the problem is not present in trusty16:38
perrito666its  https://cloud-images.ubuntu.com/releases/ the place where images should be being downloaded from?16:40
perrito666cherylj: confirmed, this happens with xenial image16:45
frankbancherylj: is http://reports.vapour.ws/releases/3868/job/run-unit-tests-xenial-amd64/attempt/788 really related to embedded-gui? or is it a known problem?16:47
cheryljfrankban: that looks like a CI issue.16:48
cheryljnot related to the branch16:48
frankbancherylj: sorry. I meant http://reports.vapour.ws/releases/3868/job/lxd-deploy-xenial-amd64/attempt/49116:48
cheryljfrankban: not an issue with your branch.  I am working to get that fixed atm :)16:49
frankbancherylj: ccol, so embedded-gui is clean (at least latest branch), I am merging another one there16:50
frankbancherylj: ty16:50
cheryljnp!16:50
cheryljfrankban: is the --no-gui bootstrap flag the last piece for that branch?16:50
frankbancherylj: yes, if QA goes ok it's the last one16:51
cheryljk, thanks!  (and thank you for having makyo pick up that bundle bug!)16:51
frankbancherylj: so I am thinking about merging embedded-gui into master tonight or tomorrow morning, after QA16:51
frankbancherylj: np16:51
cheryljfrankban: we will merge it for you once it gets a good CI run16:51
cheryljso no need to worry about that16:51
frankbancherylj: awesome!16:52
cheryljand we'll let you know if there are problems too :)16:52
frankbancherylj: cool16:52
frankbancherylj: oh, after this one lands I'd have to merge master back into embedded-gui, so that we can start QAing with new lxd stuff etc. is that a problem?16:54
perrito666cherylj: is there any other way that I can test my fix? are there xenial images in amazon?16:54
cheryljfrankban: can you work directly on master once we land embedded-gui?  If your changes aren't radical / pervasive, you should do that16:55
cheryljperrito666: yes :)  bootstrap on aws with default-series=xenial works  just fine :)16:55
perrito666tx a lot16:55
frankbancherylj: when are we landing embedded-gui?16:56
cheryljfrankban: after it gets a good CI run16:57
frankbancherylj: ok, ack16:57
cheryljso if everything passes, today or tomorrow16:57
frankbancherylj: ok, so I'll not merge master back, so we'll have a single CI run, correct?16:57
cheryljfrankban: when did you last merge master?16:58
frankbancherylj: Apr 4 13:44:1616:58
cheryljfrankban: yeah, I'd re-merge master before we do another CI run.  There have been fixes to CI failures since then16:59
frankbancherylj: ok, I'll do that asap then16:59
cheryljthx!17:00
frankbancherylj: --no-gui just landed, Il'' merge master17:00
=== redir_afk is now known as redir
redircherylj: let me know what order you prefer17:08
frankbancherylj: ok mrege job is running https://github.com/juju/juju/pull/503617:11
perrito666aghhh xenial really hates me17:12
perrito666finally booting, bbl in about one hour, mail me if anybody needs me17:12
lazyPowerSo, i'm not sure what to plug in a bug here if someone has a second to lend a hand so i can file a bug about this properly i'd appreciate the help - https://youtu.be/ApLWbc2qQ30  <-- illustration of the issue17:58
rick_h_lazyPower: you killed it!18:05
rick_h_lazyPower: so...it sure looks like storage-get somehow took down the stateserver?18:06
rick_h_lazyPower: can you juju status outside of the debug-hook?18:06
cherylj....um... I'm pretty sure that we shouldn't be setting the lxc bridge address as an API Host port for the lxd provider18:10
cheryljHas anyone deployed a service in lxd recently?18:10
alexisbcherylj, I have been trying but failing18:11
cheryljI've gotten lxd to bootstrap18:11
alexisbthough before service deploy18:11
alexisbmy controller bootstraps18:11
alexisband hosted model fails18:11
cheryljbut the service deployed just stays in " Waiting for agent initialization to finish "18:11
cheryljfudgey mcfudgeington18:11
alexisblovely18:12
alexisbheh18:12
cheryljI look at the log and it's trying to connect on the WRONG IP to the controller18:12
alexisbcherylj, dont you like opening bugs??18:12
cheryljwell, I don't really need to triage bugs that I open, so that's something, right?18:12
cheryljheh18:12
alexisbo the bright side18:12
alexisbsee rick_h_ cherylj knows how to see the bright side ;)18:13
cheryljnice18:13
rick_h_the bright side is the best side!18:14
mbruzekcherylj: I tried to bootstrap lxd today, did not get it to work.  ERROR failed to bootstrap model: waited for 10m0s without getting any addresses18:17
cheryljsinzui_: the bright side here is that I got lxd to bootstrap on the xenial slave18:17
cheryljsinzui_: the bad part is that agents talk to the wrong IP for the state server18:17
sinzui_cherylj: but the test still doesn't pass http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/lxd-deploy-xenial-amd64/492/console18:18
cheryljsinzui_:  is the revision stale?  I was getting that until I pulled master again18:18
sinzui_cherylj: ^ I can seen this setup container/vm non sense before and I had to update severalt hings18:19
cheryljmbruzek: I just worked around that issue in our CI machines.  I had to do the dpkg-reconfigure lxd step mentioned in the thread18:19
lazyPowerrick_h_ - sorry about missing the messages, not sure what happened... some weirdness with my znc i guess... 1 sec while i get you the status output18:19
sinzui_cherylj: master? maybe I should make it the most important thing to test next18:19
cheryljsinzui_: yeah, I was pulling from master18:19
mbruzekcherylj:  I read that, but if I work around it now, will I not have to undo the workaround when Juju fixes the problem?18:19
rick_h_lazyPower: yea, the question is that you seem to lost connectivity to the state server, so is that it went down for everyone? or just something on that unit you were on? or what18:19
lazyPowerjust that unit it appears, and it comes back18:20
lazyPowerits like storage-get is segfaulting or getting an invalid token?18:20
lazyPowerhttps://gist.github.com/f8f928f5927cd08c55200d6d9165275418:20
lazyPowerthere's an error to that effect in one of the debug-hook sessions18:20
mupBug #1567594 opened: upgrade-gui command isn't in the juju tab complete <juju-core:New> <https://launchpad.net/bugs/1567594>18:20
* lazyPower fishes that up as well18:20
mbruzekcherylj: so my question is if I work around it. Wil I have to undo that change when it is fixed18:21
mbruzek?18:21
cheryljmbruzek: my understanding is that the dpkg-reconfigure step is going to be a permanent thing on the lxd side as part that update.18:21
lazyPowerhttp://paste.ubuntu.com/15676295/ <- that thing is what i was referring to about the session being mismatched18:21
cheryljmbruzek: as long as there isn't another lxd update that breaks things18:21
cheryljmbruzek: this should be a one time thing18:21
cheryljmbruzek: and changes in master (not beta3) will be fixing this "for good"18:22
mbruzekcherylj: Wouldn't we want to change Juju to look for the new device?18:22
lazyPoweraha paydirt, there's a treasure trove of stacks in the log18:22
cheryljwell18:22
* lazyPower fires up a bug18:22
cheryljmbruzek: those changes are in master18:22
rick_h_lazyPower: hmm, yea. I'd go with a bug along the lines of "storage-get causes state server to 'error: bad request'18:22
cheryljmbruzek: and will be part of the release next week18:22
mbruzekcherylj: So my  understanding is the old lxd named the bridge one way, the  new lxd names it something new, so the fix is to reconfigure lxd to use the old name.18:23
mbruzekcherylj: I figured Juju was just going to start looking for the new name.18:23
mbruzekcherylj: Or am I misunderstanding the email thread?18:23
cheryljmbruzek: yeah, that's how you can get it working with beta3.  The changes in master are to make juju smarter about choosing the name, I believe18:23
cheryljnot hard coding18:23
mbruzekcherylj: perfect18:23
mbruzekcherylj: Thank you for explaining it to me.18:24
cheryljmbruzek: no worries, it helped me too...  and I see in the code that we look first for lxdbr0, then fall back to lxcbr0 if not present18:25
lazyPowerhttps://bugs.launchpad.net/juju-core/+bug/156759818:28
mupBug #1567598: storage-get causes state server to 'error: bad request' <juju-core:New> <https://launchpad.net/bugs/1567598>18:28
cheryljlatent bug alert!18:29
cheryljheh18:29
cheryljthese lxd issues have revealed a latent bug in the lxd provider18:30
mupBug #1567598 opened: storage-get causes state server to 'error: bad request' <storage> <juju-core:New> <https://launchpad.net/bugs/1567598>18:41
redircan I set an env variable via gocheck to have it pass it/set it up for tests?18:56
rediror just run with an env var set?18:57
mupBug #1567607 opened: vsphere bootstrap fails when environments.yaml has a password with the # character <bootstrap> <cdo-qa> <vsphere> <juju-core:New> <https://launchpad.net/bugs/1567607>19:02
perrito666cherylj: http://reviews.vapour.ws/r/4481/19:13
mupBug #1567635 opened: FilterLXCAddresses should be updated to also filter out lxd bridge addresses <network> <juju-core:Triaged> <https://launchpad.net/bugs/1567635>20:03
bogdanteleagahow can I get logs with debug-log from a controller?21:06
mwhudsonperrito666: ping21:23
mwhudsonperrito666: looks like mongo/mongo.go only looks for /usr/lib/juju/mongo3/bin/mongod but iiuc that's never going to be released21:24
mwhudsonperrito666: and it should look for /usr/lib/juju/mongo32/bin/mongod instead?21:24
mwhudsoner 3.221:24
redirAnyone around with knowledge of juju/juju/cmd/juju/commands/ssh bits ?21:26
mwhudsoner er this got fixed and then reverted in master a few days ago21:27
bogdanteleagaanybody from networking around?21:50
rick_h_bogdanteleaga: sorry, they're UK/EU21:51
thumperbogdanteleaga: what do you need?21:51
bogdanteleagathumper, just deployed a windows machine with 2.0 and I'm getting a *lot* of spam in the logs21:53
bogdanteleagahttp://sprunge.us/LdKN21:53
bogdanteleaganot sure what it's even trying to do21:53
thumperbogdanteleaga: that isn't just spam, but a bug21:55
thumpernotice this:21:56
thumpermachine-3: 2016-04-08 05:35:45 DEBUG juju.worker.dependency engine.go:475 "machiner" manifold worker stopped: cannot update observed network config: cannot set link-layer devices to machine "3": invalid device "Loopback Pseudo-Interface 1": Name "Loopback Pseudo-Interface 1" not valid21:56
thumpermachine-3: 2016-04-08 05:35:45 ERROR juju.worker.dependency engine.go:522 "machiner" manifold worker returned unexpected error: cannot update observed network config: cannot set link-layer devices to machine "3": invalid device "Loopback Pseudo-Interface 1": Name "Loopback Pseudo-Interface 1" not valid21:56
thumperunit-noop-2: 2016-04-08 05:35:47 INFO juju.worker.upgrader upgrader.go:178 desired tool version: 2.0-beta421:56
thumpermachine-3: 2016-04-08 05:35:48 DEBUG juju.worker.dependency engine.go:461 "machiner" manifold worker started21:56
thumpermachine 3 bis anyway21:56
thumpertrying to set an invaliddevice21:56
thumpermachiner worker stops21:57
thumperand then restarts21:57
thumperrince and repeat21:57
bogdanteleagayeah, I notice the lack of a machiner21:57
bogdanteleagaat least it gets a few seconds of activity to launch the unit \o/21:57
thumperso, obvious bug there21:57
thumperbogdanteleaga: btw, bumped into a guy over here who when to uni with a some cloudbase guys21:58
thumperadi I think21:58
thumperand someone else21:58
bogdanteleagawe've got quite a few adi's :p21:59
bogdanteleagaany chance you know his name?21:59
bogdanteleaga(the guy you met)21:59
thumperbogdanteleaga: Elvis Adomnica21:59
thumperbogdanteleaga: he now teaches at Otago Polytechnic22:00
thumpersuch a small world22:00
bogdanteleagathumper, yup :)22:02
bogdanteleagafor a second I thought you're at some sprint in europe :p22:02
thumperhe22:03
thumpernot yet22:03
thumperspeaking of which22:03
thumperalexisb: we haven't talked further about team sprint location22:03
alexisbthumper, yea22:03
alexisbI suck22:03
thumperit is a busy time22:03
thumpereveryone has other focus22:04
mupBug #1567676 opened: windows: networker tries to update invalid device and blocks machiner from working <juju-core:New> <https://launchpad.net/bugs/1567676>22:06
thumperanyone? https://github.com/juju/gomaasapi/pull/2722:11
mwhudsonum22:13
mwhudsondo the juju tests pass with mongodb 3.2?22:13
* thumper shrugs22:13
* mwhudson bangs head on nearby flat surface22:14
mwhudsoni'm seeing some failures on s390x but i bet they're not the fault of my patches22:16
mwhudson... error string = "server returned error on SASL authentication step: Authentication failed."22:16
mwhudson... regex string = "auth fail(s|ed)"22:16
thumperheh22:19
alexisbmwhudson, the only bug we have for s390 is https://bugs.launchpad.net/juju-core/+bug/156504422:19
mupBug #1565044: s390x unit tests fail because not tools for arch <ci> <jujuqa> <s390x> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1565044>22:19
mwhudsonahem ahem ahem http://data.vapour.ws/juju-ci/products/version-3869/run-unit-tests-mongodb3/build-484/consoleText22:20
mwhudson+ echo 'mongodb3 unittests are not supported by master'22:20
mwhudsonmongodb3 unittests are not supported by master22:20
mwhudson+ exit 022:20
mwhudsonalexisb: i don't want to load more onto the juju team but is someone working on this?22:21
mwhudsonif juju in xenial is going to use mongodb3.2 it seems like the tests should work with that combination...22:22
alexisbmwhudson, yep22:22
alexisbwe will get someone on it22:22
mwhudsonok cool22:22
mupBug #1567683 opened: lxd provider should filter bridge addresses from instance.Address() <lxd> <network> <juju-core:Triaged> <https://launchpad.net/bugs/1567683>22:36
alexisbthumper, is menn0 around today?22:43
thumperyep22:43
thumperheh, he reboots and forgets to reconnect to IRC22:43
alexisbI sent mail thanks22:44
alexisbfinally!22:52
thumperalexisb: finally what?22:53
alexisbgot the lxdbr0 configured correctly22:54
alexisbnow juju will work until I try and deploy a service22:54
alexisb:)22:54
alexisbit is a start22:54
* mwhudson gives up testing juju on his own machine and just gives money to amazon instead23:01
alexisbmwhudson, you can expense a certain amount23:02
alexisband we have shared amazon accounts - though I dont have all the details23:02
alexisbas I use my own23:02
mupBug #1567690 opened: Can't push charm to my new LP home <juju-core:New> <https://launchpad.net/bugs/1567690>23:03
rick_h_alexisb: mwhudson 200/mo no qiestions as long as it's juju related23:05
rick_h_more with reasons/etc23:06
anastasiamaca review plz http://reviews.vapour.ws/r/4484/ (it's an easy one-liner)23:06
mwhudsonrick_h_: i expect i'll be spending about $2 :-)23:15
anastasiamacaxw: perrito666: standup23:17
axwonm23:17
axwomw23:17
rediraxw: http://reviews.vapour.ws/r/4485/ and I'll bbiab.23:38
axwredir: cheers, looking23:38
=== redir is now known as redir_afk

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!