[00:00] jw4, I think that is a question for dimitern. I think the rule is lxcbr0 is not for general consumption. [00:00] PASS. [00:00] woot [00:00] sinzui: jw4: great! [00:00] sinzui: yeah - most of thi scode seems to be dimitern related [00:00] jw4, anastasiamac I am down grading the bug and re-describing the madness [00:00] sinzui: lol === ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: None [00:00] sinzui: jw4: looks i missed all the fun! [00:01] anastasiamac: o/ [00:01] sinzui: great find and thank u for master [00:01] jw4: o/ [00:01] jw4: thnx for being so efficient today :D [00:01] all is open [00:02] davecheney: [00:02] anastasiamac: hehe [00:02] github says the same than reviewboard [00:02] davecheney: for instance https://github.com/juju/juju/pull/1532/files#diff-a722ce6455c045798bd7ef9db08e0ceaR11 [00:03] oh, now I need to rethink how we fix the the vivid ppc64el packaging bug without my stilson-07 [00:03] davecheney: ah but the empty lines are not there [00:03] ok, rb shows garbage for your change [01:18] axw: standup [01:18] doh, thanks [01:44] thumper: when your ready, destory env: http://reviews.vapour.ws/r/627/ [01:54] anyone about? we're having a weird juju machine agent issue where its dying with "panic: runtime error: slice bounds out of range" === kadams54 is now known as kadams54-away [08:36] morning [08:40] TheMue, morning [08:40] dimitern: hey, how is your time in the warm southern sun so far? ;) [08:41] dimitern: here we've right now -3° and I'm happy I not have to leave home [08:41] TheMue, not so warm today :) it's raining lightly and it's cloudy, but yesterday it was hot [08:42] TheMue, it still is 18 degrees though [08:42] dimitern: cloudy is good, right environment for juju. but juju is also hot, hmm? :D [08:43] dimitern: hey :) hows the sprint [08:43] dimitern: thanks for your info on the refactoring btw, good to know. I'm already continuing and wanted to make sure that it isn't useless [09:05] eagles0513875_, hey :) it goes swimmingly so far [09:05] TheMue, :) [09:05] TheMue, cheers [09:10] voidspace, o/ [09:10] voidspace, did you say you're to file a swap day for friday? [09:12] dimitern: I'll be taking a swap day Friday, yes [09:12] dimitern: o/ [09:12] dimitern: read through your email, all sounds good [09:16] dimitern, are you present? [09:17] lazyPower, i am indeed [09:17] voidspace, cheers, please file a request so I can approve it ;) [09:18] dimitern: can you poke your head in #juju on canonirc? I'm starting to traverse into juju core territory that I dont understand while helping debug a customer deploy (this shouludn't take long - as its basically a "i know or dont know" situation) [09:19] lazyPower, sure [09:20] dimitern: ok, will do [09:21] dimitern: I'm currently still looking at the proxyupdater issues [09:21] dimitern: a condition of the LGTM I got was to investigate *why* the current "first run" logic (that used to set environment variables) is faulty [09:22] dimitern: it *looks* sound, like it should have worked [09:22] dimitern: "first" is set to "true" in proxyupdater.New() when the worker is created [09:22] voidspace, I can confirm that the "first run" logic is pure bs - out of omission [09:22] dimitern: in SetUp onChange is called ("first" should be true) and *then* "first" is set to false [09:23] dimitern: omission? [09:23] SetUp is called in loop() of the NotifyWorker [09:23] on that call "first" should be true, and the proxy settings are fetched from api.EnvConfig [09:23] dimitern: glad to hear it :) [09:23] voidspace, yeah - it was overlooked [09:24] eagles0513875_, :) [09:24] dimitern: I've read through the code, it seems sound enough on a casual reading [09:24] dimitern: I'm going to put some tracing code and work out what's happening [09:24] dimitern: shouldn't take long [09:24] voidspace, sgtm, sorry I'm a bit distracted on multiple fronts right now [09:24] dimitern: I'm going to create a docker image for the manual provider though - so I can repeat experiments [09:24] dimitern: no problem [09:25] dimitern: and I'd also like to fix the charmrevision worker so we don't see that error in the logs [09:25] dimitern: that may mean moving where the proxyupdater is started [09:25] dimitern: just letting you know, so long as you're happy with me working on this no need to internalise the details [09:25] dimitern: and then I'll switch to the work you suggested [09:26] voidspace, thanks, it seems to me you have all under control there [09:26] heh [09:26] dimitern: being in control of yourself is always the first step... [09:28] :D indeed === ashipika1 is now known as ashipika [10:17] morning [10:20] jamestunnicliffe: here [10:20] jamestunnicliffe: ubuntu image pulled [10:21] great. Hangout in https://plus.google.com/hangouts/_/canonical.com/juju-sapphire?authuser=0 ? [10:21] jamestunnicliffe: sure [10:29] jamestunnicliffe: http://pastebin.ubuntu.com/10051327/ [10:32] jamestunnicliffe: http://pastebin.ubuntu.com/10051372/ [10:33] voidspace, hey, I've backported your fix for 1.22 http://reviews.vapour.ws/r/859/ - nothing's changed, but I'd appreciate if you can have a look, test it and approve it if it passes, but it can wait for tomorrow - no need to switch to it now [10:33] dimitern: ok [10:33] voidspace, thanks! [10:33] dimitern: just setting up docker images for easy testing of manual provisioning [10:34] voidspace, does it make it easier to deploy/debug/redeploy? [10:34] dimitern: yes [10:34] dimitern: because "destroy-environment" with manual provider doesn't leave a clean environment for a re-deploy [10:34] dimitern: which is probably worth a bug report [10:34] voidspace, sweet, I'd like to hear about it next week so I can try it out [10:35] dimitern: the docker image should make it very easy to repeat a manual deploy [10:35] voidspace, that indeed seems worthy of a bug report [10:35] dimitern: but creating a new kvm image for manual provisioning is heavyweight and docker should be easier [10:36] voidspace: http://pastebin.ubuntu.com/10051436/ [10:36] voidspace: docker build -t juju-thing . [10:36] voidspace, +1 [10:36] voidspace, we can try using snappy for the same process at some point as well [10:37] dimitern: heh, cool [10:38] ;) [10:38] ok, gtg for now [10:42] dimitern: have you seen snappy in action in capetown? [10:50] jamestunnicliffe: what should that file be called? [10:50] voidspace: Dockerfile [10:54] jamestunnicliffe: thanks [10:55] well, it's running... [10:56] jamestunnicliffe: ah yes, ufw issues [10:56] voidspace: looks like it is best to do the firewall stuff outside of Dockerfile by telling docker itself what to allow in/out. Will let you know how soon. [10:57] jamestunnicliffe: http://askubuntu.com/questions/28215/how-can-i-fix-the-iptables-error-message-unable-to-initialize-table-filter [10:57] voidspace: https://docs.docker.com/articles/networking/#between-containers should have the answers [10:57] You need to load a kernel module for enabling the filter table. Run the next command as root: [10:57] modprobe /lib/modules/$(uname -r)/kernel/net/ipv4/netfilter/iptable_filter.ko [10:57] jamestunnicliffe: ok [10:57] jamestunnicliffe: cool [10:57] voidspace: the container is running on the host kernel, so you can't just insert stuff into the kernel :-) [10:58] right [10:59] jamestunnicliffe: https://github.com/docker/docker/issues/1251 [11:01] ah, that's using ufw on the host [11:13] voidspace: you may be best off, for now, creating a KVM / VirtualBox / whatever machine and setting up everything but Juju, then snapshotting it. [11:13] voidspace: that way you can always restore to the snapshot after playing [11:14] voidspace: this stuff with Docker is possible, but a headache. [11:35] voidspace: have you got a moment to triage a rsyslog cert issue? bug 1417875 [11:35] Bug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority

[11:59] * jamestunnicliffe goes for lunch [12:33] mgz: I can look, sorry for the delay - midwife visit [12:34] jamestunnicliffe: ok, cool - thanks [12:34] jamestunnicliffe: virt-manager has cloning but not snapshotting [12:34] jamestunnicliffe: I'll read up on it to check if I need to keep a "clonable" image around, probably... [12:36] mgz: you're probably better off with wwitzel3 on that one [12:36] mgz: he worked on x509 certs for rsyslog not-that-long-ago [12:36] bug 1417875 [12:36] Bug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority

[12:42] voidspace: I figured just thought I'd try you before he was on :) [13:09] mgz: it looks like a serious regression if it's reproducible [13:10] voidspace: pretty sure there has to be something else going on, logging in general in 1.21 has worked [13:12] mgz: right [13:13] mgz: I'm not familiar with the moving parts (except vaguely) around the certificate generation [13:29] hey guys; I'm currently working on the systemd support and have a question: [13:30] in your opinion, which would be the based place to place the unit file (bonus thanks if you also point me to how I could get that path :D [13:30] best* === cmars` is now known as cmars [15:50] bogdanteleaga, gsamfira: I have placed mongod.exe in the sys path and run the win unit tests. The job could complete. What do I need to setup in the env/go env to get the tests to match your experience? http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/run-unit-tests-win2012-amd64/29/consoleText [15:52] sinzui: first of all you should probably use one of my branches to test it since not everything is merged [15:54] bogdanteleaga, I can arrange that [15:56] sinzui: https://github.com/bogdanteleaga/juju/tree/try-ci [15:56] sinzui: try this one for now, it should work [15:57] bogdanteleaga, work as in pass? [15:57] sinzui: yes [15:58] bogdanteleaga, fab! [15:58] sinzui: actually, I think you need a branch in juju/testing as well [15:58] sinzui: that might be a bit harder to do [15:59] bogdanteleaga, I assume goarch is amd64. but what is CGO_ENABLED in yours? [15:59] me sees conflicting info on the windows machine [16:00] bogdanteleaga, I will use the make-release-tarball script to make a go tree to test with [16:02] sinzui: yes, it's 64bit, no modifications to go [16:03] natefinch: wwitzel3 standup? [16:03] thank you bogdanteleaga [16:07] sinzui: this is the one in testing I was talking about https://github.com/bogdanteleaga/testing/tree/skip-chmod [16:08] thank you bogdanteleaga [16:19] sinzui np, tell me how it goes [16:41] ericsnow: you there? [16:41] aznashwan1: OTP [16:43] for the love of me I have no idea what OTP means :)) [16:43] on the phone [16:43] natefinch: thanks, you're better than google :D [16:43] haha [16:43] took me a while to figure it out too === kadams54 is now known as kadams54-away [17:05] ericsnow: nice changes on the patch [17:05] perrito666: oh good [17:06] perrito666: wish I'd known about juju/paths when writing backups [17:06] aznashwan1: back [17:07] ericsnow: so, I had a couple of quick heads-up's [17:07] * perrito666 running the whole suite to have a fully passing one in more than a week [17:07] aznashwan1: k [17:08] ericsnow: firstly, I used the go-systemd unit to create the service file and, just as an observation, the order of the options is not consistent from one serialization to another [17:09] aznashwan1: ah, good to know [17:10] ericsnow: secondly, doing a dbus.Conn.ListUnits() only lists running/stopped/idle etc.. units, and does not help at all in determining whether a unit is enabled or not... [17:11] ericsnow: [stopped as in they actively ran and are now done] [17:11] ericsnow: i've asked the coreos guys for some help, and am waiting for their response on this... [17:11] aznashwan1: ah [17:12] aznashwan1: however, aren't those the units we want? [17:12] aznashwan1: glad you got in touch with the coreos guys [17:12] ericsnow: not in the context I was indenting on using it sadly D: [17:12] ericsnow: anyhow, on a brighter note, I've only just seen your PR on the init systems, and think it's looking great [17:13] ericsnow: was gonna' ask if you wanted me to get to putting my work over yours, so as to converge eventually [17:13] ericsnow: because what I was doing up to today would not fit too cleanly there as is D: [17:14] aznashwan1: wwitzel3 and I did something like that yesterday (based on your patch) [17:14] aznashwan1: we should sync up on this [17:15] ericsnow: well I'll get to putting it on top of yours then to spare you guys of the extra work [17:15] ericsnow: last couple of questions: [17:16] ericsnow: from what i've seen the old interface is not preserved (so the confusing Exists() and Matches() functions are gone), do you have any intention to still have both (and by the same definitions?)? [17:17] aznashwan1: there is just IsEnabled [17:18] aznashwan1: the equivalent to Exists is the service.Services.Check method [17:18] aznashwan1: but an InitSystem implementation doesn't have to worry about that [17:18] ericsnow: good, will be getting to adapting IsEnabled with go-systemd the minute I find out how [17:19] ericsnow: and I'll be doing the InitSystem implementation of systemd tomorrow [17:20] aznashwan1: check out what wwitzel3 and I did yesterday: https://github.com/wwitzel3/juju/compare/ww3-systemd [17:20] ericsnow: last thing, I swear: could you please tell me where I may get the place where you want me to put the .service (and ExtraScript optionally) (ideally point me to the actual package/function where I may get it) [17:21] aznashwan1: that was the motivator for the code in service/managed.go :) [17:22] ericsnow: i see [17:22] aznashwan1: the service.Services implementation takes care of that so the InitSystem implementations don't have to know about it (other than as the second argument to Enable) [17:25] ericsnow: great, well then I hope to get initsystem/systemd adapted tomorrow [17:25] aznashwan1: awesome [17:25] ericsnow: shall I keep rebasing over the branch the PR was created on, or this there some other one you are actively working? [17:26] ericsnow: [your* PR] [17:27] aznashwan1: for the init system stuff it's my init-service-cleanup-local branch [17:27] ericsnow: gotcha'. thanks for everything and see you soon :D [17:27] aznashwan1: keep in mind that it depends on https://github.com/juju/testing/pull/48/ and https://github.com/juju/utils/pull/108 [17:28] aznashwan1: np, thanks for working on this [17:32] wwitzel3: are you around? can you triage bug 1417875 or ask for any more information we'll need [17:32] Bug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority

[17:40] mgz: ok, taking a look [17:59] how do you determine a backend facade version from api calls? [17:59] wwitzel3: thanks [19:21] if your export_test file is over 200 lines... maybe you should just make your tests internal :/ [19:21] natefinch: lol [19:25] I hate export_test pattern :( [19:27] perrito666: me too [19:28] perrito666: the only time I think it's valid is when it's needed to avoid circular imports. [19:37] perrito666: natefinch: i will just +1 that sentiment [19:38] natefinch: if unit tests are really what we're going for, it becomes busy-work [19:38] natefinch: ps, are you alive up there in boston? yeesh! [19:39] katco: haha yeah... gotten 44" of snow in the last week and more on the way. Looks like we moved to northern Canada or something [19:39] natefinch: craziness [19:39] natefinch: i love snow, but i understand it's causing a few issues =/ [19:40] katco: at least we're (mostly) set up to deal with it. The roads are ok, the main problem is where to put all the snow that you clear off the roads. It's not too bad in the country, but in downtown, there's just no place to put it all [19:40] natefinch: shovel it into trains and ship it to the west coast! [19:40] katco: yeah. definitely glad I don't have to commute into Boston anymore... heard other people's commutes that were like 2.5 hours longer than normal (each way!) [19:40] natefinch: yeah i saw that. commuting = largest waste of ones life. [19:41] yep [19:44] dont you people cancel work for commuters on excessive snow days? [19:44] perrito666: yes, but companies vary [19:45] perrito666: it's usually individual managers who make the call for their team unless it's clearly unsafe and then the company might issue a site-wide communication [19:45] well, its a good thing I decided to go for a career in CS [19:45] g'night all [19:45] voidspace: night [19:46] voidspace: sleep well [19:46] perrito666: o/ [19:46] katco: plenty to do before sleep time :-) [19:46] but thanks, and I will... [19:46] voidspace: fair enough! [19:46] perrito666: when it's really bad they do call for travel bans.... but that's usually just during the storm. [19:46] that wonderful feeling when all tests pass [19:47] man I get so much mail from prs that my phone sounds like mario bros on steroids [19:55] sinzui: sha'ping [19:55] thumper: does that mean he has to hash himself? [19:56] haha [19:56] nah [19:56] sinzui: I have been thinking about the manual provider and ci upgrade bug with the 10.0.3.1 address issue [19:56] I'm pretty sure I know what is going on [19:56] I mentioned it in the bug [19:56] however [19:57] the new networking code from Dimiter's team has started enumerating all the network devices [19:57] to get their ip addresses [19:57] lxcbr0 by default gives 10.0.3.1 [19:57] this is classified as a class A private network [19:57] and then considered cloud private [19:57] and juju prefers A private networks? [19:57] and then the 'best address' to give out [19:57] well, there are two class A ones [19:57] I guess it picks the first? [19:57] or last? [19:57] * thumper shrugs [19:58] I think the solution is to blacklist some nic names [19:58] like 'lxcbr0' and 'virbr0' [19:58] neither of those will ever be useful addresses [19:58] probably lo* too [19:58] thumper, think this issue is the cause of bug 1417308 and bug 1416928 [19:58] Bug #1417308: Juju-ci3 cannot upgrade to 1.21.1

[19:58] Bug #1416928: juju agent using lxcbr0 address as apiaddress instead of juju-br0 breaks agents

[19:59] yup [19:59] I believe this worker was added in 1.21 [19:59] It certainly was [20:00] dave suggests that we look at network properties on the adapter rather than names [20:00] thumper, So I hope core cna provide a fix. I can make all these bugs a dup of a new bug based on your insight [20:00] so look for the loopback flag [20:00] or bridge [20:01] sinzui: we need to fix it, but I'm not entirely sure how [20:01] I'd like william and dimiter and john to talk about it in cape town [20:02] thumper, agreed [20:02] wwitzel3: did you have a chance to look at bug 1417875 yet? [20:02] Bug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority

[20:34] can someone tell me if the existence of the JUJU_ENV_UUID env attribute is an indicator of the api supporting it in the path? [20:58] hatch: it looks like it's only set as hookvars ... charms can read it, but none of the rest of juju's code seems to read it [20:58] hatch: not sure if that answers the question [20:59] natefinch: hmm I was under the impression that the api will require it soon so I was implementing it in the GUI [20:59] hatch: the API will require it, yes, but that's different than the environment variable [21:00] right - but we need to grab the uuid from the charm to supply it to the GUI [21:00] hatch: maybe I misunderstood the question [21:00] so I was wondering if I can use the existance of that attribute to be a flag to the GUI that it's now supported in juju [21:00] the gui can be deployed to many different juju versions [21:01] natefinch: basically something needs to say to the GUI 'use the uuid' or 'dont use the uuid' [21:02] the uuid environment variable has been defined for at least 5 months according to the source - I'm not sure if the API support follows the same [21:03] hatch: that sounds kind of a hackish way to determine the api version... can't you just query the API for what version it is? [21:05] ehh feature testing > version testing [21:06] version testing > depending on some random env variable and assuming you know what its existence or non-existence means :) [21:07] hatch: thumper would probably be the guy to ask about the best way to go about this [21:07] well I was just hoping to avoid writing the code to test with the uuid then fall back to w/o if it errors :) [21:10] hatch: understood.... there's probably a check you can do, I'm just not sure what it is, but checking for that env var does not seem like the most reliable thing to use [21:11] yeah - unfortunately the api doesn't tell us what features it supports [21:11] YET! [21:11] :) [21:12] granted we'd need to know that it supported the uuid to get that list first :P [21:12] haha === kadams54 is now known as kadams54-away [21:12] I was going to say - how do you know if it supports asking what features it supports? :) [21:12] damn egg/chicken [21:13] I prefer a chicken omlet anyways [21:14] somewhat harder to stir up, though [21:18] * thumper looks up [21:18] hi hatch [21:19] hatch: if there is an env-uuid in the .jenv file, use it [21:19] thumper: I am in the gui charm's start hook so I have access to the environment variables [21:19] is that an indicator that the api supports requests with it? [21:19] hmm... [21:20] yes? [21:20] haha [21:20] I can't find in the code where that url structure was added [21:21] so I can't compare to when the uuid env variable was [21:21] and then cross reference to some release # :) [21:21] hatch: I think it was 1.18 or 1.20 [21:22] you guys just do too much work [21:22] 1.20 has it [21:22] and 1.21 is stable [21:23] you can ask the version right? [21:23] wait....so even is unstable? [21:23] hatch: we don't do that now [21:23] hatch: you know what, we can just startif version > 1.20 [21:23] no need to go back all the way to the first origin of it [21:23] that works for me! [21:23] rick_h_: oh hai [21:23] rick_h_: how's cape town? [21:24] thumper: good stuff, but crazy [21:24] thumper: missing you to drink with after the day :P [21:24] we'll drink here for you [21:24] that's ok right? [21:25] :) [21:26] rick_h_: you should be sleeping now right? [21:27] it's only 9:30 [21:27] thumper: yea, should be.../me goes to bed [21:27] lol [21:27] just taking care of the team. reminding them to do reviews [21:27] that's nice of you :) [21:27] hatch: hmm, add 2hrs [21:27] oh woops, timezone fail [21:27] night all, ty thumper for helping hatch out [21:28] ^ and natefinch :) [21:28] mgz: I hadn't fired up my maas in a while so I'm just working out some issues in it so I can reproduce [21:28] wwitzel3: I misread 'maas' for 'ass' [21:29] thumper: how'd the mess demo go? [21:29] thumper: lol [21:29] lazyPower: I heard it went real well [21:29] haha [21:29] wwitzel3: ehehe [21:30] gooooooooood [21:30] * lazyPower hi5's thumper [21:30] * lazyPower resumes lurking now [21:31] mgz: I'm wondering if somehow the permission of the issuer cert is off [21:32] mgz: or if it is even there, soon as I get my setup bootstrapping again, I'll take steps to verify and reproduce and update the ticket. In the meantime I'll get it assigned to me, forgot to do that/ [21:35] wwitzel3: thanks! [21:44] got an HA juju deployment that was unplanned rebooted by the customer. been fighting with mongo corruption, respawns, and SocketExceptions. menn0 and gustavo had helped with a recovery earlier today, but it's gone bad again, member in FATAL. [21:56] thumper: if you have time, I made a fix for the joyent issue you saw: http://reviews.vapour.ws/r/861/ [21:57] jillr: is it just a single member that's down? If so, can you just kill it and then rerun juju ensure-availability to get back up to 3 state servers? [21:58] natefinch: it has not been consistently just one node, no. also we are colocating other services on the machines so I'd have to dig thru the relations [21:58] restarts of jujud-machine-# and juju-db have at times restored things, but only briefly [21:59] there was a missing record it was barfing on earlier but we thought that was resolved [22:00] or I suppose it'd be a missing transaction, sorry [22:01] jillr: I'm at EOD, but thumper, axw, waigani, and davecheney are all near the beginning of their days, so hopefully they can help out. [22:02] natefinch: cool, ta