[00:52] davecheney: ping [01:10] davecheney: all sorted [01:11] thumper: menn0: waigani: anyone else: I unblocked trunk by reverting a regression commit and reclassifying the associated bug as High for trunk (still critical for 1.22, 1.21) [01:11] this was a per agreement with curtis [01:11] ok, cool, ta [01:14] wallyworld: thanks :) [01:14] np :-) [01:24] wallyworld: tyvm [01:29] fuck fuck fuck fuck [01:30] thumper: you rang? [01:30] :P [01:30] * thumper sighs [01:30] we are starting lxc containers on a power machine with amd64 tools [01:31] and wondering why it is erroring weirdly [01:31] thumper: hmm, that seems less than ideal [01:31] I'm trying to work out where it got the wrong tools set [01:35] fark [01:48] wallyworld: bot still showing blocked [01:48] wallyworld: regression query not showing blocked [01:48] * thumper is confused [01:48] thumper: huh? anastasiamac told me she got something accepted [01:49] ok, maybe I tried too soon [01:49] there were 2 bugs - one i marked fix released, ther or as high [01:49] it was just after you mentioned above [01:49] ah, i was in the process of marking as fix released [01:49] sorry [01:50] ok [01:50] I've tried them again [01:50] we'll see if they land now? [01:54] wallyworld: i got something accepted by the bot but not landed [01:54] wallyworld: machine_test.go:1102: [01:54] c.Assert(sortedUnitNames(got), gc.DeepEquals, expect) [01:54] ... obtained []string = []string{"s2/2", "s3/2"} [01:54] ... expected []string = []string{"s2/2", "s3/1"} [01:54] sure, false error [01:54] but it wasn't rejected due to policy [01:54] wallyworld: as in re-$$merge [01:54] yep [01:55] that's one of our aewsome transient errors [02:01] wallyworld: I got bit by that one 3 or 4 times in a row last night [02:01] sigh [02:01] is there a bug raised? who does annotate say added the test? [02:03] ok, so we've run into an issue with the rsyslog direct connections, when we ensure-availability, all of the state machines end up with different rsyslog certs. [02:03] and the logging is distributed to all 3 of them, so we end up with 1 successful connection, and two failed connections for every nodes rsyslog worker, in a constant loop and it floods the error log. [02:04] wwitzel3: https://bugs.launchpad.net/bugs/1417875 ?? [02:04] Bug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority

[02:04] wallyworld: correct [02:04] so you managed to diagnose the cause :-) [02:04] wallyworld: I wasn't able to reproduce until I worked with Paul and figured out it was ensure-availability [02:04] ah, good effort! [02:04] wallyworld: yep, all my attempts to reproduce were non-HA [02:05] so manual work around is copy the rsyslog-cert and rsyslog-key from machine 0 to 1 and 2 [02:05] that resolves the issue [02:06] so the fix will be to hand out the machine 0 certs when adding new state servers? [02:06] wallyworld: right, for the rsyslog certs [02:07] or master [02:07] s/0/master [02:07] wallyworld: right [02:07] a couple of day's work i guess [02:07] wallyworld: I didn't see an easy way [02:08] yeah, me either at first glance [02:11] thumper: let's not be greedy with bot... [02:12] thumper? greedy? never [02:13] anastasiamac: it is only two [02:13] geez [02:13] thumper: hmm... keeping score?.. [02:14] anastasiamac: I'm aiming for that 'nothing better' feeling of landing code [02:14] thumper: ahh, then we'd beta not disturb [02:18] rick_h_: you around? [02:18] still? [02:27] thumper: there goes that feeling [02:58] thumper: no, I ran away [02:58] rick_h_: ok, I hear you are ill, go to bed :) [02:58] yea, keep thinking about that [02:59] figured I'd listen to you guys talk and it'll put me to sleep :P [03:10] wallyworld: ah, I just noticed there's a page 2 :) [03:10] hence why I didn't see the upgrade step before [04:06] wallyworld, I'm seeing "ERROR juju.cmd supercommand.go:323 connection is shut down" intermittently with 1.21. Do you know anything about that? [04:06] no :-( [04:07] happens when attempting 'juju deploy' [04:07] mainly seeing it with MaaS 1.8 [04:08] but I'm looking for more instances of it on other substrates === ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: None [04:09] wallyworld, the red failures from Feb 11 and 12 on this page are examples: http://juju-ci.vapour.ws:8080/job/test-maas-1_8/ [04:10] jog: sadly our CLI sucks - it doesn't report the real root cause error [04:10] it just gives that connectio shutdown [04:11] so you'd need to go look at the debug log [04:11] wallyworld, I've added --debug for future runs, will that help? [04:12] jog: maybe, but likely not - it's the server side logs (machine-X.log or allmachines.log) that would be needed [04:12] ok, I'll see what I can do about capturing those... [04:14] jog, you could bootstrap the environment with "logging-config: =INFO,unit=DEBUG" [04:15] jog, and then the unit-X.log files are likely to contain useful data, in addition to the machine logs [04:16] dimitern, alright that's an entry in the environments.yaml right? [04:17] jog, yeah; for existing envs, you can also change it with juju set-env logging-config='=INFO,unit=DEBUG' << thumper, this should work, right? [04:29] dimitern, with MaaS... juju status gives the dns-name: as a FQDN only know to the MaaS server for example: 'maas-node-402.maas' [04:29] most clients don't point there DNS as the MaaS server, so they can really do anything with that information. [04:30] Is it possible to provide the IP? dimitern if you're not the right person to ask do you know who is? [04:30] jog, that's true I guess - for that reason I had fixes a few issues around api endpoints (e.g. resolving hosts if possible) [04:30] jog, by juju you mean? [04:31] the 'juju status' output [04:32] jog, how about public-address field for units? [04:32] jog, I think this is an IP even with MAAS [04:32] it's the same, there is an example: http://juju-ci.vapour.ws:8080/job/test-maas-1_8/22/console [04:34] jog, how about juju run --unit dummy-source/0 'unit-get private-address' ? [04:34] jog, well, you can also (e.g. in a script) just try resolving the dns-name using the maas DNS server I guess [04:38] dimitern, I'll try the unit-get method and yeah I can script a way to resolve the dns-name... just wondering if it's a usability issue. [04:39] jog, it has been brought up before [04:39] ok, I thought I'd seen a bug for it before [04:39] jog, and now in pretty much all other providers we do return resolved IPs instead of hostnames, except for maas (as it prefers to refer to nodes by dns names) [04:51] dimitern, so I must be doing something wrong when setting logging-config: [04:52] I've tired both "logging-config: =INFO,unit=DEBUG" and "logging-config: '=INFO,unit=DEBUG'" in my environments.yaml [04:53] but get ERROR juju.cmd supercommand.go:323 there was an issue examining the environment: unknown severity level "INFO,unit=DEBUG" [04:53] jog, ah, I see - so the second one is fo the command line only, as in juju set-env logging-config='=TRACE' [04:54] jog, while the first one is for envs.yaml [04:54] jog, but as now I actually checked "juju help logging", the format uses "; " not "," as separators [04:55] jam: ping [05:00] review board is a piece of shit [05:01] why does it publish all your comments, when you push the "publish" buttont [05:01] ****EXCEPT***** the ones where you have commented in the review branch (not review diff) screen [05:01] that has a seperate, partially obscured, publish button [05:02] davecheney, you can do it for single comments, but IIRC you have to publish each one after adding, rather than hitting the "review-wide" publish [05:02] dimitern: the way it works out for me [05:02] there are comments when you are reviewing the diff [05:02] then there is the hovering button at the top of the page to publish or review them [05:02] there is the 'review' and 'ship it' buttons on the view diff and view review pages [05:03] but if you're having a conversation in the comments section (not on the view diff) [05:03] there is an entirely seperate and different "publish" button hidden just below the list of issues on the view reviews page [05:03] you only know if you ened to push it by looking on the dashboard [05:03] seeing the icon is orange (?) not blue, and hte mouse over hover says "comments drafted' [05:03] yep, not quite thought through UX [05:06] also, beware - if you're editing a comment and you wifi connection drops (or is down) when you submit the comment (still draft) UI-wise it appears on the page, but a few seconds later you're taken to a nasty "connection failed" page and "back" doesn't help getting your comment text back [05:17] yay, cloud! [05:44] dimitern: we append the unit=DEBUG unless they explicitly set unit to be otherwise [05:44] dimitern: so you were mostly right [05:44] be careful about setting to debug [05:44] as that'll get golxc and other libraries that also use loggo [05:45] juju=DEBUG is good [07:01] axw: i've updated the storage pools PR for your viewing "pleasure" [08:26] wallyworld: sorry, I was out before. will look shortly [08:27] morning o/ [08:33] good morning devs o/ [08:33] gsamfira: hi! adventuring myself in trying http://www.cloudbase.it/ws2012r2/ [09:14] menn0: poke [09:14] I tried poking earlier but didn't see you online [09:31] wallyworld: reviewed [09:31] sorry for the delay [09:32] dimitern: morning, I updated my branch as suggested, could you please take another look? [09:32] frankban, yes, I'm already looking [09:32] frankban, thanks for doing all that so quickly btw! [09:33] ty [09:33] frankban, 99% of the PR looks great, I just want to double-check TestMachinePrincipalUnits as I've seen it fail on ppc64el recently due to some sort ordering [09:34] dimitern: sortedUnitNames failing on ppc? [09:35] sounds weird [09:35] frankban, nope, sorry - run-unit-tests-trusty-amd64 build #2145 was the one that failed [09:35] frankban, http://juju-ci.vapour.ws:8080/job/run-unit-tests-trusty-amd64/2145/console [09:38] frankban, it seems the culprit is line #1077 - units[3], err = s3.AllUnits() [09:39] frankban, so I was thinking if I could suggest a fix for you to include there [09:40] dimitern: go ahead [09:41] frankban, will do, once I can see where's the issue and reproduce it :) [09:41] dimitern: heh, ok ty [09:42] guys - a non Juju related question here - is "sed" the best way to edit conf files via a bash script? [09:49] dimitern: can it be something related to units slices mutated by append? should we reslice in sortedUnitNames(append(a.units, a.subordinates...))? [09:52] frankban, perhaps, still looking [09:57] dimitern: TheMue: dooferlad: making coffee, be with you in a couple of minutes [09:58] voidspace: ok [09:58] voidspace, ok [10:00] team standup [10:01] TheMue, voidspace, dooferlad, oh, yeah - it's team standup now and ours after [10:05] gsamfira: ping [10:05] oh yeah [10:08] firefox crash again, yay [10:16] axw: team meeting? [10:16] wallyworld: bollocks, be there in a minute [10:16] mgz: did you want to attend the Juju team meeting ? [10:26] frankban, you've got a review BTW [10:27] dimitern: ty [10:34] axw: thanks for review, will look real soon [10:35] TheMue: dimitern: dooferlad: standup? [10:35] voidspace: works for me [10:35] voidspace: sounds good for me [10:35] voidspace, TheMue, dooferlad, I need 5m [10:36] ok, I'll go make more coffee [10:36] aka one cigarette ;) *scnr* [10:36] but go ahead :) [10:42] voidspace, dooferlad, ok, let's do it [10:43] omw [11:00] dimitern: done, can I $$merge$$ it? [11:03] frankban, I'm having a last look [11:03] frankban, go for it, and thanks again! [11:04] dimitern: ty, should I have to propose it also against 1.21 and 1.22? [11:04] gsamfira: ping! [11:04] Muntaner: pong [11:06] frankban, yes, please, but in both 1.21 and 1.22 my patch has landed, so you'll need to do some cherry picking [11:07] dimitern: should we land the revert patch first? [11:08] frankban, I don't think it's needed, as your patch builds on top of it anyway [11:14] hi gsamfira :) [11:14] just installed the Windows Server 2012 R2 evaluation by cloudbase [11:14] the VM is in the same subnet of Juju VMs [11:14] now... how can I interact with this VM with Juju? [11:14] or am I confused about what I can do? [11:19] Muntaner: a few basics first. Juju has several "providers" it can use to create a juju environment (manual, local, MaaS, OpenStack, etc). The manual provider attempts to connect to an already installed machine and add it to juju. This provider however is not yet supported on windows. It will be in the (hopefully) not to distant future, just not yet. [11:20] Muntaner: if you used the image I pointed you to, that means you have an Openstack environment all set up [11:20] correct? [11:20] gsamfira: yes, everything is working fine atm [11:21] downloaded the qcow2 image of Windows, created a VM in the same subnet of juju VMs [11:21] juju actually is bootstrapped on this OpenStack environment [11:21] cool [11:21] (which is a private cloud, basically) [11:21] if you need screens/logs, ask me whatever it's needed :) [11:22] have you done the whole juju-metadata generate-image song and dance? Or are you using the official simplestream feed? [11:23] gsamfira: did nothing with juju :/ just downloaded the image, added it to OpenStack via glance, and created a VM with the image in it [11:24] wallyworld_: review can wait for tomorrow, but FYI http://reviews.vapour.ws/r/923/ [11:24] gsamfira: I did that stuff in the past days, to bootstrap my OpenStack with Juju [11:24] axw: np, will try and look [11:24] and basically I used an Ubuntu Cloud 14.04 image to do all the stuff [11:24] Muntaner: okay. So you have not yet deployed a massive indispensable infrastructure using juju yet :). There is one last hoop to jump through to get windows in. And that is juju agent and simplestreams that includes a Windows image [11:26] gsamfira: afraid I don't know how that works: ( [11:26] Muntaner: no worries. I can guide you through it. [11:26] I just need to get a handle on what you already have [11:26] gsamfira: ask me whatever you need to know :) [11:32] gsamfira: background. We set up a private OpenStack cloud, with all of the components installed on a single server. We have everything working (cinder, glance, swift, etc). Then, we bootstraped Juju on this Openstack. After some problems we got with metadatas (I opened a launchpad about it), also this step has been successfull, and I have Juju working on this Openstack. Now, I wanted to understand how Juju and Windows machine are r [11:34] so first things first. Lets get you the widows agent tools. [11:34] I'm guessing you are using 1.21.2 ? [11:35] 1.21.1-trusty-amd64, gsamfira [11:35] perfect. Building you a windows binary now [12:20] dimitern: 1.22 backport https://github.com/juju/juju/pull/1594 [12:21] frankban, looking [12:23] frankban, LGTM [12:23] dimitern: thanks, shipping it [12:25] frankban, +1 [13:08] so I am thinking in the rsyslog worker there is the handle config changes, that reacts based on new state servers getting added, so I think I can force the recreation of the rsyslog server certs there and that should fix up the connection issues [13:09] re: https://bugs.launchpad.net/bugs/1417875 [13:09] Bug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority [13:09]

[13:35] * dimitern steps out for 1/2h === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 === kwmonroe_ is now known as kwmonroe [16:15] dimitern: when all the pieces are in place, what's the command line incantation to create a new container on a machine? [16:21] voidspace, sorry, in a meeting, bbiab [16:21] dimitern: no problem, dooferlad remembered anyway [16:21] +1 [16:21] phew, first restructured test works. but nested map[string]interface{} for marshaling tests are a pain in the ass [16:22] * TheMue needs to write some helper [16:31] TheMue, consider using something like type M map[string]interface{} + helpers around it in the tests [16:31] dimitern: that's how I changing it currently ;) [16:31] TheMue, cheers :) [16:33] voidspace, so I'll propose the PCII() API tomorrow as it happens [16:34] This is why you shouldn't use map[string]interface{} in the first place :) [16:34] natefinch: +1 [16:34] natefinch, :) if you can avoid it, yeah [16:35] natefinch, however writing generic json serialization tests with strings like `....25 lines later...`[1:] is even worse [16:35] ah, this looks definitely better [16:36] :) I had a feeling it will [16:36] ok [16:36] I'm outta here [16:36] dimitern: yeah, strings are even worse [16:36] g'night all, see you tomorrow ;) [16:36] dimitern: cu [16:36] dimitern: take a look at that PR tomorrow? [16:37] katco, oh, sorry - I promise (bookmarking now) [16:37] dimitern: no worries at all... have a wonderful evening :) [16:37] dimitern: here is the 1.21 backport: https://github.com/juju/juju/pull/1595 could you please take a look? (slightly different code base there) [16:37] katco, thanks! [16:38] frankban, ok, I can take one last review I guess :) [16:38] dimitern: i see how it is! haha jk ;) [16:38] (for a backport I reviewed already..) [16:39] dimitern: ty [16:40] dimitern: oh, didn't notice you were done for the day, sorry [16:40] voidspace: FYI, the webhook worked for PR 1588 yesterday when I tried (after the RB downtime) [16:40] frankban, np [16:40] ericsnow: cool, good to hear [16:40] frankban, ship it! [16:41] dimitern: thanks! and have a nice evening [16:41] ta! [16:45] sinzui: how do I spell that https://github.com/juju/juju/pull/1595 fixes that bug? [16:47] frankban, include fixes-1420403 [16:47] frankban: $$fixes-1420403$$ I believe [16:47] sinzui: in the description? [16:47] frankban: in the merge comment [16:47] you just need the string fixes-1420403 in a comment somewhere, and then the standard $$merge$$ will work [16:48] frankban: doesn't need to be in the $$ but it's common [16:48] ^ [16:48] ty === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 === liam_ is now known as Guest13534 === kadams54 is now known as kadams54-away === kadams54-away is now known as kadams54 === kadams54 is now known as kadams54-away [18:54] is anyone familiar with the WADL format? [18:57] right, going jogging [18:57] probably back around in a bit [18:57] have fun voidspace [19:00] katco: thanks, good luck with the WADL :-) [19:00] voidspace: hehe thanks [19:03] abentley, my azure delete script says juju-functional-backup-restore-tyx8o23ojf is about 24 hours old. If you agree via the console it is more than 12 hours old, I will run the script in real [19:04] sinzui: that last sentences terrified me [19:05] perrito666, I am dealing with azure instances being left behind after destroy env. and since the Azure console doesn't work well with chrome or firefox, I am scripting an executioner [19:05] the onlypart that terrified me is backup-restore :p [19:06] btw, just to fill you with expectancy, I PRd the last piece of the new restore [19:07] thank you perrito666 [19:08] I think the restore test job dies because of azure. we never actually bootstrapped http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/functional-backup-restore/2180/console [19:08] well, I can see we aren't running any env like that, so I will use my new axe [19:12] who here knows anything about multiwatcher? [19:20] perrito666: a very little bit... [19:20] git says I am looking for francesco banconi [19:20] perrito666: that's frankban who's EOD working with dimiter [19:21] perrito666: ah === kadams54-away is now known as kadams54 [19:21] rick_h_: tx, ill try to catch him in my AM [19:21] perrito666: we use it on the GUI to track all the things [19:22] rick_h_: I assume you mean megawatcher and not frankban :p [19:22] perrito666: yea pretty much :) [19:23] "so frank, here is the debug log, you read it all and let us know when things change" [19:23] for everyone running a gui, hope you read fast [19:24] lol [19:25] * perrito666 sees a chance to "import" a new dell for him.. anyone got one of the new xps13 and ran linux on that? [19:25] I kind of lack return policy where I am :p [19:27] perrito666: google will help, or try posting on warthogs. I think the XPS laptops are generally pretty far up the chain of laptops that Linux geeks want support for (and that Dell wants linux support for), so usually they work pretty well pretty soon after release. [19:31] sinzui: looking... [19:32] sinzui: Not seeing it. Did you go ahead? [19:34] abentley, I did. I found the job that created it and it was 24 hours old [19:41] now, this is rather frustrating: Error: entity mismatch; got len 1; want 1 [19:44] perrito666: uh oh.. which test? [19:45] storeManagerStateSuite.TestStateBackingGetAllMultiEnv [19:46] perrito666: hmm - just makes me nervous because I was in that general area recently [19:47] jw4: I know, but this broke something that its still not prd [19:47] so worry not, alwhough the test's output is all but helpful, I might fix that [19:47] perrito666: lol [19:47] whew [19:51] perrito666: interesting... it looks like setUpScenario returned only one entity instead of two (my naive reading of it) === liam_ is now known as Guest73606 === kadams54 is now known as kadams54-away [19:54] perrito666: units not entities [19:55] jw4: ? [19:55] perrito666: n/m -- I'm chasing the squirrel of understanding your error [19:56] jw4: you should see my codebase for it [19:56] :) [19:56] perrito666: heh [19:57] I have a big change in the middle that breaks Unit.SetStatus and Unit.Status [19:57] perrito666: that explains it [19:57] jw4: buuut [19:57] there is a deepequals for 2 maps [19:58] and then, if that fails, you tell me their lenght missmatches, assuming that the only thing that can fail is getting different lenghts [19:58] perrito666: that's making more sense as I look at the trunk version of assertEntitiesEqual too [19:59] although it looks like there should be more descriptive information after that first message [20:00] but logged at INFO instead of ERROR [20:00] jw4: there is, but its Logf [20:00] which sucks in my opinion [20:00] perrito666: +1 [20:10] abentley, a newly stuck machine in azure [20:10] * sinzui get's his new axe [20:12] grc does not work with go's log output... what have I dont to deserve this monochromatic nightmare [20:53] sinzui: do you know why ssl-hostname-verification could be not working for canonistack? [21:01] perrito666, no, I never worked that out [21:01] sinzui: that means this is a known issue? [21:02] perrito666, it is and I am looking for the bug. === kadams54 is now known as kadams54-away [21:28] jw4: ok I might ask for your help [21:28] :) [21:29] do you know how does *watcher prompt for status changes on units? [21:36] perrito666: ah [21:36] gimme a sec [21:40] perrito666: if I understand your question... allWatcherStateBacking has a method Watch(in chan<- watcher.Change) which sets up a watcher for each collection it's watching [21:40] jw4: tx a lot [21:40] in this case a backingUnit [21:42] bbl [21:45] who created juju-backup ? [21:45] marcoceppi: thumper? [21:45] nope [21:46] marcoceppi: the plugin was created by a team effort [21:46] just over a year ago [21:46] while I was on leave :) [21:46] thumper: it's not a very good plugin, it doesn't have a --description or --help flag and breaks juju help plugins [21:46] ;) [21:48] is it in the juju-core repo? [21:48] I'll submit a patch [21:48] not sure... maybe [21:48] marcoceppi: yes [21:48] marcoceppi: cmd/plugins/juju-backup/juju-backup.go [21:48] cmd/plugins/juju-backup [21:49] thumper: well, interesting. the juju-backup I have is a bash script [21:49] marcoceppi: yep, that's the one [21:49] there is no .go at the end [21:49] oic [21:49] oh, yep :) [21:50] I was thinking of restore [21:50] * marcoceppi goes for is 3rd commit to core [21:50] well [21:51] it's kind of moot now, why even maintain the juju-backup command if it just shells out to core again? [21:55] marcoceppi: juju-backup is a bad plugin for a lot more reasons than its description and help ;) [21:56] marcoceppi: we're replacing the old backup command with a built-in command. It is a lot more maintainable [21:56] sorry, gotta run, but ericsnow and perrito666 are the backup and restore masters :) [21:59] well, it's worse than that, everytime anyone runs juju help plugins, the backup plugin is executed [21:59] so it's just dumping backups everywhere [21:59] it's a pretty big bug, but if 1.21 fixes this (by removing the old logic) then fine [22:00] but 1.21-beta4 still executes the plugin from what I can tell [22:02] 1.22 is when we added the new backup command (and updated the plugin) [22:02] marcoceppi: we kept the plugin around for backward compatibility [22:03] marcoceppi: but it should probably still handle --help and --description [22:03]