/srv/irclogs.ubuntu.com/2015/02/04/#juju-dev.txt

sinzuijw4, I think that is a question for dimitern. I think the rule is lxcbr0 is not for general consumption.00:00
sinzuiPASS.00:00
jw4woot00:00
anastasiamacsinzui: jw4: great!00:00
jw4sinzui: yeah - most of thi scode seems to be dimitern related00:00
sinzuijw4, anastasiamac I am down grading the bug and re-describing the madness00:00
jw4sinzui: lol00:00
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: None
anastasiamacsinzui: jw4: looks i missed all the fun!00:00
jw4anastasiamac: o/00:01
anastasiamacsinzui: great find and thank u for master00:01
anastasiamacjw4: o/00:01
anastasiamacjw4: thnx for being so efficient today  :D00:01
sinzuiall is open00:01
perrito666davecheney:00:02
jw4anastasiamac: hehe00:02
perrito666github says the same than reviewboard00:02
perrito666davecheney: for instance https://github.com/juju/juju/pull/1532/files#diff-a722ce6455c045798bd7ef9db08e0ceaR1100:02
sinzuioh, now I need to rethink how we fix the the vivid ppc64el packaging bug without my stilson-0700:03
perrito666davecheney: ah but the empty lines are not there00:03
perrito666ok, rb shows garbage for your change00:03
anastasiamacaxw: standup01:18
axwdoh, thanks01:18
waiganithumper: when your ready, destory env: http://reviews.vapour.ws/r/627/01:44
bradmanyone about?  we're having a weird juju machine agent issue where its dying with "panic: runtime error: slice bounds out of range"01:54
=== kadams54 is now known as kadams54-away
TheMuemorning08:36
dimiternTheMue, morning08:40
TheMuedimitern: hey, how is your time in the warm southern sun so far? ;)08:40
TheMuedimitern: here we've right now -3° and I'm happy I not have to leave home08:41
dimiternTheMue, not so warm today :) it's raining lightly and it's cloudy, but yesterday it was hot08:41
dimiternTheMue, it still is 18 degrees though08:42
TheMuedimitern: cloudy is good, right environment for juju. but juju is also hot, hmm? :D08:42
eagles0513875_dimitern: hey :) hows the sprint08:43
TheMuedimitern: thanks for your info on the refactoring btw, good to know. I'm already continuing and wanted to make sure that it isn't useless08:43
dimiterneagles0513875_, hey :) it goes swimmingly so far09:05
dimiternTheMue, :)09:05
dimiternTheMue, cheers09:05
dimiternvoidspace, o/09:10
dimiternvoidspace, did you say you're to file a swap day for friday?09:10
voidspacedimitern: I'll be taking a swap day Friday, yes09:12
voidspacedimitern: o/09:12
voidspacedimitern: read through your email, all sounds good09:12
lazyPowerdimitern, are you present?09:16
dimiternlazyPower, i am indeed09:17
dimiternvoidspace, cheers, please file a request so I can approve it ;)09:17
lazyPowerdimitern: can you poke your head in #juju on canonirc? I'm starting to traverse into juju core territory that I dont understand while helping debug a customer deploy (this shouludn't take long - as its basically a "i know or dont know" situation)09:18
dimiternlazyPower, sure09:19
voidspacedimitern: ok, will do09:20
voidspacedimitern: I'm currently still looking at the proxyupdater issues09:21
voidspacedimitern: a condition of the LGTM I got was to investigate *why* the current "first run" logic (that used to set environment variables) is faulty09:21
voidspacedimitern: it *looks* sound, like it should have worked09:22
voidspacedimitern: "first" is set to "true" in proxyupdater.New() when the worker is created09:22
dimiternvoidspace, I can confirm that the "first run" logic is pure bs - out of omission09:22
voidspacedimitern: in SetUp onChange is called ("first" should be true) and *then* "first" is set to false09:22
voidspacedimitern: omission?09:23
voidspaceSetUp is called in loop() of the NotifyWorker09:23
voidspaceon that call "first" should be true, and the proxy settings are fetched from api.EnvConfig09:23
eagles0513875_dimitern: glad to hear it :)09:23
dimiternvoidspace, yeah - it was overlooked09:23
dimiterneagles0513875_, :)09:24
voidspacedimitern: I've read through the code, it seems sound enough on a casual reading09:24
voidspacedimitern: I'm going to put some tracing code and work out what's happening09:24
voidspacedimitern: shouldn't take long09:24
dimiternvoidspace, sgtm, sorry I'm a bit distracted on multiple fronts right now09:24
voidspacedimitern: I'm going to create a docker image for the manual provider though - so I can repeat experiments09:24
voidspacedimitern: no problem09:24
voidspacedimitern: and I'd also like to fix the charmrevision worker so we don't see that error in the logs09:25
voidspacedimitern: that may mean moving where the proxyupdater is started09:25
voidspacedimitern: just letting you know, so long as you're happy with me working on this no need to internalise the details09:25
voidspacedimitern: and then I'll switch to the work you suggested09:25
dimiternvoidspace, thanks, it seems to me you have all under control there09:26
voidspaceheh09:26
voidspacedimitern: being in control of yourself is always the first step...09:26
dimitern:D indeed09:28
=== ashipika1 is now known as ashipika
perrito666morning10:17
voidspacejamestunnicliffe: here10:20
voidspacejamestunnicliffe: ubuntu image pulled10:20
jamestunnicliffegreat. Hangout in https://plus.google.com/hangouts/_/canonical.com/juju-sapphire?authuser=0 ?10:21
voidspacejamestunnicliffe: sure10:21
voidspacejamestunnicliffe: http://pastebin.ubuntu.com/10051327/10:29
voidspacejamestunnicliffe: http://pastebin.ubuntu.com/10051372/10:32
dimiternvoidspace, hey, I've backported your fix for 1.22 http://reviews.vapour.ws/r/859/ - nothing's changed, but I'd appreciate if you can have a look, test it and approve it if it passes, but it can wait for tomorrow - no need to switch to it now10:33
voidspacedimitern: ok10:33
dimiternvoidspace, thanks!10:33
voidspacedimitern: just setting up docker images for easy testing of manual provisioning10:33
dimiternvoidspace, does it make it easier to deploy/debug/redeploy?10:34
voidspacedimitern: yes10:34
voidspacedimitern: because "destroy-environment" with manual provider doesn't leave a clean environment for a re-deploy10:34
voidspacedimitern: which is probably worth a bug report10:34
dimiternvoidspace, sweet, I'd like to hear about it next week so I can try it out10:34
voidspacedimitern: the docker image should make it very easy to repeat a manual deploy10:35
dimiternvoidspace, that indeed seems worthy of a bug report10:35
voidspacedimitern: but creating a new kvm image for manual provisioning is heavyweight and docker should be easier10:35
jamestunnicliffevoidspace: http://pastebin.ubuntu.com/10051436/10:36
jamestunnicliffevoidspace: docker build -t juju-thing .10:36
dimiternvoidspace, +110:36
dimiternvoidspace, we can try using snappy for the same process at some point as well10:36
voidspacedimitern: heh, cool10:37
dimitern;)10:38
dimiternok, gtg for now10:38
TheMuedimitern: have you seen snappy in action in capetown?10:42
voidspacejamestunnicliffe: what should that file be called?10:50
jamestunnicliffevoidspace: Dockerfile10:50
voidspacejamestunnicliffe: thanks10:54
voidspacewell, it's running...10:55
voidspacejamestunnicliffe: ah yes, ufw issues10:56
jamestunnicliffevoidspace: looks like it is best to do the firewall stuff outside of Dockerfile by telling docker itself what to allow in/out. Will let you know how soon.10:56
voidspacejamestunnicliffe: http://askubuntu.com/questions/28215/how-can-i-fix-the-iptables-error-message-unable-to-initialize-table-filter10:57
jamestunnicliffevoidspace: https://docs.docker.com/articles/networking/#between-containers should have the answers10:57
voidspaceYou need to load a kernel module for enabling the filter table. Run the next command as root:10:57
voidspacemodprobe /lib/modules/$(uname -r)/kernel/net/ipv4/netfilter/iptable_filter.ko10:57
voidspacejamestunnicliffe: ok10:57
voidspacejamestunnicliffe: cool10:57
jamestunnicliffevoidspace: the container is running on the host kernel, so you can't just insert stuff into the kernel :-)10:57
voidspaceright10:58
voidspacejamestunnicliffe: https://github.com/docker/docker/issues/125110:59
voidspaceah, that's using ufw on the host11:01
jamestunnicliffevoidspace: you may be best off, for now, creating a KVM / VirtualBox / whatever machine and setting up everything but Juju, then snapshotting it.11:13
jamestunnicliffevoidspace: that way you can always restore to the snapshot after playing11:13
jamestunnicliffevoidspace: this stuff with Docker is possible, but a headache.11:14
mgzvoidspace: have you got a moment to triage a rsyslog cert issue? bug 141787511:35
mupBug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority <canonical-bootstack> <regression> <juju-core:New> <https://launchpad.net/bugs/1417875>11:35
* jamestunnicliffe goes for lunch11:59
voidspacemgz: I can look, sorry for the delay - midwife visit12:33
voidspacejamestunnicliffe: ok, cool - thanks12:34
voidspacejamestunnicliffe: virt-manager has cloning but not snapshotting12:34
voidspacejamestunnicliffe: I'll read up on it to check if I need to keep a "clonable" image around, probably...12:34
voidspacemgz: you're probably better off with wwitzel3 on that one12:36
voidspacemgz: he worked on x509 certs for rsyslog not-that-long-ago12:36
voidspacebug 141787512:36
mupBug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority <canonical-bootstack> <regression> <juju-core:New> <https://launchpad.net/bugs/1417875>12:36
mgzvoidspace: I figured just thought I'd try you before he was on :)12:42
voidspacemgz: it looks like a serious regression if it's reproducible13:09
mgzvoidspace: pretty sure there has to be something else going on, logging in general in 1.21 has worked13:10
voidspacemgz: right13:12
voidspacemgz: I'm not familiar with the moving parts (except vaguely) around the certificate generation13:13
aznashwan1hey guys; I'm currently working on the systemd support and have a question:13:29
aznashwan1in your opinion, which would be the based place to place the unit file (bonus thanks if you also point me to how I could get that path :D13:30
aznashwan1best*13:30
=== cmars` is now known as cmars
sinzuibogdanteleaga, gsamfira: I have placed mongod.exe in the sys path and run the win unit tests. The job could complete. What do I need to setup in the env/go env to get the tests to match your experience? http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/run-unit-tests-win2012-amd64/29/consoleText15:50
bogdanteleagasinzui: first of all you should probably use one of my branches to test it since not everything is merged15:52
sinzuibogdanteleaga, I can arrange that15:54
bogdanteleagasinzui: https://github.com/bogdanteleaga/juju/tree/try-ci15:56
bogdanteleagasinzui: try this one for now, it should work15:56
sinzuibogdanteleaga, work as in pass?15:57
bogdanteleagasinzui: yes15:57
sinzuibogdanteleaga, fab!15:58
bogdanteleagasinzui: actually, I think you need a branch in juju/testing as well15:58
bogdanteleagasinzui: that might be a bit harder to do15:58
sinzuibogdanteleaga, I assume goarch is amd64. but what is CGO_ENABLED in yours?15:59
sinzuime sees conflicting info on the windows machine15:59
sinzuibogdanteleaga, I will use the make-release-tarball script to make a go tree to test with16:00
bogdanteleagasinzui: yes, it's 64bit, no modifications to go16:02
perrito666natefinch: wwitzel3 standup?16:03
sinzuithank you bogdanteleaga16:03
bogdanteleagasinzui: this is the one in testing I was talking about https://github.com/bogdanteleaga/testing/tree/skip-chmod16:07
sinzuithank you bogdanteleaga16:08
bogdanteleagasinzui np, tell me how it goes16:19
aznashwan1ericsnow: you there?16:41
ericsnowaznashwan1: OTP16:41
aznashwan1for the love of me I have no idea what OTP means :))16:43
natefinchon the phone16:43
aznashwan1natefinch: thanks, you're better than google :D16:43
natefinchhaha16:43
natefinchtook me a while to figure it out too16:43
=== kadams54 is now known as kadams54-away
perrito666ericsnow: nice changes on the patch17:05
ericsnowperrito666: oh good17:05
ericsnowperrito666: wish I'd known about juju/paths when writing backups17:06
ericsnowaznashwan1: back17:06
aznashwan1ericsnow: so, I had a couple of quick heads-up's17:07
* perrito666 running the whole suite to have a fully passing one in more than a week17:07
ericsnowaznashwan1: k17:07
aznashwan1ericsnow: firstly, I used the go-systemd unit to create the service file and, just as an observation, the order of the options is not consistent from one serialization to another17:08
ericsnowaznashwan1: ah, good to know17:09
aznashwan1ericsnow: secondly, doing a dbus.Conn.ListUnits() only lists running/stopped/idle etc.. units, and does not help at all in determining whether a unit is enabled or not...17:10
aznashwan1ericsnow: [stopped as in they actively ran and are now done]17:11
aznashwan1ericsnow: i've asked the coreos guys for some help, and am waiting for their response on this...17:11
ericsnowaznashwan1: ah17:11
ericsnowaznashwan1: however, aren't those the units we want?17:12
ericsnowaznashwan1: glad you got in touch with the coreos guys17:12
aznashwan1ericsnow: not in the context I was indenting on using it sadly D:17:12
aznashwan1ericsnow: anyhow, on a brighter note, I've only just seen your PR on the init systems, and think it's looking great17:12
aznashwan1ericsnow: was gonna' ask if you wanted me to get to putting my work over yours, so as to converge eventually17:13
aznashwan1ericsnow: because what I was doing up to today would not fit too cleanly there as is D:17:13
ericsnowaznashwan1: wwitzel3 and I did something like that yesterday (based on your patch)17:14
ericsnowaznashwan1: we should sync up on this17:14
aznashwan1ericsnow: well I'll get to putting it on top of yours then to spare you guys of the extra work17:15
aznashwan1ericsnow: last couple of questions:17:15
aznashwan1ericsnow: from what i've seen the old interface is not preserved (so the confusing Exists() and Matches() functions are gone), do you have any intention to still have both (and by the same definitions?)?17:16
ericsnowaznashwan1: there is just IsEnabled17:17
ericsnowaznashwan1: the equivalent to Exists is the service.Services.Check method17:18
ericsnowaznashwan1: but an InitSystem implementation doesn't have to worry about that17:18
aznashwan1ericsnow: good, will be getting to adapting IsEnabled with go-systemd the minute I find out how17:18
aznashwan1ericsnow: and I'll be doing the InitSystem implementation of systemd tomorrow17:19
ericsnowaznashwan1: check out what wwitzel3 and I did yesterday: https://github.com/wwitzel3/juju/compare/ww3-systemd17:20
aznashwan1ericsnow: last thing, I swear: could you please tell me where I may get the place where you want me to put the .service (and ExtraScript optionally) (ideally point me to the actual package/function where I may get it)17:20
ericsnowaznashwan1: that was the motivator for the code in service/managed.go :)17:21
aznashwan1ericsnow: i see17:22
ericsnowaznashwan1: the service.Services implementation takes care of that so the InitSystem implementations don't have to know about it (other than as the second argument to Enable)17:22
aznashwan1ericsnow: great, well then I hope to get initsystem/systemd adapted tomorrow17:25
ericsnowaznashwan1: awesome17:25
aznashwan1ericsnow: shall I keep rebasing over the branch the PR was created on, or this there some other one you are actively working?17:25
aznashwan1ericsnow: [your* PR]17:26
ericsnowaznashwan1: for the init system stuff it's my init-service-cleanup-local branch17:27
aznashwan1ericsnow: gotcha'. thanks for everything and see you soon :D17:27
ericsnowaznashwan1: keep in mind that it depends on https://github.com/juju/testing/pull/48/ and https://github.com/juju/utils/pull/10817:27
ericsnowaznashwan1: np, thanks for working on this17:28
mgzwwitzel3: are you around? can you triage bug 1417875 or ask for any more information we'll need17:32
mupBug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority <canonical-bootstack> <regression> <juju-core:New> <https://launchpad.net/bugs/1417875>17:32
wwitzel3mgz: ok, taking a look17:40
perrito666how do you determine a backend facade version from api calls?17:59
mgzwwitzel3: thanks17:59
natefinchif your export_test file is over 200 lines... maybe you should just make your tests internal :/19:21
jw4natefinch: lol19:21
perrito666I hate export_test pattern :(19:25
natefinchperrito666: me too19:27
natefinchperrito666: the only time I think it's valid is when it's needed to avoid circular imports.19:28
katcoperrito666: natefinch: i will just +1 that sentiment19:37
katconatefinch: if unit tests are really what we're going for, it becomes busy-work19:38
katconatefinch: ps, are you alive up there in boston? yeesh!19:38
natefinchkatco: haha yeah... gotten 44" of snow in the last week and more on the way.  Looks like we moved to northern Canada or something19:39
katconatefinch: craziness19:39
katconatefinch: i love snow, but i understand it's causing a few issues =/19:39
natefinchkatco: at least we're (mostly) set up to deal with it.  The roads are ok, the main problem is where to put all the snow that you clear off the roads.  It's not too bad in the country, but in downtown, there's just no place to put it all19:40
katconatefinch: shovel it into trains and ship it to the west coast!19:40
natefinchkatco: yeah. definitely glad I don't have to commute into Boston anymore... heard other people's commutes that were like 2.5 hours longer than normal (each way!)19:40
katconatefinch: yeah i saw that. commuting = largest waste of ones life.19:40
natefinchyep19:41
perrito666dont you people cancel work for commuters on excessive snow days?19:44
katcoperrito666: yes, but companies vary19:44
katcoperrito666: it's usually individual managers who make the call for their team unless it's clearly unsafe and then the company might issue a site-wide communication19:45
perrito666well, its a good thing I decided to go for a career in CS19:45
voidspaceg'night all19:45
perrito666voidspace: night19:45
katcovoidspace: sleep well19:46
voidspaceperrito666: o/19:46
voidspacekatco: plenty to do before sleep time :-)19:46
voidspacebut thanks, and I will...19:46
katcovoidspace: fair enough!19:46
natefinchperrito666: when it's really bad they do call for travel bans.... but that's usually just during the storm.19:46
perrito666that wonderful feeling when all tests pass19:46
perrito666man I get so much mail from prs that my phone sounds like mario bros on steroids19:47
thumpersinzui: sha'ping19:55
mgzthumper: does that mean he has to hash himself?19:55
thumperhaha19:56
thumpernah19:56
thumpersinzui: I have been thinking about the manual provider and ci upgrade bug with the 10.0.3.1 address issue19:56
thumperI'm pretty sure I know what is going on19:56
thumperI mentioned it in the bug19:56
thumperhowever19:56
thumperthe new networking code from Dimiter's team has started enumerating all the network devices19:57
thumperto get their ip addresses19:57
thumperlxcbr0 by default gives 10.0.3.119:57
thumperthis is classified as  a class A private network19:57
thumperand then considered cloud private19:57
sinzuiand juju prefers A private networks?19:57
thumperand then the 'best address' to give out19:57
thumperwell, there are two class A ones19:57
thumperI guess it picks the first?19:57
thumperor last?19:57
* thumper shrugs19:57
thumperI think the solution is to blacklist some nic names19:58
thumperlike 'lxcbr0' and 'virbr0'19:58
thumperneither of those will ever be useful addresses19:58
thumperprobably lo* too19:58
sinzuithumper, think this issue is the cause of bug 1417308 and bug 141692819:58
mupBug #1417308: Juju-ci3 cannot upgrade to 1.21.1 <api> <ci> <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1417308>19:58
mupBug #1416928: juju agent using lxcbr0 address as apiaddress instead of juju-br0 breaks agents <api> <lxc> <network> <juju-core:Triaged> <juju-core 1.21:Triaged> <juju-core 1.22:Triaged> <https://launchpad.net/bugs/1416928>19:58
thumperyup19:59
thumperI believe this worker was added in 1.2119:59
sinzuiIt certainly was19:59
thumperdave suggests that we look at network properties on the adapter rather than names20:00
sinzuithumper, So I hope core cna provide a fix. I can make all these bugs a dup of a new bug based on your insight20:00
thumperso look for the loopback flag20:00
thumperor bridge20:00
thumpersinzui: we need to fix it, but I'm not entirely sure how20:01
thumperI'd like william and dimiter and john to talk about it in cape town20:01
sinzuithumper, agreed20:02
mgzwwitzel3: did you have a chance to look at bug 1417875 yet?20:02
mupBug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority <canonical-bootstack> <regression> <juju-core:New> <https://launchpad.net/bugs/1417875>20:02
hatchcan someone tell me if the existence of the JUJU_ENV_UUID env attribute is an indicator of the api supporting it in the path?20:34
natefinchhatch: it looks like it's only set as hookvars ... charms can read it, but none of the rest of juju's code seems to read it20:58
natefinchhatch: not sure if that answers the question20:58
hatchnatefinch: hmm I was under the impression that the api will require it soon so I was implementing it in the GUI20:59
natefinchhatch: the API will require it, yes, but that's different than the environment variable20:59
hatchright - but we need to grab the uuid from the charm to supply it to the GUI21:00
natefinchhatch: maybe I misunderstood the question21:00
hatchso I was wondering if I can use the existance of that attribute to be a flag to the GUI that it's now supported in juju21:00
hatchthe gui can be deployed to many different juju versions21:00
hatchnatefinch: basically something needs to say to the GUI 'use the uuid' or 'dont use the uuid'21:01
hatchthe uuid environment variable has been defined for at least 5 months according to the source - I'm not sure if the API support follows the same21:02
natefinchhatch: that sounds kind of a hackish way to determine the api version... can't you just query the API for what version it is?21:03
hatchehh feature testing > version testing21:05
natefinchversion testing > depending on some random env variable and assuming you know what its existence or non-existence means :)21:06
natefinchhatch: thumper would probably be the guy to ask about the best way to go about this21:07
hatchwell I was just hoping to avoid writing the code to test with the uuid then fall back to w/o if it errors :)21:07
natefinchhatch: understood.... there's probably a check you can do, I'm just not sure what it is, but checking for that env var does not seem like the most reliable thing to use21:10
hatchyeah - unfortunately the api doesn't tell us what features it supports21:11
hatchYET!21:11
hatch:)21:11
hatchgranted we'd need to know that it supported the uuid to get that list first :P21:12
natefinchhaha21:12
=== kadams54 is now known as kadams54-away
natefinchI was going to say - how do you know if it supports asking what features it supports? :)21:12
hatchdamn egg/chicken21:12
hatchI prefer a chicken omlet anyways21:13
natefinchsomewhat harder to stir up, though21:14
* thumper looks up21:18
thumperhi hatch21:18
thumperhatch: if there is an env-uuid in the .jenv file, use it21:19
hatchthumper: I am in the gui charm's start hook so I have access to the environment variables21:19
hatchis that an indicator that the api supports requests with it?21:19
thumperhmm...21:19
thumperyes?21:20
hatchhaha21:20
hatchI can't find in the code where that url structure was added21:20
hatchso I can't compare to when the uuid env variable was21:21
hatchand then cross reference to some release # :)21:21
thumperhatch: I think it was 1.18 or 1.2021:21
hatchyou guys just do too much work21:22
thumper1.20 has it21:22
thumperand 1.21 is stable21:22
thumperyou can ask the version right?21:23
hatchwait....so even is unstable?21:23
thumperhatch: we don't do that now21:23
rick_h_hatch: you know what, we can just startif version > 1.2021:23
rick_h_no need to go back all the way to the first origin of it21:23
hatchthat works for me!21:23
thumperrick_h_: oh hai21:23
thumperrick_h_: how's cape town?21:23
rick_h_thumper: good stuff,   but crazy21:24
rick_h_thumper: missing you to drink with after the day :P21:24
thumperwe'll drink here for you21:24
thumperthat's ok right?21:24
rick_h_:)21:25
thumperrick_h_: you should be sleeping now right?21:26
hatchit's only 9:3021:27
rick_h_thumper: yea, should be.../me goes to bed21:27
hatchlol21:27
rick_h_just taking care of the team. reminding them to do reviews21:27
thumperthat's nice of you :)21:27
rick_h_hatch: hmm, add 2hrs21:27
hatchoh woops, timezone fail21:27
rick_h_night all, ty thumper for helping hatch out21:27
hatch^ and natefinch :)21:28
wwitzel3mgz: I hadn't fired up my maas in a while so I'm just working out some issues in it so I can reproduce21:28
thumperwwitzel3: I misread 'maas' for 'ass'21:28
lazyPowerthumper: how'd the mess demo go?21:29
wwitzel3thumper: lol21:29
thumperlazyPower: I heard it went real well21:29
hatchhaha21:29
mgzwwitzel3: ehehe21:29
lazyPowergooooooooood21:30
* lazyPower hi5's thumper21:30
* lazyPower resumes lurking now21:30
wwitzel3mgz: I'm wondering if somehow the permission of the issuer cert is off21:31
wwitzel3mgz: or if it is even there, soon as I get my setup bootstrapping again, I'll take steps to verify and reproduce and update the ticket. In the meantime I'll get it assigned to me, forgot to do that/21:32
mgzwwitzel3: thanks!21:35
jillrgot an HA juju deployment that was unplanned rebooted by the customer. been fighting with mongo corruption, respawns, and SocketExceptions. menn0 and gustavo had helped with a recovery earlier today, but it's gone bad again, member in FATAL.21:44
natefinchthumper: if you have time, I made a fix for the joyent issue you saw: http://reviews.vapour.ws/r/861/21:56
natefinchjillr: is it just a single member that's down?  If so, can you just kill it and then rerun juju ensure-availability to get back up to 3 state servers?21:57
jillrnatefinch: it has not been consistently just one node, no. also we are colocating other services on the machines so I'd have to dig thru the relations21:58
jillrrestarts of jujud-machine-# and juju-db have at times restored things, but only briefly21:58
jillrthere was a missing record it was barfing on earlier but we thought that was resolved21:59
jillror I suppose it'd be a missing transaction, sorry22:00
natefinchjillr: I'm at EOD, but thumper, axw, waigani, and davecheney are all near the beginning of their days, so hopefully they can help out.22:01
jillrnatefinch: cool, ta22:02

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!