/srv/irclogs.ubuntu.com/2015/03/10/#juju-dev.txt

perrito666cherylj: back00:00
menn0anastasiamac: i'll address those review comments and get those merged into the feature branch separately. thanks.00:03
anastasiamacmenn0: really? thnx :D00:03
cheryljhi perrito666, I'm looking at bug 1424069 where juju resolved fails with the error "ERROR unit "ubuntu/0" is not in an error state"00:06
mupBug #1424069: juju resolve doesn't recognize error state <regression> <resolved> <juju-core:Triaged by cherylj> <https://launchpad.net/bugs/1424069>00:06
cheryljand it looks like what is happening, is that when the install hook fails, the Unit Agent status gets set to error00:07
cheryljbut when we try to run juju resolved, it looks at the state of the Unit, not the UnitAgent00:07
perrito666cherylj: I ffed up that then.00:07
perrito666cherylj: it should in all cases use unitAgent00:08
perrito666unit should not be used yet00:08
perrito666cherylj: that particular behavior seems to be missing a test or it would have failed when I made the change00:08
cheryljperrito666: did you want to take this bug?  I can help out if you tell me what needs to happen, if you're running low on time :)00:10
perrito666cherylj: well it deppends is this alreay critical?00:12
cheryljyes00:12
perrito666how fun00:12
perrito666cherylj: what time is it for you?00:12
wallyworld_we have a couple of days00:12
wallyworld_1.23 is not going to be released until thursday most likely00:12
cheryljperrito666:  7PM00:12
perrito666cherylj: well its unfair that you are working so late because of me00:12
perrito666assign it to me00:13
cheryljperrito666: well it sounds like we have a little bit of time00:13
perrito666wallyworld_: yes, but his bug is most likely blocking CI00:13
cheryljperrito666: I do need to put my daughter to bed now, but I can still help out once she's down.00:13
wallyworld_oh, nothing is in the topic00:13
perrito666cherylj: could you point me to where you found the general error happening? ill grow some decent test and fix it00:13
perrito666wallyworld_: its critical, I assume it is locking00:14
cheryljthere's a testcase in the bug00:14
cheryljwhich I was able to reproduce easily on my local system00:14
wallyworld_perrito666: yeah, marked as critical regression, but topic not updated00:15
perrito666wallyworld_: I believe that is done by hand00:15
wallyworld_perrito666: it's late for you too, i can pick this up00:15
cheryljperrito666: I added some logging and found that before the regression, upon the install hook failure, the state of the unit was set to StatusError, so juju resolved worked fine00:15
cheryljThe current behavior is that upon the install hook failure, the UnitAgent status is updated to StatusError (and nothing ever changes with the status of the Unit)00:16
perrito666cherylj: that is expected behavior00:16
cheryljBut the juju resolved command still gets the status of the Unit to determine whether or not the unit is in an error state00:16
perrito666the issue seems to be the code checking if unit is broken00:16
cheryljperrito666: ok00:16
perrito666cherylj: tx a lot00:17
perrito666Ill go have dinner and fix it afterwards00:17
cheryljperrito666: np, let me know if there's anything else I can help with00:17
cheryljI'll bbl too00:17
wallyworld_perrito666: i've taken the bug, fixing now00:22
perrito666wallyworld_: aren't you a nice person?00:32
wallyworld_perrito666: sometimes :-) it's fixed, including a test, but i'm just looking at any other tests that may be needed00:33
perrito666wallyworld_: well, that test is going to be very useful on the change that I am practicing now00:34
perrito666you know the worse part? I need to change that back to unit00:39
wallyworld_sigh00:39
wallyworld_axw: a small one to fix blocker http://reviews.vapour.ws/r/1109/01:00
axwlooking01:00
axwwallyworld_: seems odd that we'd be checking the agent status, rather than the unit status... what's the distinction meant to be?01:03
wallyworld_axw: agent status is what we check now before any health related work01:03
wallyworld_moving forward that will change01:04
axwwallyworld_: I see. can you please add a comment to that effect?01:04
wallyworld_sure01:04
ericsnowcould I get another pair of eyes on http://reviews.vapour.ws/r/1102/?01:15
=== kadams54-away is now known as kadams54_
=== kadams54_ is now known as kadams54-away
axwanastasiamac wallyworld_: this one is ready for review now - http://reviews.vapour.ws/r/993/02:34
wallyworld_ok02:34
axwwallyworld_: I ended up taking out all of the implementation of the storage watching code in this branch, because it was ~2000 LOC all up :)02:35
axwnow about 60002:35
wallyworld_sure, i may duck out to get lunch and look after02:35
axwnps02:35
=== kadams54-away is now known as kadams54_
=== kadams54_ is now known as kadams54-away
=== kadams54-away is now known as kadams54_
=== kadams54_ is now known as kadams54-away
wallyworld_thumper: why do we have the agents starting in debug mode prior to determining what logging level to use?04:03
thumperhisterical raisins04:05
thumperwallyworld_: we didn't want to miss important bits at startup04:05
wallyworld_thumper: thought so. but that means we leak credentials :-(04:05
wallyworld_we can log apiserver request/response at trace level, but will also liekly need to look to suppress cloud init logging etc04:07
thumperwallyworld_: here's an idea04:07
thumperwhen we write out the service script, we look at the current log level for the juju module04:07
thumperif it is set to DEBUG or below, we start with that04:08
thumperotherwise start with INFO04:08
thumperwhat this means though04:08
thumperis that some agents may start with info and some with debug if changes are made during deployment04:08
thumperthe original idea was to have a defined, known starting point that was the same everywhere04:08
thumperthere is no technical reason why it can't be different04:08
wallyworld_i agree with that - i just wish we din't include credentials04:09
thumperthe original design went for consistency04:09
thumperwe are outputting everything04:09
wallyworld_the issue is that even if we start with info and then change to debug, we'll still leak04:09
thumperyes04:09
wallyworld_so messing with logging levels is only a band aide which is not that effective04:10
thumpercorrect04:10
wallyworld_sigh04:10
wallyworld_seems like it needs to be done properly04:10
thumperso... alternative options04:10
thumperdon't write out credentials in the log04:10
davecheney15:10 < thumper> don't write out credentials in the log04:11
davecheney^ yes, that04:11
thumperhowever, since we log all api traffic at debug04:11
thumperwe need to do one of two things:04:11
wallyworld_we need to be able to strip credentials but easier said than done04:11
thumperbe able to identify the values that are creds04:11
thumperor not write out all the traffic04:11
wallyworld_agreed, latter is flawed because sometimes you need all traffic04:12
wallyworld_without leaking04:12
thumperfor debugging yes, for normal users, no04:12
thumperI don't think it is feasible to say "we04:12
thumper"we'll log everything except credentials"04:12
wallyworld_well, users need it even if they don't know it04:12
thumperit doesn't make sense04:12
wallyworld_since users will upload all-machines.log and then expect us to fix their problem04:13
wallyworld_or we need to turn on debug logging04:13
wallyworld_so we do need to allow for verbose logging minus secrets04:14
thumperyep, that's hard04:15
wallyworld_for now need a quick fix - can log api server requests at trace level04:17
wallyworld_and look at not dumping cloud init script at startup unless also at trace04:17
wallyworld_well, can log api server request metadata at debug04:18
wallyworld_but not contents04:18
thumperseems like a reasonable approach04:21
wallyworld_i'll do that04:22
wallyworld_axw: rb not updating; this one fixes a 1.23 release issue with leaking creds https://github.com/juju/juju/pull/178205:00
axwlooking05:03
axwwallyworld_: done05:06
wallyworld_ty05:06
wallyworld_axw: i'm off for a bit to go see Wicked, won't be able to look at your latest PRs till later tonight after I get back, or maybe tomorrow morning06:00
axwwallyworld_: no worries. enjoy :)06:00
axwno great rush, I've got things to carry on with06:01
wallyworld_axw: wil do, if you are bored, there's also the storage provisioner one http://reviews.vapour.ws/r/1108/06:01
axwwallyworld_: sure, I'll finish off this branch then take a look06:01
wallyworld_no hurry of course :-)06:01
anastasiamacaxw: wallyworld_: i mite be able to look at some of these too :D06:02
axwanastasiamac: great, thanks06:02
wallyworld_great06:02
mattywmorning all08:16
Muntanerhello devs08:39
MuntanerI'm trying to use the load balancer charm HAProxy for my charm. I set up everything, but when I try to access to the public IP of the load balancer, I get a 503 Service Unavailable. How can I diagnose my situation?08:39
davecheneyrogpeppe3: oh pooh , you left08:46
rogpeppe3davecheney: no i didn't08:46
=== rogpeppe3 is now known as rogpeppe1
=== rogpeppe1 is now known as rogpeppe
mupBug #1430205 was opened: lxc template needs refreshing every 24 hours <juju-core:New> <https://launchpad.net/bugs/1430205>08:47
dooferladmorning! o/08:58
dimiterndooferlad, o/09:00
dimiterndooferlad, I've been reading your kvm notes from yesterday09:05
dimiterndooferlad, how did that reject rules appear in iptables? from libvirt?09:06
dooferladdimitern: I have no idea! I haven't tracked it down yet.09:07
dimiterndooferlad, also, have you checked the nat table? there should be an SNAT rule09:07
TheMuemorning o/09:07
dimiternTheMue, o/09:07
dooferladTheMue: o/09:07
dooferladdimitern: bother, I didn't save those :-( I will get a duplicate environment up and get back to you.09:11
dimiterndooferlad, cheers09:12
mupBug #1430220 was opened: Swift container (but not objects) deleted, bootstrap and destroy-environment fail <juju-core:New> <https://launchpad.net/bugs/1430220>09:21
coreycbTheMue, do you know the syntax for specifying an action-get key?09:25
TheMuecoreycb: sorry, not directly. would have to look too09:31
* TheMue is looking09:32
coreycbTheMue, ok np. figured I'd ask you since the other guys are not likely online.09:32
coreycbI'd tried a few ways based on https://jujucharms.com/docs/authors-charm-actions#action-get but wasn't getting anything back09:33
TheMuecoreycb: yes, it's a bit early09:33
TheMuecoreycb: the example shows a nested variable name. "outfile" and "name" are parts of it. if your "foo" is on the top level it's only "action-get foo".09:35
coreycbTheMue, ok thanks.  yeah that example's a bit odd since 'name' is never defined as far as I can see.  as an aside, I'm running with an experimental version of juju that has leadership elections so I'm going to try on 1.22-beta5.09:37
TheMuecoreycb: indeed, it doesn't match the actions.yaml shown above. "compression.type" would be a better example.09:40
mupBug #1430225 was opened: juju br0/juju-br0 does not observe dhcp mtu settings <juju-core:New> <https://launchpad.net/bugs/1430225>09:54
dimiternTheMue, standup?10:02
TheMuedimitern: yes, trying to connect. somehow my browser dislikes me today :(10:02
TheMuedimitern: aargh, not only the browser, all new connections. will reboot now10:04
mattywanastasiamac, excellent questions on your review, thanks very much10:11
mupBug #1430245 was opened: Transient error with juju-unit-agent on windows hosts <windows> <juju-core:New> <https://launchpad.net/bugs/1430245>10:36
coreycbTheMue, ok so I found the issue10:58
TheMuecoreycb: what has it been?10:58
jamdimitern: ping10:59
dimiternjam, pong10:59
jamI'm running into an intermittent failure, that we've tracked down to maybe a network issue11:00
jamspecifically in master11:00
coreycbI was using 'params:' instead of 'properties:' in actions.yaml.  params doesn't seem to work but properties does.  so maybe the doc just needs an update, not sure.11:00
coreycbTheMue, ^11:00
jamif you run: cd worker/uniter/filter11:00
jam$ time go test -c && for i in `seq 10`; do ./filter.test -gocheck.v -gocheck.f ConfigEvents & sleep 0.01; done11:00
jamthat runs the one test 10 times in parallel11:01
jamit seems we are getting an extra AddressChanged event11:01
jamand when you touch the test to remove the line where it sets the address11:01
jamit starts passing consistently, when we shouldn't be getting *any* events since the address wouldn't have been set11:01
dimiternjam, I'll have a look11:02
dimiternjam, which test is failing?11:02
jam-gocheck.f ConfigEvent11:02
jamTestConfigEvents and TestInitialAddressEventIgnored both suffer from this11:03
TheMuecoreycb: as I understand it "params" is for the definition parameters (top level), "properties" for their potential additional properties11:03
dimiternjam, ok, thanks, will let you know if I can reproduce it locally11:04
coreycbTheMue, that's what the doc seems to say too, but it doesn't appear to be the case11:04
jamdimitern: you might need to bump up "10" 1-in-10 fails for me here11:04
TheMuecoreycb: oh, have to talk to bodie and jw4 about it. could you pastebin your actions.yaml?11:05
dimiternjam, so far 2-in-20 failed11:07
dimitern3 even11:07
dimiternnow 411:09
coreycbTheMue, very basic, this works but if you swap params for properties it didn't work for me  --  http://pastebin.ubuntu.com/11:09
jamdimitern: right, so often enough that we have a clear problem :)11:09
coreycbsorry, http://pastebin.ubuntu.com/10573902/11:09
dimiternjam, it seems to fail twice as frequently here - always with github.com/juju/juju/testing/channel.go:63 - unexpected receive11:09
dimiternjam, is that the same issue you're seeing?11:10
jamdimitern: oh, I'm not saying the count, just that it fails regularly. 10 was enough that every attempt failed for me11:10
jamdimitern: but UnexpectedReceive is the failure11:10
TheMuecoreycb: ok, thanks. will discuss it with bodie and jw4 and come back to you later11:11
coreycbthanks11:11
dimiternjam, ok, why do you think it's a network issue?11:11
jamdimitern: if you comment out the line in the test SetNetworkAddress, the test *passes*, when the test expects nothing to set the address11:12
jamsince it didn't do it11:12
jamdimitern: the test is exercising the logic that just setting address or just setting charm url doesn't generate an event until the other one occurs11:13
jamdimitern: now this is a test running in JujuConnSuite, and the SetUpTest is using s.unit.AssignToNewMachine()11:14
jamso I don't know what that gets running in the background11:14
jambut apparently we're assigning addresses to the machine being tested11:15
jamand if it happens at exactly the right moment11:15
dimiternjam, well, commenting out SetAddresses before starting the watcher papers over the problem I think11:15
jamwe end up with 2 changed events, and the test fails11:15
jamdimitern: the point is that something in JujuConnSuite is forcing addresses for machines, and the test assumes it isn't, I don't *directly* care about what the fix is, it is just an intermittent test that blocked me landing code because I thought I broke something here11:16
perrito666morning11:17
TheMueperrito666: o/11:17
dimiternjam, right, will dig deeper into it11:17
jamcertainly some sort of "don't touch me" flag here would be nice11:17
jamI can understand that for most tests maybe we want to set an address because we have to have something to work with11:17
jamand this particular test was written when nothing was assigned11:18
dimiternjam, I guess removing setAddresses from there should do the trick11:18
jamdimitern: so just commenting out SetAddress in the test breaks what the test actually cares about testing11:18
jam(that we don't fire events until both address and charm url are set)11:18
jamdimitern: I guess ideally we'd be more decoupled, but it is written using JujuConnSuite, and I don't know we want to rework all of that11:18
jamdimitern: Just know that TestInitialAddressEventIgnored also runs into this problem11:19
jamdimitern: https://bugs.launchpad.net/juju-core/+bug/142639411:19
mupBug #1426394: TestConfigEvents random failure <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1426394>11:19
jamso mgz is seeing it in CI11:20
dimiternjam, I think the problem is s.machine in that suite is machine 0, which shouldn't be11:25
anastasiamacmattyw: :D np - m enjoying reading the code! yvm for ur PR :D11:27
mattywanastasiamac, I just had one question emailed you about it11:28
jamdimitern: I don't think the test cares, so if you want to have SetUpSuite create 2 machines, I'm ok with that. Though definitely comment why11:39
anastasiamacmattyw: is this the comment about returning a valid data when error occurs?11:47
mattywanastasiamac, that's the one, I wasn't sure if you were just commenting on expecting action11:51
anastasiamacmattyw: i liked the 3 part return :D11:52
anastasiamacmattyw: but in every similar piece of code i've seen, when an error is returned11:52
anastasiamacmattyw: all other parts are either nil or the most basic "empty" default11:53
anastasiamacmattyw: whereas you were returning a MeterNotAvailable value...11:53
anastasiamacmattyw: all i was saying that it jumped at me :)11:53
mattywanastasiamac, ah I see11:55
anastasiamacmattyw: :D11:56
* TheMue steps out for a moment12:41
mupBug #1430225 changed: juju br0/juju-br0 does not observe dhcp mtu settings <cts> <juju-core:New> <https://launchpad.net/bugs/1430225>12:42
jamanyone else upgraded to go 1.4?12:45
jamgo fmt stopped working if you have a symlink in your PWD12:45
jamand it changed how imports are sorted12:46
jam:(12:46
dimiternjam, I found the issue13:16
jamdimitern: yay13:16
dimiternjam, as usual in these cases a 1 line change :)13:16
jamdimitern: all about finding that line and knowing it won't break other expectations13:17
mgzdimitern: oo, go on, what's the change?13:17
dimiternjam, can you try it to see if you can still reproduce it? just add seenConfigChange = false after f.outConfig = nil on lin 47013:17
jamdimitern: that sounds like a very bad change13:18
jamit means that future address changes won't be notified until config changes again13:18
dimiternjam, I've successfully run both TestConfigEvents and TestInitialAddressEventIgnored multiple times with seq 5013:18
dimiternjam, well, the config watcher is getting restarted on setCharm13:18
jamdimitern: so the code intends to not send the *first* notification until it sees both Config and Address changes, but future changes to either should always trigger a change.13:19
dimiternjam, and that's not accounted for in the maybePrepareConfigEvent13:19
dimiternjam, not quite13:19
dimiternjam, if the config watcher was restarted, we *always* get the initial empty event13:20
jamdimitern: but if we just did setCharm is it actually wrong to say the config changed?13:22
dimiternjam, well another disturbing thing is the configChanges chan gets reset to the newly started configw.Changes() after it gets started13:23
dimiternjam, (again in setCharm event handling), so there's a possibility of a race reading from the old configChanges while configw is getting restarted13:24
dimiternjam, this might be the other missing piece, as running all tests 50 times in parallel I only got 3 failures - now trying with that additional fix - adding configChanges = nil; seenConfigChange = false just after "changing charm to %q"13:26
* dimitern 's machine is visibly lagging when running 50 instances of the filter tests in parallel13:28
jamdimitern: well I believe that is also running 50 mongod instances in parallel13:28
dimiternyeah :)13:29
dimiternjam, ok, so only 2 failures now - both in TestGetMeterStatus13:30
dimiternjam, correction - TestMeterStatusEvents13:36
dimiternjam, with seq 20 I can't reproduce it, and since my machine was under quite a load and I can see in the log just before "timeout waiting for receive" that the meter status was indeed sent13:38
dimiternjam, I'd attribute that these failures to heavy lagging under load13:39
jamdimitern: if it is just ShortWait sort of timeout, that would be ok13:40
jamdepends on the failure13:40
dimiternjam, it is a LongWait I think, but I'm running the tests a few more times13:41
dimiternjam, ok, now 1 failure out of 50 for all tests - this time TestActionEvents and it *is* a ShortWait13:47
jamdimitern: so the change sounds good given the testing we have in place, but I wouldn't be surprised if our testing wasn't actually as complete as we would like. So this is something that we'd want to think through well. Possibly running past fwereade13:48
dimiternjam, ok, so far we've established I believe it's not a networking issue but a watcher/race sort of thing; I'll propose a fix with those few lines added and provide a way to test it using your snippet13:49
dimiternjam, I'd13:49
dimiternjam, (really bad lag!)13:50
dimiternjam, I'd ask you to retry reproducing it with the fix13:50
jamdimitern: I'm certainly willing to try it. My concern isn't that it doesn't fix what I saw, but if it breaks other expectations13:51
dimiternjam, agreed, I'll ask fwereade to have a look as well13:52
alexisball we have a critical bug blocking 1.22, I need some volunteers:13:54
alexisbhttps://bugs.launchpad.net/juju-core/+bug/143004913:54
mupBug #1430049: unit "ceph/0" is not assigned to a machine when deploying with juju 1.22-beta5 <oil> <juju-core:Triaged> <https://launchpad.net/bugs/1430049>13:54
alexisbwallyworld_, has done initial investigation but it will require some f2f time with jhobbs in US hours13:55
dimiterndooferlad, TheMue, voidspace, guys, I'd appreciate if some of you could join the maas+juju interlock in 2 minutes13:57
mgzgsamfira: bug 143034013:57
mupBug #1430340: Failing to create tempdir in tests on windows <test-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1430340>13:57
gsamfiramgz: thanks :D13:57
gsamfiragrabbing a coffee and looking13:58
dooferladdimitern: on it13:58
dimiterndooferlad, cheers!13:58
voidspacedimitern: yep, just grabbing coffee13:58
dimiternvoidspace, ta!13:59
mupBug #1430340 was opened: Failing to create tempdir in tests on windows <test-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1430340>14:00
mgzgsamfira: for interest, the fix I'd made previously was https://github.com/juju/testing/pull/52 for IsolationSuite usage in juju, making sure we actually passed all the windows variables through to subprocesses regardless of case14:00
gsamfiracool! Windows does not realy care about case, but I remember doing case insensitive match when I pushed this. Might be mistaken though14:02
gsamfirabe roght back14:02
gsamfira*right14:02
wwitzel3perrito666: ping14:06
perrito666wwitzel3: pong14:09
perrito666wasnt me14:09
perrito666wwitzel3: ahh dst14:11
wwitzel3perrito666: standup14:11
perrito666wwitzel3: yup sorry I kieep thinking its in one hour14:12
mupBug #1430340 changed: Failing to create tempdir in tests on windows <test-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1430340>14:12
mupBug #1430340 was opened: Failing to create tempdir in tests on windows <test-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1430340>14:18
axwfwereade: if you have any time today, I'd appreciate a review of the storage hook source: http://reviews.vapour.ws/r/1113/14:20
gsamfiramgz: running tests now14:30
mgzgsamfira: ace14:30
gsamfiramgz: http://paste.ubuntu.com/10574918/ :)14:40
gsamfiragoing to fix tests today. We need that CI up as soon as humanly possible :D14:41
gsamfirarecloned master just to make sure14:44
gsamfirayup, my bad. unclean env on my machine. Apologies14:45
mgzyeah, wondered if it was a dep issue14:45
gsamfiratesting again14:47
axwcan someone please review https://github.com/juju/juju/pull/1789 - fixes the critical blocker in 1.2214:48
dimiternaxw, LGTM14:48
axwdimitern: thanks14:49
gsamfiramgz: I mentioned this before, and I don't know if its possible, but using WinRM to run the tests might avoid environment issues you get with OpenSSH. You can still use OpenSSH to scp files, just don't add it to the environment14:49
gsamfiramgz: there is a python script that can be used for this, and it can be run from any Linux box14:49
gsamfiramgz: https://github.com/cloudbase/pywinrm/blob/master/wsmancmd.py14:50
gsamfiramgz: we use this in the Hyper-V OpenStack CI to run the tempest tests14:51
mgzgsamfira: I think the ssh stuff is pretty clean - I was worried we were polluting with cygwin junk but turned out not to be the case14:55
dimiternTheMue, please update the status on bug 142843014:56
mupBug #1428430: AllWatcher does not remove last closed port for a unit, last removed service config <api> <juju-core:Triaged by themue> <https://launchpad.net/bugs/1428430>14:56
TheMuedimitern: will do14:56
mupBug #1403955 was opened: DHCP's "Option interface-mtu 9000" is being ignored on bridge interface br0 <cts> <juju-core:New> <isc-dhcp (Ubuntu):Confirmed> <https://launchpad.net/bugs/1403955>15:00
axwdimitern: I'm off to bed, would you mind keeping an eye on that merge and restart it if needed?15:04
dimiternaxw, sure, np15:05
voidspacedimitern: so only the State can run queries (like "find all IP addresses for this machine ID"15:05
voidspacedimitern: because the collection names are private15:05
voidspacedimitern: so a new state method to find them all15:05
voidspacedimitern: or I could just have a state method to remove them all and move all that code15:06
dimiternvoidspace, yeah, that sounds good15:06
voidspacedimitern: (it's only iterate over the returned collection and call addr.Remove)15:06
voidspacedimitern: a state method to fetch them all, or a state method to remove them all - which do you think?15:06
voidspacedimitern: I think maybe just a method to find them all15:06
dimiternvoidspace, I think the latter is better15:06
voidspacehah15:06
voidspacedimitern: ok, maybe15:06
voidspaceit's what we specifically need I guess15:06
voidspacefairy 'nuff15:07
dimiternvoidspace, if we need to find them we'll add another method15:07
voidspacelet a thousand methods bloom15:07
dimiternvoidspace, :)15:07
dimiternvoidspace, however, let me check a thing first15:07
=== jorge is now known as jcastro
voidspacedimitern: this is the core logic15:11
voidspacedimitern: http://pastebin.ubuntu.com/10575094/15:11
voidspacedimitern: it needs some error handling around the addr.Remove() and it's in the wrong place (needs to be in State)15:11
voidspacebut that's all it's doing15:11
dimiternvoidspace, yeah - no existing "find all ips for a machine/subnet"15:12
voidspacedimitern: I was pretty sure there wasn't as I've worked with most of the IPAddress code15:12
voidspaceit's new for the networking stuff15:13
voidspacewe're only modelling IP addresses allocated for containers so far15:13
dimiternvoidspace, yeah, so I guess it does sound better to have a machine.AllocatedIPAddresses() method15:15
dimiternvoidspace, and then call ip.Remove() on each.15:15
voidspacedimitern: ok, I'll do that15:18
voidspacedimitern: thanks15:18
voidspacedimitern: for machine id I want container.InstanceId() right?15:18
dimiternvoidspace, no, just machine id15:18
voidspacedimitern: I have a state.Machine called container15:19
dimiternvoidspace, we only need instanceId when talking to the provider - i.e. ReleaseAddress15:19
voidspaceah15:19
voidspaceso ID() instead of InstanceId()15:19
voidspaceor at least Id()15:20
dimiternvoidspace, yes15:26
dimiternfwereade, I'd appreciate if you can have a look at this http://reviews.vapour.ws/r/1118/ which should fix a uniter filter intermittent test failure15:27
mupBug #1307728 changed: ensure-availability command should show actions performed <ha> <ui> <juju-core:Fix Released> <https://launchpad.net/bugs/1307728>15:30
mupBug #1403955 changed: DHCP's "Option interface-mtu 9000" is being ignored on bridge interface br0 <cts> <juju-core:Invalid> <isc-dhcp (Ubuntu):Confirmed> <https://launchpad.net/bugs/1403955>15:30
dimiternjam, there's the fix btw ^^ if you can give it a try and see if you still reproduce the issue that'll be great15:37
sinzuidimitern, which milestone should bug 1428439 be on16:32
mupBug #1428439: retry-provisioning launches instances for containers; cannot retry containers at all <juju-core:Triaged> <https://launchpad.net/bugs/1428439>16:32
sinzuiericsnow, do you think we need a two vivid slaves with systemd and upstart to be certain juju works with both configurations?16:35
mupBug #1403955 was opened: DHCP's "Option interface-mtu 9000" is being ignored on bridge interface br0 <cts> <kvm> <lxc> <network> <juju-core:Triaged> <isc-dhcp (Ubuntu):Confirmed> <https://launchpad.net/bugs/1403955>16:37
ericsnowsinzui: not really, considering that vivid operates under upstart only in "old" pre-release images and the official releases will all be systemd16:37
dimiternsinzui, 1.23-beta1 is ok I think16:37
ericsnowsinzui: then again, but it also depends on if upstart will be officially supported as the booted init systemd on vivid16:38
sinzuiericsnow, what about people upgrading from utopic. they don't switch unless Ubuntu make further packaging changes16:38
ericsnowsinzui: I doubt it will be, but if it is then 2 slaves may make sense16:38
ericsnowsinzui: good point16:38
=== kadams54 is now known as kadams54-away
sinzuiericsnow, I am still unsure. maybe I should just take the time and have two slaves. I can turn one off later16:39
ericsnowsinzui: so does that mean upstart will be supported up to the next LTS?16:39
ericsnowsinzui: sounds good16:39
sinzuiericsnow, It might be required as a legacy.16:39
ericsnowsinzui: it should continue to work under upstart regardless16:39
sinzuiericsnow, yeah, and the current setup proves that. I want to be sure it still works if Ubuntu about-face16:40
ericsnowsinzui: but it does mean having 2 slaves for each release starting with vivid, no?16:40
ericsnowsinzui: there *is* a contingency plan for dropping systemd in vivid, so good thinking16:41
sinzuiericsnow, maybe? its about Ubuntu making systemd-sysv a requirement16:42
=== kadams54-away is now known as kadams54
=== kadams54 is now known as kadams54-away
=== kadams54 is now known as kadams54-away
ericsnowsinzui: do we use feature flags in CI?17:44
sinzuiericsnow, no17:44
sinzuiericsnow, are the set in environments.yaml?17:45
=== kadams54-away is now known as kadams54
ericsnowsinzui: I believe it's from environment variables17:47
sinzuiericsnow, it it works from shell env vars, I think we can reconfigure jobs as needed. If it comes from the juju env, we need to make config and or code changes to use them17:48
ericsnowsinzui: if we run a vivid slave with upstart then it may make sense to use a feature flag to indicate upstart-on-vivid17:48
sinzuiericsnow, understood17:49
ericsnowsinzui: there's a fallback case for init system discovery that hard-codes vivid (and subsequent releases) to systemd17:49
sinzuiericsnow, I am now at the point of wondering what will happen when I try to provision a new slave. The charm/package uses upstart17:49
sinzuiericsnow, if there is no legacy upstart support, the slave wont run without me writing a systemd script17:50
ericsnowsinzui: yeah, we'll see :(17:50
ericsnowsinzui: there is some accommodation for that, but I don't know to what degree (they were still discussing it a month+ ago)17:51
ericsnowsinzui: I'll try to find out17:52
ericsnowsinzui: can you tell if the latest vivid image is booting systemd?17:52
sinzuiericsnow, I haven't looked17:53
ericsnowsinzui: k17:53
sinzuiericsnow, since clouds don't support ubuntu devel images, nor does the current juju support systemd. I need to build this by hand17:55
ericsnowsinzui: ah, makes sense17:55
ericsnowsinzui: it's like that for each new series, huh?17:55
sinzuiI have done this before. it just take longer for me to type and upload things that juju and charms17:56
sinzuiericsnow, I usually wait for an official release of juju after a series opens. that gives me a juju and an agent. then I provision the previous series in a cloud, dist-upgrade, the manually provision. that is how we got the trusty, utopic, and current vivid17:57
gsamfiraany chance I can get a review on this: http://reviews.vapour.ws/r/111918:13
gsamfiraso we can start testing on windows? :)18:13
sinzuiericsnow, I am having a misadventure on the current vivid-slave. lxc blewup. I am watching a case were we upgrade from stable juju to unstable, which might be upset because 1.21.3 doesn't know about systemd. I will report real details in a bit18:17
ericsnowsinzui: k18:17
alexisbhttp://www.opencompute.org/community/events/summit/ocp-us-summit-2015-live-streaming18:22
alexisbthanks kwmonroe for the pointer ^^18:22
alexisbperrito666, this is our OCP demo18:22
=== kadams54 is now known as kadams54-away
* gsamfira is going through second bowl of popcorn18:23
alexisbgsamfira, :)18:24
sinzuiericsnow, looks like lxc template creation is broken. 1.21.3 cannot create a template with the current vivid lxc images. I am going to clean, upgrade, then retest that unstable juju loves the new image. Then try the upgrade again18:24
alexisbshouldnt you be asleep18:24
ericsnowsinzui: right18:24
gsamfiraalexisb: not yet. Its 20:24 here :D18:24
gsamfiraand I actually grabbed a few hours last night :)18:24
ericsnowsinzui: we send upstart-specific commands to cloud-init and in the LXC clonetemplate code18:25
sinzuiericsnow, I believe we will not be counting the upgrade test until 1.23.0 is released18:25
ericsnowsinzui: so if vivid is booting off systemd now then you'll see that sort of failure18:25
sinzuiericsnow, thank you. that predicts my question18:25
=== kadams54-away is now known as kadams54
ericsnowsinzui: can you check PID 1?18:26
sinzuiericsnow, I cannot even get into the container. it is a zombie18:26
ericsnowsinzui: yuck18:27
sinzuiI am kill lxc procs because the controls are useless18:27
ericsnowsinzui: not even with lxc-console?18:27
sinzuiit is stalled18:27
ericsnowsinzui: bummer18:27
ericsnowsinzui: oh18:27
ericsnowsinzui: there's a known issue here18:28
ericsnowsinzui: https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/134702018:28
mupBug #1347020: systemd does not boot in a container <systemd-boot> <lxc (Ubuntu):Fix Released by stgraber> <lxc (Ubuntu Trusty):Triaged by stgraber> <https://launchpad.net/bugs/1347020>18:28
sinzuiericsnow, that looks similar18:28
ericsnowsinzui: basically, vivid with systemd in a container on a non-vivid host won't work18:29
alexisbwith natefinch out is there someone who can review this for cloudbase (aka gsamfira):18:29
alexisbhttps://github.com/juju/juju/pull/179118:29
alexisbit is fixing our windows tests that will soon have the power to block trunk18:29
ericsnowsinzui: well, it works on trusty and utopic if you update a few things from e.g. "trusty-updates"18:30
ericsnowsinzui: I expect it won't work on precise18:31
hazmatanybody who wants to help a user debug while local provider is broken on vivid should join #juju and talk to lamont18:32
hazmats/while/why18:32
=== b is now known as Guest49617
=== Guest49617 is now known as bteleaga
=== kadams54 is now known as kadams54-away
=== kadams54-away is now known as kadams54
sinzuiericsnow, the comedy gets better. The jenkins upstart job got removed. I am now committed to rebuilding a slave that meets your needs. I am going to disable all the vivid jobs until the replacement is ready18:40
ericsnowsinzui: okay18:44
cmarsgsamfira, i'll take a look at it.18:46
alexisbcmars, thank you!18:46
gsamfiracmars: thank you! :)18:52
ericsnowsinzui: I'm going to land the patch that specifies that vivid uses systemd18:59
voidspaceg'night all19:00
ericsnowsinzui: then write a follow up patch that adds a feature flag for the vivid-on-upstart case19:00
sinzuiericsnow, I suppose you should as it clearly doesn't want me using upstart any more19:00
ericsnowsinzui: also, the ubuntu folks verified that vivid did indeed switch yesterday19:01
ericsnowsinzui: I've landed the vivid-is-systemd patch so vivid should be able to work now (LXC host issues aside)19:21
sinzuiericsnow, thank you.19:22
=== kadams54 is now known as kadams54-away
cmarsgsamfira, reviewed19:58
thumpermorning19:59
ericsnowsinzui: upstart feature-flag patch: http://reviews.vapour.ws/r/1121/20:00
sinzuifab20:00
ericsnowsinzui: that should help in the case we have a bot running upstart20:01
sinzuiericsnow, I am writing a systemd script for jenkins. I wonder how many charms will need to change20:02
alexisbmorning thumper20:04
ericsnowsinzui: well, since charms are series-specific the impact is explicitly limited--vivid charms will have to make sure they work under systemd20:07
ericsnowsinzui: how many charms will have to adjust?  good question20:07
sinzuiericsnow, yeah. We mark our charms because we need them the day the series is born20:08
thumpercherylj: if you want to start our 1:1 early, I'm fine with that20:14
cheryljthumper: sure, I'll join the hangout again20:16
perrito666now, that disconnect button is too big20:58
thumpercmars: anything you want to catch up with today?21:09
mupBug #1420049 was opened: ppc64el - jujud: Syntax error: word unexpected (expecting ")") <deploy> <openstack> <regression> <uosci> <juju-core:In Progress by axwalk> <juju-core 1.22:Fix Released by axwalk> <https://launchpad.net/bugs/1420049>21:10
=== kadams54-away is now known as kadams54
=== kadams54 is now known as kadams54-away
ericsnowthumper: thanks for taking a look at that vivid-LXC issue21:52
menn0thumper, davecheney: https://github.com/juju/utils/pull/11522:18
* thumper looks22:23
ericsnowmenn0: FYI, the github-RB hooks work for the utils repo too22:24
menn0ericsnow: yep I saw that. good stuff.22:25
ericsnowmenn0: I've been meaning to do it for the other repos but need round tuits22:26
thumpermenn0: review done22:33
* thumper hands ericsnow a round tuit22:34
=== kadams54-away is now known as kadams54
* jw4 learned something new today - round tuits - :)22:35
mgznot to be confused with square tuits22:36
jw4haha22:36
perrito666you both just made me google22:37
perrito666uff, finally all tests pass22:38
jw4perrito666, I blame it on ericsnow ... I was 50 years behind22:46
ericsnowjw4: :)22:46
perrito666ok, EOD22:47
perrito666cheers22:47
jw4bye perrito66622:47
=== kadams54 is now known as kadams54-away
jjoxhazmat: hi there o/ - fyc -> https://code.launchpad.net/~jjo/juju-deployer/fix-juju-deployer-diff-for-multiple-relations-between-servicepair/+merge/25227123:07
hazmatjjox: solid23:24
hazmatjjox: did you see my comments on the bug... much of this can be simplified by simply mocking juju23:25
jjoxhazmat: ah, hm ... hadn't - will peek tomorrow23:31
wallyworldaxw: just started looking at the local state PR. is there a reason not to reuse stuff from uniter/operation/state.go?23:47
=== kadams54-away is now known as kadams54

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!