=== kadams54-away is now known as kadams54
perrito666katco: ping00:05
=== kadams54 is now known as kadams54-away
davecheneythumper: https://bugs.launchpad.net/juju-core/+bug/145639800:47
mupBug #1456398: cmd/jujud/agent: runtime panic during tests <juju-core:Confirmed for dave-cheney> <https://launchpad.net/bugs/1456398>00:47
davecheneythis is me today00:47
davecheneygit bisect for the git gods00:47
mupBug #1456398 was opened: cmd/jujud/agent: runtime panic during tests <juju-core:Confirmed for dave-cheney> <https://launchpad.net/bugs/1456398>00:50
mupBug #1456398 changed: cmd/jujud/agent: runtime panic during tests <juju-core:Confirmed for dave-cheney> <https://launchpad.net/bugs/1456398>01:02
mupBug #1456398 was opened: cmd/jujud/agent: runtime panic during tests <juju-core:Confirmed for dave-cheney> <https://launchpad.net/bugs/1456398>01:08
menn0davecheney: i'm a big fan of git bisect01:24
thumperaxw: around?01:43
axwthumper: hey01:47
thumperaxw: oh hai01:47
thumperaxw:  just talking to wallyworld about the storage provider and multiple environments01:47
thumperaxw: just bitten us01:47
axwthumper: bugger. what's the problem?01:48
menn0thumper or wallyworld: txn pruning integration with Juju: http://reviews.vapour.ws/r/1720/01:48
thumperaxw: runner exited: watching volumes: permission denied01:48
thumperaxw, wallyworld: likely to be a simple fix01:49
thumperjust finding the right line to add01:49
* thumper tries to run up to speed01:50
thumperaxw: the storage provider is a machine agent worker?01:50
axwthumper: the "storageprovisioner" worker is run twice: once in the env manager, once in each machine agent01:50
axwthumper: kinda like how we have a env-level provisioner and one per machine for containers01:51
thumperaxw: well... we have an env-level provisioner per environment01:51
thumpernot just one on the state server machine01:51
thumperI'm wondering if this is the gotcha01:51
axwthumper: quite possibly01:52
thumperaxw: what is the worker called?01:52
thumperfound it01:52
axwthumper: if you get stuck, and can give me a repro, I can debug it01:54
thumperthe environ-storageprovisioner is running per environment01:54
thumperso that should be fine01:54
thumperwaigani: where was the error exactly?01:54
thumperaxw: it was waigani that ran into the issue, you two can talk directly so I can go smash the cli some more01:57
axwokey dokey01:57
waiganithumper: ERROR juju.worker runner.go:21901:57
thumperwaigani: hey, there is this cool function you can add...01:58
* thumper goes to search for it01:58
thumperI wrote it for this type of situation...01:58
thumperI think01:58
waiganithumper: I created a hosted env, deployed wordpress, work-load status hangs on: message: Waiting for agent initialization to finish. Debug-log then is full of perm denied errors01:58
anastasiamacaxw: if it helps, I saw similar error in featuretests - i wasn't on multiple env tho01:59
anastasiamacaxw: i was runing one cli command and than followed by another in the same test02:00
wallyworldmenn0: review done02:00
waiganiso there is this line: return isEnvironManager && tag == st.EnvironTag()02:00
thumpersee worker/provisioner/provisioner.go line 18102:00
anastasiamacaxw: i was going to talk to u about it but got distracted :(02:00
=== kadams54 is now known as kadams54-away
thumperif the error is not nil and the feature flag is set, it logs the error stack02:01
thumpervery handy for debugging02:01
thumperwhen you really don't expect an error return there02:01
axwwaigani: st should be scoped to the env at that point? or not?02:01
waiganiif it is okay to watch volumes in hosted envs, the fix should just be a matter of removing the isEnvironManager line02:01
waiganiaxw: yes, it should be02:02
waiganiaxw: I can double check though02:02
menn0wallyworld: thank you02:02
axwwaigani: don't understand, why would you want to remove that line?02:02
menn0anastasiamac: thanks for your review too02:03
waiganiaxw: I haven't tested, but it seems that would be what is stopping volumes from being watched in the hosted env02:03
axwwaigani: we only want state servers to be able to access environments02:04
waiganiaxw: as it will return false if it's hosted02:04
waiganiaxw: you've got this comment in the code:02:04
waigani/ TODO(axw) allow watching volumes in alternative02:04
waigani// environments? Need to check with thumper.02:04
anastasiamacmenn0: always nice :D tyvm for PR02:04
axwwaigani: yeah, I don't think that's relevant. that was before all the JES stuff went in, now I see we have one worker per env02:05
axwwaigani: before I thought we might have one worker that spoke to multiple envs02:05
waiganiaxw: ah okay02:05
thumperwaigani: where is the isEnvironManager line you are talking about?02:09
waiganithumper: line 7902:09
thumperwaigani: of which file02:09
thumperno one will ever log in to an apiserver with an environ tag02:11
thumperthat block should be removed02:11
thumperthe only things that log in are machines, units or users02:12
waiganishould we add cases for units and users?02:12
waiganias it currently doesn't have any02:13
axwthumper: what do you mean nothing will log in with an environ tag? what bit of code leads you to think that's happening?02:13
thumperline 72 or apiserver storageprovisioner.go02:14
axwthumper: the "case names.EnvironTag" is not the tag used to log in, it's a tag that the storage provisioner uses to scope things02:14
thumperthis seems overly complicated and ackward....02:15
axwthumper: no worker ever passes the login tag to canAccess02:15
thumperI think more logging around this access would show where the problem is02:15
* thumper leaves it to waigani02:15
* thumper runs away02:15
waiganithanks :/02:15
axwwaigani: the gist is: the worker asks to watch volumes. it passes a scope tag, and that is either a names.MachineTag or a names.EnvironTag02:16
axwwaigani: if it's an EnvironTag, the apiserver will watch volumes scoped to that environ02:16
axwwaigani: if it's a MachineTag, it'll watch volumes scoped to that machine02:16
waiganiaxw: right, that makes sense02:17
axwwaigani: so, if it's an EnvironTag we want to make sure the logged-in user is an environ manager02:17
axwand also that the state's EnvironTag matches the scope02:17
axwwaigani: my understanding of JES is limited, so that last bit may be where things are broken02:18
waiganiaxw: when you say scope, you mean the state passed in to NewStorageProvisionerAPI ?02:19
axwwaigani: no, I mean the tag passed in the API request, in WatchVolumes/WatchFilesystems02:20
axwwaigani: those methods take params.Entities, each of which is a MachineTag or EnvironTag as described above02:21
waiganiaxw: yeah but "matching the scope" is matching the EnvrionTag passed in with that of the state obj used to make the api, right?02:25
axwwaigani: yes02:25
waiganiaxw: so the EnvironTag case is never going to pass in a hosted env02:27
axw<axw> waigani: st should be scoped to the env at that point? or not?02:27
axw<waigani> axw: yes, it should be02:27
axwso the "state.State" is *not* scoped to the env?02:28
axw(hosted env)02:28
waiganiaxw: yes, it should be scoped, but I'll check. I'm just looking to see which user is isEnvironManager() checking?02:34
axwwaigani: AuthEnvironManager02:35
axwi.e. a machine agent with JobManageEnviron02:36
waiganiaxw: so in the hosted env scenario, which machine would that be?02:37
axwwaigani: you're asking me how JES works? :)02:38
axwit should be a state server machine?02:38
waiganiaxw: haha, okay I'm going to dig into isEnvironManager02:38
waiganiaxw: if it's not the state server machine, it's going to fail. Also, as you point out if st isn't scoped, it's going to fail02:39
axwwaigani: yup02:39
waiganiaxw: so let me dump out some info and we can go from there02:39
waiganiaxw: shall I remove your comment?02:40
axwwaigani: yes please, if you're making changes02:41
waiganiaxw: I'll take it out now, in case this turns into a PR02:41
wallyworldaxw: it does appear that azure really wants people to stop using affinity groups which is good for us. let me know if you have time for a quick chat as there's a couple of points i'd like to clarify. no rush02:58
axwwallyworld: yeah, that sounds like the case. can chat now, tanzanite standup?03:00
waiganigot it04:22
waiganiso st is scoped to hosted env, but the tag passed in is always the server env, so they never match. jujud's  ReadConfig (cmd/jujd/agent/agent.go:106) takes a string tag and uses that to determine the location of the agent's config file to read.04:29
waiganithe string tag is "machine-0"04:31
waigani(which can no longer be assumed to be the state server, as the first machine in a hosted env is also machine-0)04:31
waiganiagent.ConfigPath returns the agent.conf of machine-0 the state server, not machine-0 the first machine in hosted env04:37
anastasiamacwaigani: well done :D04:40
waiganiaxw: thumper: ^^04:43
axwwaigani: are you saying you can have two machines called machine-0?04:45
waiganiaxw: yes04:45
axwsounds crazy, but ok04:45
axwwaigani: so... is that a jujud bug? that it's reading the agent.conf for the wrong agent?04:46
waiganiaxw: yep04:46
waiganiaxw: so will most likely have side effects elsewhere04:46
waiganiaxw: juju environment create env1; juju deploy wordress04:47
waiganiaxw: the juju status and checkout the machine name04:47
thumperwaigani: I'm pleased you found it04:50
axwwaigani: hrm, I'm a little confused. I would think that machine-0 the state server would be running env workers per hosted env?04:50
thumperaxw: machine-0 machines are in different environments04:50
* thumper takes the dog out while it is still light04:51
waiganithumper: can we talk this one over tomorrow morning?04:51
waiganiI need to run also04:51
=== urulama is now known as urulama__
* thumper heads off to make dinner06:33
axwwallyworld: looks like the whole azure model has changed. the little bit I've read makes it sound a lot more like other IaaS now, and Cloud Services are no more (with the new ARM API). portal is excruciatingly slow though, so can't actually tell for sure...06:45
axwwallyworld: if that is the case, we might be able to migrate away from what the either-or HA/placement implementation06:47
wallyworldaxw: yeah, they are changing terminology to use Region etc as well - so i think it's going to be goof07:31
axwwallyworld: yes, looks good07:31
axwwill be nice to kill some of that old hacky code07:32
axwrelegate it anyway07:32
rogpeppe3anyone fancy a small review of a LoggingSuite change? https://github.com/juju/testing/pull/6907:52
voidspacedimitern:  http://reviews.vapour.ws/r/1714/08:16
dimiternvoidspace, looking08:16
voidspacedimitern: plus http://reviews.vapour.ws/r/1715/08:16
dimiternvoidspace, will do08:17
voidspacedimitern: basically the same code as landed, with minor fixes for 1.24/master plus adding the upgrade step to steps124 too08:17
voidspacedimitern: thanks08:17
dimiternvoidspace, have a look at our shiny new and empty board https://canonical.leankit.com/sapphire08:17
voidspacedimitern: oooh...08:17
voidspacedimitern: shiny :-)08:17
dimiternvoidspace, we'll start using it by friday I hope08:18
voidspacedimitern: cool08:18
=== ashipika1 is now known as ashipika
TheMuedimitern: but then we'll soon fill it and it won't be so shiny anymore *scnr*08:35
TheMuedimitern: hehe, and opening the parking lot show's already some cards08:36
dimiternTheMue, :) that's right, yeag08:41
dimiternyeah even08:41
dimiternI'll groom the parking lot cards a bit and see what we didn't manage to finish by thursday08:42
TheMuedimitern: now it only has to be a visual frontend to our tasks in launchpad ;)08:42
dimiternvoidspace, both reviewed08:58
voidspacedimitern: ta08:58
voidspacedimitern: I'll have to go and fix those on 1.23 as well08:58
dimiternvoidspace, well, at least check if with 1.23 and string ids there the test still passes, other than that it's too late for 1.23 I think09:00
voidspacedimitern: the tests pass09:00
voidspacedimitern: the branch landed and I ran tests for 1.24 and master09:00
dimiternvoidspace, cool, I think it's fine then09:01
voidspaceon 1.24 (current version not my branch) featuretests time out09:01
voidspacethey pass on master09:01
voidspaceodd and annoying09:01
rogpeppe3dimitern: where's the right place to submit bugs against juju-core - github or launchpad?09:20
dimiternrogpeppe3, lp I guess09:21
rogpeppe3dimitern: "you guess"? :)09:21
=== rogpeppe3 is now known as rogpeppe
rogpeppedimitern: ok, will submit against lp09:21
dimiternrogpeppe, I guess >> it was like that originally, but I know there are some GH issues filed now09:22
rogpeppedimitern: ok.09:22
rogpeppedimitern: i'll file on lp anyway09:23
rogpeppei guess :)09:23
rogpeppedimitern: https://bugs.launchpad.net/juju-core/+bug/145651909:25
mupBug #1456519: apiserver/client: clientSuite.TestProvisioningScript is flaky <juju-core:New> <https://launchpad.net/bugs/1456519>09:25
rogpeppei guess noone else is using go1.3 or later to run juju tests09:26
rogpeppedimitern: any chance of a review of https://github.com/juju/testing/pull/69 please?09:28
rogpeppedimitern: (it's very small)09:28
rogpeppeor anyone else ( voidspace? fwereade?)09:29
fwereaderogpeppe, ship it09:30
rogpeppefwereade: ta!09:30
dimiternrogpeppe, I'll have a look after standup09:30
dimiternrogpeppe, ah, ok then09:30
rogpeppefwereade: it would be nice to have your LGTM on it09:31
fwereaderogpeppe, it does? http://reviews.vapour.ws/r/1724/09:31
rogpeppefwereade: (we need two, and an explicit signoff from -core is always good)09:31
rogpeppefwereade: ha! i'm not used to reviews.vapour.ws!09:32
rogpeppefwereade: i was looking at github only09:32
rogpeppefwereade: ta!09:32
mupBug #1456519 was opened: apiserver/client: clientSuite.TestProvisioningScript is flaky <juju-core:New> <https://launchpad.net/bugs/1456519>09:33
=== urulama__ is now known as urulama
dimiternreviews on http://reviews.vapour.ws/r/1725/ please - it fixes bug 144225710:43
mupBug #1442257: lxc network.mtu setting not set consistently across hosts <addressability> <lxc> <network> <juju-core:Triaged> <juju-core 1.23:In Progress by dimitern> <juju-core 1.24:Triaged by dimitern> <https://launchpad.net/bugs/1442257>10:43
dimiternvoidspace, TheMue, dooferlad, ^^10:49
dooferladdimitern: *click*10:49
dimiterndooferlad, ta!10:50
TheMuedimitern: back from lunch, looking too11:10
dimiternTheMue, cheers11:20
jamwallyworld: perrito666: we're still on for the relation status discussion, right?11:52
jamwwitzel3: we're also supposed to meet sometime, right? Do you want to meet before the retrospective tomorrow?11:53
wwitzel3jam: that sounds good to me12:53
jamperrito666: on the concept of VR hangouts, I'm told that having the world move for you is the really-really big producer of nausea13:03
perrito666jam: I only used a VR headset once and I got really confused about not seeing my hands13:09
perrito666the movement was not very annoying when sitting13:09
alexisbjam, I was just about to schedule a planning meeting in that slot :)13:26
jamalexisb: for our scheduling?13:26
perrito666alexisb: did you just cancel the interlock?13:27
alexisbperrito666, yes I forgot to do it yesterday13:27
* perrito666 notices google calendar loves a refresh after a couple of weeks up13:27
alexisbcloudbase is at ODS this week13:27
perrito666ah true13:27
alexisbjam, yes13:28
jamalexisb: sure. if you get it scheduled first you win :). I also sent a message to cmars about if there is another time we could use13:28
alexisbyeah the hour after the leads call is definitely my preference13:29
alexisbbut I am going to grab that weds slot just in case13:29
katconatefinch: standup14:10
ericsnowvoidspace: weren't you the one that moved us away from "anything" on our SSL cert code?14:10
sinzuivoidspace, is this bug Fix Committed in 1.24 https://bugs.launchpad.net/juju-core/+bug/134866314:26
katcofwereade: hey want to chat about the container stuff a bit early? we're all in moonstone?14:26
mupBug #1348663: DHCP addresses for containers should be released on teardown <maas-provider> <network> <oil> <juju-core:Triaged by mfoord> <juju-core 1.24:Triaged by mfoord> <MAAS:Invalid> <https://launchpad.net/bugs/1348663>14:26
sinzuivoidspace, I think your last commit to 1.24 fixes bug 144120614:26
fwereadekatco, just making coffee, will join you in there though if that makes sense?14:29
mupBug #1441206: Container destruction doesn't mark IP addresses as Dead <destroy-unit> <network> <juju-core:In Progress by mfoord> <juju-core 1.24:In Progress by mfoord> <https://launchpad.net/bugs/1441206>14:29
katcofwereade: yeah, we're all in ther efrom the standup14:29
katcofwereade: https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=1&hceid=Y2Fub25pY2FsLmNvbV9pYTY3ZTFhN2hqbTFlNnMzcjJsaWQ5bmhzNEBncm91cC5jYWxlbmRhci5nb29nbGUuY29t.7g85fo35lqi339e7ppmrdgeoi014:29
voidspaceericsnow: no, that was wwitzel314:32
voidspacesinzui: it does fix a bug14:32
voidspacesinzui: let me confirm14:32
=== kadams54 is now known as kadams54-away
voidspacesinzui: I've marked 1441206 as fix committed for 1.24, still trying to merge into master14:50
sinzuivoidspace, thank you14:51
natefinchjw4: you around?15:01
voidspacecommand line tool to open hangouts at your next meeting15:11
jw4natefinch: yeah - I'm OTP for the next 30 minutes or so, but I'm somewhat around15:28
natefinchjw4: quick question - am I correct in my observation that action results are always converted to strings?15:28
jw4natefinch: hmm; that doesn't seem quite right to me15:29
natefinchjw4: for example, I set a value of 16, but it gets converted into "16"15:29
jw4natefinch: I don't know for certain, but I *thought* result types would be preserved15:29
jw4natefinch: bodie_ worked on that part so he may remember more accurately15:29
jw4natefinch: I'll investigate too when I get a chance15:30
natefinchjw4: ok no worries.  Could well be my fault, too.15:30
natefinchheh, interesting, from python's time formatting, about parsing seconds: "The range really is 0 to 61; this accounts for leap seconds and the (very rare) double leap seconds."15:36
dimiternreviews on http://reviews.vapour.ws/r/1726/ are most welcome15:37
TheMuedimitern: *click*15:37
mupBug #1454697 was opened: jujud leaking file handles <cpec> <stakeholder> <juju-core:Triaged> <juju-core 1.22:In Progress by cherylj> <juju-core 1.23:New> <juju-core 1.24:New> <https://launchpad.net/bugs/1454697>15:46
alexisbkatco, you around?15:49
katcoalexisb: yep15:49
natefinchone thing I have to give it to python for, it is very googable for common questions15:50
abentleytvansteenburgh: Thanks for the merge.15:53
=== kadams54 is now known as kadams54-away
jw4natefinch: it looks like all the result values are converted to strings...16:14
jw4natefinch: the tests seem to explicitly expect that16:14
natefinchwhy even use yaml if you can't use nice types? :/16:15
jw4teh suk16:16
natefinchwell, at least I'm not crazy16:17
jw4natefinch: correction "at least this situation doesn't demonstrate whether you're crazy"  - you may still be but not as shown by this situation :)16:18
natefinchjw4: I was going to make that joke, but couldn't figure pout a way to word it.16:18
jw4natefinch: well, I didn't do too well either16:18
mupBug #1456703: action-set converts everything to strings <actions> <juju-core:New> <https://launchpad.net/bugs/1456703>16:19
natefinchat least it's recorded now16:19
mupBug #1456703 was opened: action-set converts everything to strings <actions> <juju-core:New> <https://launchpad.net/bugs/1456703>16:28
natefinchway to be on the ball there, mup16:28
=== kadams54-away is now known as kadams54
* natefinch really needs pyformat16:34
natefinchpython peeps... is there an official or semi official pyformat?16:36
natefinchI see pyformat, I see autopep8 ... what should I use and do I have to worry that it'll blow up my code?16:37
ericsnownatefinch: nothing official16:38
voidspacenatefinch: autopep8 is the closest16:38
natefinchgood enough for me16:38
rick_h_natefinch: run flake8 and that's the basics16:46
rick_h_natefinch: it's what we've been using + a few manual ideas copied from LP days for all our python stuff16:47
=== kadams54 is now known as kadams54-away
=== kadams54-away is now known as kadams54
mupBug #1456714 was opened: assignCleanSuite.TearDownTest fails <ci> <intermittent-failure> <unit-tests> <juju-core:Incomplete> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1456714>17:01
=== natefinch is now known as natefinch-afk
=== kadams54 is now known as kadams54-away
=== kadams54-away is now known as kadams54
=== kadams54 is now known as kadams54-away
=== kadams54-away is now known as kadams54
=== urulama is now known as urulama__
=== kadams54 is now known as kadams54-away
=== kadams54-away is now known as kadams54
=== kadams54 is now known as kadams54-away
katcowallyworld: hey got a few minutes to talk about a spec?21:55
wallyworldkatco: yeah, give me a minute21:55
katcowallyworld: ack21:55
katcowallyworld: doh, nevermind the family just got home. i'll catch up with you later21:56
wallyworldok, ping me , i ave a meeting in an hour but free after that21:56
thumpercmars: around?\22:12
cmarsthumper, hey22:12
cmarsthumper, yeah, hang on22:12
thumpercmars: I'm in our hangout22:12
noodles775wwitzel3: Hi! Just wondering if the reason you switched the status of bug 1453644 was related to the bug itself, or just simply you've got other priorities you need to switch to etc.22:13
mupBug #1453644: destroy-env leaves one lxc which remains pending on redeploy <destroy-environment> <local-provider> <lxc> <reliability> <repeatability> <juju-core:Triaged by wwitzel3> <https://launchpad.net/bugs/1453644>22:13
davecheneythumper: git bisect success22:17
davecheneymachine handed off to rsc22:17
davecheneyi'm off for a coffee22:17
davecheneyand the offending diff looks like a plausible candidate this time22:17
davecheneynot a change to an unrelaed .h file22:18
mwhudsondavecheney: \o/22:46
wallyworldaxw: i can't get the meeting link to work, can you?23:00
axwwallyworld: nope23:00
sinzuiwallyworld, do you have a moment to review http://reviews.vapour.ws/r/1729/23:07
=== noodles775 is now known as michaeln
=== michaeln is now known as mnelson
=== mnelson is now known as noodles775
sinzuiwallyworld, 386 unit tests might fail for 1.24. The failures for the first 2 tries are flaky, so I am declaring build 2664 releasable for beta4.23:10
thumperwallyworld: I may be slightly late to the 1.23 retrospective, but I will be there23:15
=== noodles775 is now known as miken
davecheneymwhudson: thanks for breaking the seal on the 'compiler is slow argument, let's no gc'23:59
davecheneyi was thinking of writing the same to russ privately23:59
davecheneybut public is better for everyone's karma23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!