/srv/irclogs.ubuntu.com/2014/09/03/#juju-dev.txt

davecheneymenn0: reviewing00:04
menn0davecheney: thanks00:05
menn0davecheney: sorry that it's a fairly big one00:05
mbruzekwallyworld_, note sent, please reply if I missed something.00:08
wallyworld_will do, thank you00:08
mbruzekdavecheney, sinzui, wallyworld_  thanks very much for helping me on this problme.00:09
wallyworld_anytime00:09
mbruzekhave a good day/night00:09
wallyworld_you too00:09
davecheneymenn0: review done00:11
menn0davecheney: thanks very much00:11
waiganithumper: any hints on how to get resources and authorizer to call the NewClient? Do we have mocks somewhere?00:44
davecheneywaigani: there is a mock for Authorizer00:51
davecheneyapiserver/testing.FakeAuthorizer00:51
davecheneytakes a tag00:51
waiganiyes!00:51
waiganithank you00:51
davecheneydunno about resources00:51
davecheneynot even really sure what they are00:51
ericsnowaxw: ping01:02
axwericsnow: hey, just got back from drop off01:02
ericsnowaxw: cool, could you take a look at bug #1364438?01:03
mupBug #1364438: utopic lxc tools.tar.gz and aria2c not found <ci> <local-provider> <lxc> <regression> <utopic> <juju-core:Triaged> <https://launchpad.net/bugs/1364438>01:03
axwericsnow: ah, yep, thanks01:04
ericsnowaxw: I'm pretty sure it's related to the aria2 change from the other day01:04
axwindeed01:04
ericsnowaxw: thanks!01:05
waiganithumper: ping01:23
thumperwaigani: hey01:28
thumperwaigani: look at common.NewResources()01:29
waiganithumper: yeah got that01:29
waiganion to the next prob now, state.getCollection borks01:29
waiganibecause st.authenticated is a nil pointer01:29
waiganii'm guessing because of the mock authenticator not setthing it?01:30
waigani*is not setting it01:30
thumperwaigani: that means that the open call was done without a user/passwrod01:30
waiganiopen call?01:31
thumperwaigani: how are you getting state*.State01:31
waiganifrom the base suite01:31
* thumper thinks01:32
waiganiJujuConnSuite01:33
thumperI think I need to see the code01:33
waiganisure01:34
waiganiscreen share, pastebin or push pr?01:34
thumperum... push pr probably easiest01:38
waiganithumper: http://pastebin.ubuntu.com/8219767/01:40
waiganithis is how I'm setting up the suite01:40
wwitzel3ok, so I changed RunCommands in every place I could find .. yet I still am getting an error from the rpc.Register, http://paste.ubuntu.com/8219764/01:41
wwitzel3having trouble tracking down why the exported method isn't suitable01:41
thumperwwitzel3: I think you may have to restructure so it has only one in param and one out param01:42
wwitzel3thumper: I will give that a shot, thanks01:43
thumperwwitzel3: there is weird magic in the api layer registration01:43
wwitzel3lol, good enough explantion for me01:44
wwitzel3:)01:44
thumperwaigani: that looks ok to me...01:45
thumperwaigani: although...01:46
thumperwaigani: you probably want to change the auth tag01:46
thumperwaigani: to be names.NewUserTag(state.AdminUser)01:46
thumperall that is changing in my branch01:46
waiganiah, sure01:46
thumperbut it is likely to be a problem for you01:46
thumperas the code that does the lookup01:46
thumperwon't find that user01:46
waiganithumper: I'll hunt down the st.authorized problem - got a few leads :)01:47
waiganioracle is helping for once01:47
waiganithumper: dummy provider does not set Tag or Password in mongo.MongoInfo before opening a connection: provider/dummy/environs.go:11202:21
thumperwaigani: but other tests pass... so figure out why :-)02:22
waiganisigh02:22
thumperwaigani: think of it as a learning exercise02:23
thumperand to help things out, it is probably something simple and stupid02:24
thumperat least it will seem that way once you have found it02:24
wwitzel3I always feel that way02:36
wwitzel3:/02:36
axwreview please, https://github.com/juju/juju/pull/661 - fixes CI blocker02:45
menn0davecheney: PTAL https://github.com/juju/juju/pull/660/02:47
menn0davecheney: you may find it quicker to just look at the second commit which is where I've addressed your review feedback02:48
waiganithumper: so s.State is nil in SetUpSuite, even though I've called s.baseSuite.SetUpSuite(c)02:52
waiganithumper: fix would be to init the client in SetUpTest, but why is state nil?02:52
thumperdon't know...02:53
thumperand I'm busy reading...02:53
thumperchat with menn0 :)02:53
davecheneymenn0: ta02:53
davecheneylooking02:53
waiganinames.ParseUserTag("user@provider") returns names.UserTag{name:"", provider:""}03:06
waiganiis that expected?03:06
waiganidavecheney ^?03:07
davecheneywaigani: nope03:16
davecheneyis there a test case covering this ?03:16
menn0davecheney: I've responded to the your remaining comment for PR 66003:16
waiganidavecheney: just hit it03:16
waiganiso I'd say no03:16
davecheneywaigani: hang on, parseUserTag returns two values03:17
waiganiyeah, so when err is nil03:17
waiganiit's still an empty tag03:17
davecheneylucky(~/src) % go run tt.go03:18
davecheneyuser- "dave@deathstar" is not a valid tag03:18
waiganiah shit ignore me, sorry for the noise - we are not returning on err03:19
davecheneyok03:19
axwwallyworld_: I'm just rebasing then should be in a position to push my tools-in-state branch up. it's grown quite a bit, so I think I'll have to try and split it up a bit04:06
wallyworld_ok04:07
axwI'll push it up anyway if you want to take a look at the core bits though04:07
axwwallyworld_: FYI https://github.com/axw/juju/compare/state-tools-catalogue04:47
axwwould appreciate a glance over state/tools.go specifically04:47
wallyworld_ok04:47
axwthat's the bit that uses blobstore04:47
wallyworld_shit, i still gotta fix that04:47
axwwallyworld_: would you like me to create a bug against 1.21 so we don't release without it?04:48
wallyworld_yeah, that would be good actually04:48
axwwallyworld_: https://bugs.launchpad.net/juju-core/+bug/136475004:53
mupBug #1364750: blobstore's hashing needs improvement <juju-core:New> <https://launchpad.net/bugs/1364750>04:53
wallyworld_ta04:53
axwwallyworld_: just realised I got the databases back-to-front for the managed storage. IIANM, the blobs should go in their own DB and the catalogue can go in the juju db05:02
wallyworld_yep05:02
wallyworld_axw: also, the path for tools storage should be prefixed with /tools05:02
wallyworld_cause there'll also be a /charms etc05:02
axwwallyworld_: atm it's "tools-", so I should change that to "tools/"? is the leading slash required?05:03
wallyworld_yeah change to tools/, leading slash not required. storage will preprend with /environs/<uuid>/05:04
axwthought so. cool05:04
wallyworld_axw: also, i was thinking we'd have a ToolsStorage interface with an implementing struct that is constructed with NewToolsStorage(), rather than bulking up state.State with more methods05:09
wallyworld_NewToolsStorage() would take the environ uuid etc as parameters, and probably a txnRunner of some sort05:11
wallyworld_the txnRunner could just be state instance passed in05:11
wallyworld_this then allows for easier, standalone testing using dependency injection etc05:12
axwwallyworld_: how would the ToolsStorage be accessed?05:12
wallyworld_how is it accessed now in yur branch?05:13
axwit's currently used in two places IIRC (not counting the myriad tests): by bootstrap, and by the apiserver/tools.go code05:14
axwand apiserver/common/tools.go05:14
wallyworld_it looks like we construct a ToolsGetter passing in state05:15
wallyworld_so that would change to pass in a ToolsStorage, which is constructed from state05:15
wallyworld_that's one example in tools.go05:16
axwI'll have a look at how it can be split out, I'm not seeing a lot of benefit right now though05:17
wallyworld_adding more and more to state kinda sucks05:19
wallyworld_i'd much prefer smaller, standalone components05:19
wallyworld_easier to test and reuse05:19
wallyworld_axw: if you have time at some point, i'd love a review of this PR so i can give something to the landscape guys when they come on. the next OCR people aren't on for a while :-( maybe if you are sick of tools and want something slightly different for a short time. https://github.com/juju/juju/pull/66205:37
axwwallyworld_: can you please elaborate on what is only called at bootstrap time?05:39
wallyworld_at bootstrap, the floating ip is tracked and associated with the instance, and the addresses correctly stored. then, the address poller queries the instances running and gets their addresses and overwrites what was done at bootstrap, because the api used bu the instance poller did not take account of the floating ip05:41
axwwallyworld_: ah, I see, thanks05:44
wallyworld_i tested live on hp cloud and it worked05:44
wallyworld_took a little extra testing because their use a funny address range that highlights the bug and we don't05:45
axwwallyworld_: I'm just looking at goose/nova.ServerDetail, and I think there's already fields in there for floating IPs05:48
axwnamely, AddressIPv4 and AddressIPv605:48
axwmay be a simple matter of just using them in getAddresses()05:48
wallyworld_hmmm, could be, i didn't see those05:48
wallyworld_i'll check it out05:49
axwwallyworld_: tests are broken now, but can you see if this is more to your liking? https://github.com/axw/juju/compare/state-toolstorage06:11
wallyworld_sure06:12
axwwallyworld_: code implementation moved into state/toolstorage, with a method on State to create one06:12
axwcore impl*06:12
=== urulama-afk is now known as urulama
wallyworld_axw: looks better. we can now test the tools stuff without instantiating a state at all. just mongo and a txn runner06:16
axwokey dokey. I'll fix up the tests and propose this in isolation06:18
axwthen I'll get back to the rest06:18
thumper(╯°□°)╯︵ ┻━┻06:18
wallyworld_axw: sadly, those AddressIP4 and AddressIP6 fields are never filled in. i've run up an hp cloud env and even explicity querying for server details after assigning the pubic address, they come back empty :-(06:20
axwrats06:21
axwwallyworld_: welp, in that case my comment on the PR stands06:21
wallyworld_keeping the code to process the floating ips in the Instances() calls means also it is only done once06:21
axwwallyworld_: yep, fair enough - that's why I went looking at ServerDetails06:22
wallyworld_not each time an individual instance's Addresses() is called06:22
axwso it would need to be done on AllInstances too06:22
axwI think those are the only two places06:22
wallyworld_ok, i didn't notice AllInstances06:22
wallyworld_i'll fix after school pickup06:22
=== liam_ is now known as Guest81233
wallyworld_axw: one though i had also - tools storage is really associated with an environment, so i wonder if the toolstorage package should be environs/toolsstorage not state/toolsstorage. it also means that stuff under state doesn't depend on environs06:47
axwwallyworld_: it's inherently tied to mongo, so I don't think it's a good idea06:53
axwthere are other environment-specific things in state06:53
axwlike, most of state06:53
wallyworld_ok, fair enough06:56
wallyworld_axw: i updated the PR to fix AllInstances()07:13
axwwallyworld_: just commented on it07:13
mattyw_morning folks07:16
axwjam: the reason I set it as critical is that we should not release code with blobstore without fixing it first. is there a better way to flag that?07:19
jamaxw: Critical means drop everything, IMO.07:19
jamaxw: and FWIW I actually think we'd be perfectly safe releasing with SHA-107:19
jamit isn't *more safe* to also use MD5 and SHA-256 is better07:20
jambut we aren't leaving a critical security hole open by only using SHA-107:20
axwjam: we're not using SHA-1, we're using MD5+SHA-256 concatenated07:20
axwwe want to drop MD507:20
jamaxw: which is fine, but still not a *security* issue07:20
axwif we release *with* it, we'll have a migration problem to deal with07:21
jamI agree with the improvement.07:21
voidspacemorning all07:34
wallyworld_jam: did that email thread conclude we could just use SHA-384?07:34
wallyworld_instead of SHA256 and MD507:34
mattywis landing still blocked on https://bugs.launchpad.net/juju-core/+bug/1348477?07:35
mupBug #1348477: userAuthenticatorSuite.TearDown failure <ci> <regression> <test-failure> <juju-core:Triaged by cmars> <https://launchpad.net/bugs/1348477>07:35
wallyworld_oh, also, we weren't concatenating07:35
wallyworld_jam: the implementation required the user to know both checksums as per marks's original directive to william. if that's what is meant by concatenating, then we were I guess07:37
wallyworld_but the checksums were specified separately07:37
wallyworld_mattyw: hmmm, that's not a regression07:37
wallyworld_that bug test failure has been around for a while07:38
wallyworld_it's intermittent07:38
mattywwallyworld_, yes, but I was sure it had appeared as a reason for not allowing landing yesterday07:39
mattywwallyworld_, unless it's been downgraded07:39
wallyworld_could have done i suppose, i haven't been keeping up07:39
wallyworld_it's still marked as a regression07:40
wallyworld_i think i'm going to change it07:40
mattywwallyworld_, I think that's probably a wise move07:40
mattywwallyworld_, is there a simple query I can do on lp to get the list of bugs that block landing at any moment? is it just critical bugs with ci + regression tags?07:43
wallyworld_mattyw: I think so yes07:43
mattywwallyworld_, looks like there are 4 at the moment?07:43
wallyworld_could be, i haven't looked07:43
axwwallyworld_: cleaned it up, https://github.com/juju/juju/pull/66307:43
axwwallyworld_: I meant (md5(x), sha256(x)), as opposed to md5(sha256(x)) or sha256(md5(x))07:44
mattywthis one could probably be changed to not include ci, regression as well https://bugs.launchpad.net/juju-core/+bug/136441007:44
mupBug #1364410: Timeout TestManageEnviron MachineSuite in ppc64el <ci> <intermittent-failure> <ppc64el> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1364410>07:44
wallyworld_axw: sorry, what's the context?07:45
axwwallyworld_: that's what I meant by concatenating07:45
axwbasically, what you said07:45
wallyworld_axw: oh, i didn't realise that you mentioned concatenating, i think t came from the email thread in my mind07:46
wallyworld_but yes, i think we agree07:46
wallyworld_mattyw: i see 2 critical ci regression bugs now07:47
wallyworld_i'm not sure that either should block landings07:48
wallyworld_but i'm also reluctant to override curtis without talking to him07:48
mattywwallyworld_, what search are you doing?07:48
mattywwallyworld_, agreed, I'll talk to him when he's around07:48
wallyworld_https://bugs.launchpad.net/juju-core/?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=ci+regression+&field.tags_combinator=ALL&field.has_cve.used=&field.omit_dupes.used=&field.07:48
wallyworld_omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on07:48
wallyworld_i forgot to filter on critical07:49
wallyworld_axw: i gotta run to soccer, i'll look at your PR when I get back07:50
axwwallyworld_: cheers, have fun07:50
voidspacejam: http://pastebin.ubuntu.com/8221800/08:28
voidspacejam: an unstable replicaset has members in state Recovering08:29
voidspaceand sometimes state Unknown it seems08:30
voidspace(states 3 & 6 respectively)08:31
voidspacejust setting up an lxc with nbb so I can hammer replicasets with a slow disk08:31
voidspaceAfter adding members they start in Unknown (state 6)08:32
voidspaceremoving members causes some of them to go into Recovering (state 3)08:32
fwereadedimitern, ping08:47
dimiternfwereade, hey09:00
fwereadedimitern, I have been adding a few comments to https://github.com/juju/juju/pull/517/files that I think are relevant to your interests09:01
jamvoidspace: so that seems ok, though I think we need some amount of logic about how long we'd be willing to wait.09:01
fwereadedimitern, cast an eye over it and let me know if anything springs to mind09:01
dimiternfwereade, sure, will do09:04
dimiternfwereade, thanks for taking the time to review it09:04
fwereadedimitern, sorry that one's been languishing so long :(09:04
dimiternfwereade, I'm planing to update the document later today and convert comments to proposals, where relevant09:05
fwereadedimitern, awesome09:05
dimiternfwereade, no worries09:05
perrito666morning09:25
voidspacejam: sure, that just returns true or false09:40
voidspacejam: we could use that in an attemptLoop (for example)09:40
mattywmgz, ping?09:41
voidspacejam: another question is, do we always wait for *all* members to be healthy09:42
jamvoidspace: for the purposes of the test I think we do09:43
jamvoidspace: for "realsiez" we probably just wait for the majority ?09:43
voidspacejam: right, but don't we want something that backup (et al) can use09:43
voidspaceright09:43
mattywwho would like to talk to me about the presence watcher?09:46
mattywfwereade, quick favour?09:59
fwereademattyw, what can I do for you?09:59
mattywfwereade, I've made a new pr for my metric cleanup pr: https://github.com/juju/juju/pull/665. The bot was ignoring $$merge$$ on the original10:00
mattywfwereade, could you just give it a once over to confirm it's the same as the one that has already been LGTM'd10:00
mattywit should be exactly the same10:00
fwereademattyw, as long as it is the same, which you'd know best, I'm happy to trust you to self-LGTM with a link to the original PR for context10:01
mattywfwereade, ok thanks10:01
mattywfwereade, who's a good person to pester about the presence watcher?10:02
voidspacejam: I need to leave for hospital appointment. *Probably* not back in time for standup. I have a branch with extraneous "Remove" removed - works fine but not yet tested with a slow disk. I also know how to check replicaset health (as discussed) but also want to test that on a slow disk. I'm now getting nbd working and mounted (wrestling a bit with nbd-server config). Will then create an lxc on the nbd mounted disk.10:03
voidspaceCreating an lxc container with the backing filesystem on another disk is straightforward. I should be able to share home directory without having to setup a full dev environment.10:04
voidspaceIt's only mongo that needs to be running on the ndb device, not jujud.10:05
voidspaceI might then need to look at trickle, but just using nbd should add a significant latency I would expect.10:05
voidspaceanyway, gotta go10:05
jamvoidspace: hope it all goes well10:10
fwereadejam, dimitern has a power cut10:17
jamfwereade: thanks for relaying the message10:25
jamTheMue: looks like its just you and me today10:46
jamapparently it was just me10:56
hazmatwhat's the trick to not have bootstrap destroy the environment on failure?10:59
hazmatah.. keep-broken10:59
=== urulama is now known as uru-bot
=== uru_ is now known as urulama
mattywperrito666, ping?12:02
perrito666mattyw:12:02
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Open critical bugs: 1348477 1364410 1359837
natefinchdamn, the day  I need stuff reviewed is the day I'm on-call :/13:08
perrito666natefinch: well, self review :p13:11
natefinchperrito666: doesn't matter, CI is blocked anyway, just realized13:12
perrito666natefinch: I am finishing a couple of tests and then I can take a look to the blockers if no one else is on them13:15
mattywfolks, I'm looking into this bug: https://bugs.launchpad.net/juju-core/+bug/1348477. I'm trying to work out if there is any significance in us calling s.State.SetAdminMongoPassword("") right before we attempt to close. I wonder if making that call while the presence worker is doing it's thing is causing the error13:19
mupBug #1348477: userAuthenticatorSuite.TearDown failure <ci> <regression> <test-failure> <juju-core:Triaged by cmars> <https://launchpad.net/bugs/1348477>13:19
mattywbecause the auth fails error is coming from the presence watcher13:20
jcw4rick_h_: I want to make sure we have a good corpus of use cases for Actions from the perspective of the GUI - is there someone I should work with from your team?13:21
rick_h_jcw4: we were actually just talking about how we could use actions for our current work13:22
jcw4rick_h_: sweet13:22
rick_h_jcw4: but for a corpus, we've not built a list yet. I'd send an email to juju-dev mailing list and use the power of the masses to generate a list.13:23
jcw4rick_h_: +113:23
rick_h_jcw4: and we'll be sure to reply with out use case13:24
jcw4rick_h_: thanks13:24
marcoceppiGood morning, seeing an issue that I confirmed was not in the cloud images (as best as I can tell)13:25
marcoceppiOpenstack charms are failing (and any charm with charmhelpers) to deploy on manual provider but not any other provider (including LXC on manual provider)13:26
marcoceppiDoes manual provider use cloud-init to to setup the image?13:26
natefinchmarcoceppi: I don't think it can.  The whole point is that it's an already-running machine that you want to plop juju on.  Pretty sure it just ssh's in and runs stuff.13:27
marcoceppinatefinch: that's the issue then, well at least thats what it seems to be13:27
marcoceppipython-yaml isn't being installed, likely because of lack of cloud-initing, causing a discrepency in all other provider images and the "manual" provider experience13:28
wwitzel3natefinch: ping13:29
natefinchwwitzel3: howdy13:30
wwitzel3natefinch: I'm in moonstone13:31
perrito666I need to chose a rommie or I will be automatically added :p anyone wants to be my roomate? I wake up too early, take showers equally early and sleep late :p and I might disassemble my laptop13:31
wwitzel3natefinch: if you have time for the 1on113:31
natefinchwwitzel3: ahh yeah, sure13:31
natefinchwwitzel3: belay that... kiddo has 102.8 temp... back in a bit13:33
wwitzel3np13:33
perrito666ouch that is like 39C :(13:37
mattywsinzui, ping?13:46
sinzuihi matty13:47
mattywsinzui, good morning, I'm still looking into https://bugs.launchpad.net/juju-core/+bug/1348477. Do you think it should be blocking landing?13:48
mupBug #1348477: userAuthenticatorSuite.TearDown failure <ci> <regression> <test-failure> <juju-core:Triaged by cmars> <https://launchpad.net/bugs/1348477>13:48
sinzuimattyw, I think it should, though it didn't appear in the last three test runs. I would like to say the bug has to prove itself to be rare13:49
mattywsinzui, ok13:50
mattywsinzui, understoof13:50
mattywsinzui, also understood13:50
alexisbTheMue, I will be missing our 1x1 today as I have a customer meeting13:52
mattywperrito666, ping?13:53
perrito666mattyw: pong13:54
* perrito666 match, game, set13:54
TheMuealexisb: ok, almost forgot it. good that you remind me. ;)13:54
perrito666or something like that, I never understood those sports scoring systems13:54
natefinchalexisb, wwitzel3, hazmat: I'm going to miss the TOSCA call in a few minutes... need to take my daughter to the pediatrician's office.13:54
TheMuealexisb: beside my current fight with versioning ;) I also have nothing special13:55
perrito666natefinch: ok, keep ups posted on how she goes and divert our way anything that we can take care of for you13:55
mattywperrito666, in state/presence/presence.go:231. If you change the period from time.Second to time.Millisecond you can make the auth fails error much more often I think13:55
perrito666mattyw: sweeeeeet13:55
alexisbnatefinch, hope all is ok13:58
alexisbwwitzel3, you joining us?14:02
wwitzel3alexisb: yep, sorry14:05
dimiternalexisb, thanks for the travel approval,  i've sent a booking request to bts14:32
alexisbdimitern, sweet14:32
dimiternanyone willing to review my fatal #666 PR? :) https://github.com/juju/juju/pull/666 - added more error tracing and logging to help catch a few CI bugs14:37
dimiternnatefinch, wallyworld_, voidspace, fwereade, others? ^^14:38
* fwereade is briefly irrationally jealous of dimitern14:39
voidspacedimitern: looking14:39
dimiternthanks! :)14:39
perrito666dimitern: I presume I am the person for that14:39
voidspaceI finally have an lxc container created inside an nbd device14:39
dimiternhehe14:39
voidspaceit took ages, I couldn't get the standard way to work at all and had to force the server to start without config file and using "old style" exports (so I could specify port manually)14:40
voidspaceany other way and nbd-server just failed to do anything14:40
voidspaceusing the lxc container seems appropriately slow14:41
dimiternI know it's bad, but if the PR looks ok, I'll try merging it with __fixes-1348477__ to overcome the bot block and see more context for the errors14:41
voidspacenow to see if I can get juju tests running inside it14:41
perrito666dimitern: no need there is a flag for that14:42
perrito666for "this will not fix anything but I really really need it up"14:42
dimiternperrito666, oh, what's that flag?14:42
perrito666JFDI14:42
dimiternoh :) i like it14:42
perrito666please provide a justification for it, since it has been abused in the past14:43
voidspacedimitern: what does errors.Trace do? wraps the error I presume14:43
mattywperrito666, is that right?14:43
perrito666mattyw: please disambiguate "that"14:43
mattywperrito666, JFDI14:44
perrito666yup14:44
mattywI'd feel bad using it, but that doesn't mean I'm not tempted14:44
voidspacedimitern: so this essentially fixes a bug caused by the fact that some errors are wrapped and some aren't?14:44
voidspaceplus adds more consistent error handling14:45
perrito666mattyw: you should not be using it unless you have a very good reason for it14:48
perrito666dimitern: the extra info you added could help discover wtf is happening with the auth error14:49
voidspacedimitern: and on line 815/816 you add a branch that explicitly doesn't wrap the error with Trace14:49
voidspacedimitern: is this because it's already wrapped, or some other reason14:49
voidspacedoes Cause recursively unwrap (root cause) or just one layer?14:50
dimiternvoidspace, the root cause14:51
voidspacedimitern: subject to those questions LGTM14:52
voidspacedimitern: and the reason for not using Trace on lines 815/816 of the diff?14:52
dimiternvoidspace, it's because tomb and some other packages like juju/txn check if err == SomeExactErrVar14:52
dimiternvoidspace, which file is that for lines 815/816 ?14:52
voidspacedimitern: right, so wrapping screws that up14:52
dimiternvoidspace, exactly14:53
perrito666state14:53
voidspacedimitern: state/state.go14:53
voidspaceah, line numbers are per file14:53
voidspace} else if err == jujutxn.ErrExcessiveContention {14:54
perrito666voidspace: yup, its a unified dif14:54
voidspacedimitern: for ErrExcessiveContention you explicitly avoid Trace14:54
dimiternvoidspace, yes, exactly because juju/txn internally check for ErrExcessiveContention and some other errors with if err == X14:54
voidspacedimitern: right14:54
voidspaceLGTM then14:54
perrito666voidspace: that is an error expected to be handled14:54
dimiternvoidspace, cheers!14:55
voidspaceah, perrito666 was doing it too :-)14:55
voidspacedouble LGTM14:55
perrito666dimitern: you could drop a comment there since its cause for doubts14:55
perrito666voidspace: is your lgtm as worthless as mine?14:55
perrito666:p14:55
dimiternperrito666, which comment to drop?14:56
voidspacehehe, not officially I believe14:56
voidspacebut in practise...14:56
dimiternperrito666, ah, you mean add a comment why ErrExcCont is not wrapped?14:56
dimiternperrito666, sure14:56
perrito666dimitern: sorry perhaps a bad translation, add a small comment on why you are not trace wrapping that particular err14:56
mattywdimitern, how much stuff do you know about the jujuconnsuite teardown logic?15:01
mattywcmars, and and I are looking into it now - there's a few bits that seem odd to us15:01
alexisbgsamfira, do we have a hangout for the meeting?15:01
gsamfirano, will you create one ?15:01
gsamfiraalexisb ^15:02
alexisbyep15:02
dimiternmattyw, not much, but my PR there aims to help debugging this mess15:03
alexisbgsamfira, sent you an invite15:03
mattywdimitern, cmars and I are looking into it now, we've a few ideas and we can recreate the issue *sort of*15:06
ericsnownatefinch, perrito666, wwitzel3: standup?15:07
perrito666ericsnow: nate is not available afaik15:07
perrito666wwitzel3: did you finish your meeting?15:08
wwitzel3yep15:08
wwitzel3if I have an endpoint can I get a relation_id from that? ..15:08
wwitzel3the int value that is15:08
natefinchericsnow: here but can't standup, sleeping sick baby on my lap and only one hand to type15:09
ericsnownatefinch: no worries15:09
natefinchped appt is later, unfortunately, but at least she's sleeping.15:10
perrito666natefinch: all is well I hope15:10
perrito666is the doc going to your house?15:11
perrito666wwitzel3: ?15:11
natefinchperrito666: nope, they don't do that here15:11
perrito666:(15:11
dimiternmattyw, sorry, in a meeting, so i'm responding when i can; i'm interested to hear your ideas how to repro it?15:12
mattywdimitern, it seems timing related so it's not sure fire: http://paste.ubuntu.com/8224259/15:14
ericsnowwwitzel3: standup?15:16
mattywtasdomas, was just looking at this: https://github.com/juju/juju/pull/66715:44
mattywtasdomas, I'm probably trying to do too many things at once but I couldn't work out what the significance of those changes is15:44
perrito666mattyw: hey, I am all yours now, can I give you a hand fixing the auth fails issue?15:46
=== hatch__ is now known as hatch
=== psivaa_ is now known as psivaa
mattywdimitern, ping?16:17
dimiternmattyw, pong16:19
mattywdimitern, do you have time to talk about this auth fails bug?16:19
dimiternmattyw, trying to merge my PR now, and if it happens to fail on the bot with auth fails or something, we'll see the logging/error tracing16:19
dimiternmattyw, yes, I have some time16:19
voidspaceheh, rate limiting my ndb drive to 200kps up/down with trickle means the lxc container living there will take about a week to start...16:54
voidspaceunless I kill it first...16:54
wwitzel3lol16:55
voidspacethe drive was running at 147 mb/s before - so 200kb/s is probably a bit too slow....16:55
wwitzel3why are you rate limiting it?16:56
voidspacewwitzel3: to simulate a slow disk17:04
wwitzel3voidspace: ahh, cool17:05
voidspacewwitzel3: to work with mongo replica sets17:05
voidspacewwitzel3: nbd lets you serve a volume over tcp and access it over tcp17:05
voidspacenbd-client and nbd-server17:05
voidspaceso I rate limit the server and then mount the volume17:05
voidspaceand there's an lxc container living on the rate limited volume17:05
voidspaceand the intention is to have mongo running on that17:06
voidspaceI haven't actually got that far yet17:06
voidspaceI think I'll need a reboot as I had to kill a mount command and now can't mount the volume...17:06
voidspacebut first - jogging...17:06
* natefinch is back19:07
natefinchericsnow: great writeup on your charm19:09
ericsnownatefinch: I hope it's useful19:10
ericsnownatefinch: like I said before, it went pretty smoothly and the only criticisms I have are pretty mild19:10
natefinchericsnow: definitely.  It gives me a lot to think about to compare with my experience, which I think was more frustrating than yours, not that it wasn't insurmountable.19:11
ericsnownatefinch: the order of operations I outlined is mostly a best guess and undoubtedly not accurate, but should capture the bulk of what I did19:12
natefinchericsnow: maybe I expect too much, but I was annoyed with a lot of the process, most of which is our own fault19:16
natefinchericsnow: like debugging a hook.... I have one unit of one service deployed, it has one failed hook..... juju debug-hooks should just do the right thing and put me into the hook context on that machine.19:17
ericsnownatefinch: oh, I didn't bother with that stuff.  I opened the logs directly, tweaked my charm accordingly, removed it, and re-deployed19:19
ericsnownatefinch: I never tried debug-hooks, and only tried debug-log once19:19
natefinchericsnow: I had to do my testing on Amazon, because my charm didn't work in LXC, so re-deploying was painful19:19
ericsnownatefinch: ah, so maybe that is the big difference19:19
ericsnownatefinch: it would have been much more frustrating if it hadn't been on local provider19:20
natefinchericsnow: yeah, a lot of my complaints were around that19:20
natefinchericsnow: I also found the local repository stuff to be unecessarily complicated.  Why can't I just say juju deploy --local=<path> ?19:23
ericsnownatefinch: oh, yeah, that's a good one (I would have had it on my writeup if I'd remembered)19:24
natefinchericsnow: btw did you send that to anyone else or just me? :)19:27
ericsnownatefinch: just you, I figured you would know to whom to forward it19:28
ericsnownatefinch: or if you like I can just post it to the juju-dev list (or some other more appropriate list)19:28
natefinchWriting to juju-dev is a good idea19:29
ericsnownatefinch: will do19:29
ericsnownatefinch: done19:47
natefinchmarcoceppi: if I use --config when I deploy, can I access those config variables during the install hook with config-get?20:06
marcoceppinatefinch: theoretically, yes20:07
* natefinch squints at marcoceppi 20:07
natefinchmarcoceppi: what does that mean? :)20:07
marcoceppinatefinch: yes, I'm like 90% sure you can20:07
marcoceppias in, I can't remember, but I'm pretty sure you can20:07
natefinchok :)20:07
natefinchthat's cool20:07
marcoceppiyou have access to everything up until the execution of that hook context, then it's locked until next hook20:08
marcoceppiso you can even juju set before the install hook runs and it'll make it in20:08
marcoceppiagain 90% sure20:08
natefinchI would hope so, but it didn't occur to me until just now that I might be able to.  It helps skip a restart if I can prepare config during install20:08
ericsnowmarcoceppi: I think you're right because that behavior actually broke a charm I'm using :P20:12
natefinchhaha20:14
natefinchsorry20:14
ericsnowperrito666: too bad you didn't get PR #666 :)20:21
TheMueso, next step reached, tests run, merge done, conflicts resolved. time to go to bed21:05
TheMuegood n8 folks21:05
wallyworld_katco`: hi, how was your day?21:19
thumperwallyworld_: what is the current plan with the ci regression blockers?21:28
thumperwallyworld_: do we have one?21:28
rick_h_and with split diffs there was much rejoicing! https://github.com/juju/juju-gui/pull/526/files?diff=split21:28
wallyworld_thumper: i understood that matt was going to ask about taking those intermittent test failures off the blocker list last night21:29
wallyworld_i commented on one of the bugs that i didn't think it was a regression21:30
sinzuiwallyworld_, What info do we need to ask for to know how to fix bug 136503521:30
mupBug #1365035: MAAS provider bootstrap: Timeout, server <server> not responding. <bootstrap> <cloud-installer> <landscape> <maas-provider> <timeout> <tools> <juju-core:Triaged> <https://launchpad.net/bugs/1365035>21:30
wallyworld_sinzui: i'll read the bug, sec21:30
thumperrick_h_: nice21:31
katco`wallyworld_: good... 3 PRs up now21:31
wallyworld_katco`: 3!!!21:31
katco`wallyworld_: well not all today21:31
katco`wallyworld_: just 1 today haha21:31
katco`wallyworld_: working on another21:31
wallyworld_katco`: i'm ocr today but also 1/2 my day is filled with meetings, sigh21:32
katco`wallyworld_: have i mentioned how nice of a person you are? :)21:32
wallyworld_katco`: maybe, but you cn remind me any time :-)21:32
katco`wallyworld_: oh great wally, ruler of all things -- for this is your world -- please look upon my PRs favorably21:33
wallyworld_lol21:33
wallyworld_katco`: and sacrifices, i love sacrifices21:34
sinzuiwallyworld_, okay, I see the bug I was looking to backport was already targeted by you to 1.20.821:34
* katco` sacrifices bugs at the alter of wally21:34
wallyworld_sinzui: i'm not sure how "Timeout, server tesla.beretstack not responding." isn't a network failure?21:34
sinzuiwallyworld_, yep21:34
sinzuiwallyworld_, the attached log looks a lot like cloud-init-output.log. I think cloud-init did its job and that the agent could talk to the other party21:35
wallyworld_sinzui: yeah, looks like it. let's put our head in the sand till 1.20.7 is out and they can have the option to leave the broken system running and then poke around21:36
wallyworld_sinzui: so what was the other bug?21:37
sinzuiwallyworld_, bug 136137421:37
mupBug #1361374: maas provider assumes machine uses dhcp for eth0 <addressability> <maas-provider> <network> <juju-core:Fix Committed by dimitern> <juju-core 1.20:Triaged> <https://launchpad.net/bugs/1361374>21:37
wallyworld_sinzui: ok, i was asked about that one by jorge so i already added it to 1.20.8 to be backported21:38
sinzuiwallyworld_, I was pretending I was going to release 1.21-alpha1 today so I started reviewing all the bugs21:38
wallyworld_there's a lot of them21:38
sinzuikatco`, you are awesome. You are in a comfortable third place in fixes https://launchpad.net/juju-core/+milestone/1.21-alpha121:40
waigani_thumper: testing server side is done. How shall I test the client side? mock out the FacadeCall ?21:40
katco`sinzui: wow, really??21:40
wallyworld_sinzui: with the ci blockers - i thinks there's a couple of intermittent test failures marked as regressions. i don't think they should be as those issues have been around for a while and people are working on improving the tests now for part of every week and there's no way we can quickly fix those21:41
sinzuiwallyworld_, when we are pressured to release unblessed code, those tests are in the way.21:42
thumperwaigani_: ideally the client should test against a mock for all but one to show that it is in fact connected21:42
waigani_right, I remember you saying that now21:42
wallyworld_sinzui: they are agreed. but they are intermittent failures and hard to track down. i fear we will be blocked for a long time if we keep them21:43
waigani_thumper: so one unit test which uses the real facade and others using a mock21:43
thumperwaigani_: yes21:43
sinzuiwallyworld_, given that ci has only 6 blessed revisions for 1.20.6 after 10 weeks, I think someone can easily say I haven't been strictly enforcing quality. I have a mortgage to pay21:45
* wallyworld__ is so sick of this kernel bug killing his network all the time :-(21:47
mattywwallyworld__, sinzui I spent time today looking into https://bugs.launchpad.net/juju-core/+bug/1348477. I have some idea what's causing it, and I think I can almost reproduce it on demand21:49
mupBug #1348477: userAuthenticatorSuite.TearDown failure <ci> <intermittent-failure> <regression> <test-failure> <juju-core:Triaged by cmars> <https://launchpad.net/bugs/1348477>21:49
mattyw^^ but I need to talk to someone more familiar with the code to know for sure how to fix it21:50
cmarsmattyw, that's great news21:50
wallyworld__mattyw: awesome21:50
wallyworld__mattyw: what are your thoughts?21:50
perrito666nites people21:52
mattywwallyworld__, making this change is enough to cause the error to happen on demand: http://paste.ubuntu.com/8227111/21:53
mattywwallyworld__, although it's still timing related so the more tests you run the more likely you are to see it21:53
mattywwallyworld__, and it appears that the call to SetAdminMongoPassword in juju/testing/conn.go:434 is deleting the admin user - quite a number of watchers access state directly using the admin user. so if that admin user is deleted and a watcher does some work before state is closed you'll get that error21:55
wallyworld__mattyw: yeah, that's part of the problem - a lot of our tests suck because of timing issues so running them on different platforms triggers issues21:55
wallyworld__mattyw: hmm, i had thought that people had removed direct access to mongo from the business logic21:56
mattywwallyworld__, I experimented with splitting the state.Close call into two function. One to stop the watchers and another to close the session - that seemed to fix that problem - but we got more errors during the call to dummy.Reset()21:56
wallyworld__sounds like a good start21:57
sinzuithere are 168 tested commits in 1.20-alpha1, only 11 passed. all of which were 2 weeks ago21:57
perrito666mattyw: hey man, any luck?21:57
mattywperrito666, we have a mongo log that more or less shows us trying to connect with an old user name21:58
sinzuiwallyworld__, mattyw ^ I know I am being difficult. We cannot delude ourselves into thinking it is okay to add features when there is no evidence that this juju version is good21:59
wallyworld__sinzui: i agree except that we know the test failures are due to issues with the tests22:00
mattywsinzui, I understand22:01
mattywsinzui, wallyworld__ if we could make the auth fails bug less likely to happen is that something that would be useful in the interim?22:01
wallyworld__mattyw: sounds good to me22:02
mattywwallyworld__, I'll be going to bed soon - but in the morning I'll see if I can land some stuff to make it less likely to happen - although obviously  whether or not it actually works enough for ci to run will remain to be seen22:04
mattywsinzui, does that sound acceptable to you?22:04
mattywsinzui, also, I think this has been fixed already https://bugs.launchpad.net/juju-core/+bug/136512422:05
mupBug #1365124: "juju deploy --to <non-existent-machine> <charm-name>" juju still tries to deploy the service. <deploy> <placement> <juju-core:New> <https://launchpad.net/bugs/1365124>22:05
sinzuimattyw, me too, I couldn't find the bug...22:06
sinzuimattyw, but I can get an interesting error reproducing it Juju tells me know way22:06
mattywsinzui, ?22:07
sinzuimattyw, status still lists am impossible service22:08
mattywsinzui, werid - if you send me the steps to reproduce it I'd be happy to take a look tomorrow22:09
cmarssinzui, copy me on that one as well plz22:09
* thumper takes a deep breath22:12
thumperdown to only 6 failing tests, and all in cmd/juju22:12
mattywthumper, failing how?22:12
thumpermattyw: I've been removing "admin"22:12
mattywthumper, admin?22:12
thumpermattyw: the first user is no longer "admin", but the name of the logged in user22:13
thumpermattyw: has far reaching impact22:13
thumpercurrent unified diff is over 4k22:13
thumperwill break it down somewhat22:13
mattywthumper, wow22:13
mattywthumper, what have you done with Mr cheney?22:14
thumpermattyw: what do you mean?22:14
mattywthumper, he's top of my list of people I want to talk to22:14
thumpermattyw: he starts in about 45 min22:14
mattywthumper, oh right yeah - I was sure he was already up this time yesterday22:15
thumpermattyw: there was an early meeting yesterday22:15
mattywthumper, that's pretty inconsiderate22:15
perrito666mattyw: its like 11pm for you right?22:17
mattywperrito666, yeah - I'm not really working22:17
mattywperrito666, don't worry - I'm not that dedicated22:18
perrito666your non working you is amazingly similar to your working you22:18
perrito666the proof is on the private ping you just answered ona working communication mean22:18
perrito666:p22:18
mattywperrito666, that's laziness - not closing the connection there22:19
voidspacewwitzel3: ping22:19
* perrito666 has no moral grounds for this discussion22:19
mattywperrito666, isn't it 9pm for you?22:20
perrito666nah 7:3022:20
perrito666but I will be here at 9:3022:20
perrito666and even later lol22:20
perrito666brb22:20
voidspac_wwitzel3: ping22:38
* perrito666 chopping onions and reading flaky test, not sure which one is the one provoking the crying23:21
thumperperrito666: are you looking at mattyw's branch and the flakey teardown?23:21
perrito666thumper: the teardown in tip23:23
davecheneyperrito666: if you are working on that bug23:24
davecheneyplease mark it in progress23:24
perrito666davecheney: I am working on another caused by the same Issue, Ill mark it as soon as I make sure tim and I are not working on the same thing :p23:25
thumperok... tests pass23:38
* thumper runs make check to test23:38
menn0davecheney, thumper: so I've been looking at that CI blocker and the way we find a free port to use for API servers and mongod in tests is crazy23:54
menn0just like davecheney said23:54
menn0fixing it is hard though23:54
davecheneymenn0: i found the bit that finds a port for the api server and it is sane23:57
davecheneybut the way we do for mongo is not sane23:57
davecheneymenn0: suggestion, add one to the port the port finding thinggy thinks23:57
menn0davecheney: the same function and approach is used for both23:57
davecheneymenn0: nah, for mongo we bind, then close then give that address to mongo23:58
davecheneyfor the api server it shuld just do a bind :023:58
davecheneyand use that listener23:58
menn0davecheney: adding 1 to the port isn't much better. some other process is just as likely to have grabbed that one23:58
davecheneymenn0: i don't believe so23:58
davecheneywe have a roughly 1:10000 chance of two tests getting the same port23:58
davecheneymy assertoin is the Close() leaves the port still in use23:59
davecheneyfor a very short amount of time23:59
davecheneyso mongo can't bind to that port23:59
menn0davecheney: for the jujud tests (i.e. what that ticket is talking about) FindTCPPort is used to generate the state server config and there could be a signficant time between when the port was determined and when the API server is started23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!