/srv/irclogs.ubuntu.com/2013/08/20/#juju-dev.txt

* davecheney waves at thumper00:14
bigjoolsarosales: hi00:18
bigjoolswhat governs which tools get picked?00:24
bigjoolsthat's not provider-specific is it?00:24
davecheneybigjools: it is and it isn't00:24
bigjoolshelpful :)00:24
davecheneythe version is not provider specific, it is determined by the version rules we have00:24
davecheneyie, if x.y.z, y % 2 == 0 then we'll look for stable tools with an exact match00:25
davecheneyif y % 2 != 0 then it's a devel version and we take the latest .z version00:25
davecheneyhowever there is a bigger problem00:25
davecheneyfor each provider, there are two locations to look for tools00:26
bigjoolshttp://paste.ubuntu.com/6004647/00:26
davecheneythe public (shared) bucket00:26
davecheneyand private (control) bucket00:26
davecheneyif no public bucket exists, or does not match the exct version for released tools00:26
davecheneyi *think* the current behavior is to invoke the juju sync-tools function00:26
davecheneycopy all the tools from the master bucket in s300:26
davecheneyinto your private control bucket00:27
davecheneythis control bucket will be deleted when you destroy the environment00:27
bigjoolsnot sure I really understand tbh00:27
bigjoolsmy knowledge of the juju internals is pretty limited00:28
davecheneybigjools: two secs, gotta hang out the washing00:29
davecheneythen I will return and regail you with tails of woe00:29
bigjoolshanging out the washing ... love it00:35
bigjoolswe've not heavily tested (read: no testing) of using public tools with azure yet00:36
davecheneybigjools: +1 for testing00:39
davecheneybigjools: i'm probably wrong00:39
davecheneybut does azure have the concept of a public bucket ?00:39
bigjoolsyou can have public containers, if that's the same thing00:40
bigjoolsbut I don't think ec2's concepts map directly00:40
bigjoolsnobody uploaded public tools to azure yet I guess00:42
davecheneybigjools: nope, if00:42
davecheney1. we do that00:42
davecheney2. provide some sort of link to then in environments.yaml00:43
davecheneythis will solve the problem00:43
bigjoolsthere is config for it00:43
davecheneybigjools: so I think what happens is00:43
davecheneyyou manually download the tools from the s3 bucket00:44
davecheneythen manually upload them to the azure public bucket00:44
davecheneyyou might be able to arrange you environments.yaml to be able to use sync-tools to automate this00:44
davecheneywally/jam/gz could probably explain how to do this will less pain00:44
bigjoolsthere is no public Canonical storage account in Azure AFAIK00:45
* bigjools starting to understand00:45
bigjoolsthere's config for public-storage-account-name and public-storage-container-name once tools are uploaded00:45
bigjoolswho generally uploads new tools?00:47
davecheneybigjools: unknown00:49
davecheneyit's handled very ad hockly00:49
bigjoolsso when there's a new release, how does it work?00:50
davecheneythere is a vague requirement that 'tools should be uploaded as part of the release'00:50
davecheneybut that assumes I/we/someone knows where all the places tools sohuld be uploaded00:50
davecheneyat the moment it's just ec2 and hp cloud providers00:50
davecheneyif you're not one of those00:50
davecheneyyou're probably on your own00:50
bigjools\o/00:50
davecheneythis is why wallyworld_ and I want to stop doing the public bucket00:50
davecheneyand distribute the tools as part of the client00:50
bigjoolswell put it like this, we're no longer doing azure, so I don't think it's us who are on our own00:51
wallyworld_the simplestreams stuff i am working on will solve a lot of this00:51
bigjoolshey wallyworld_ how's the boat race?00:51
wallyworld_not too bad. still fucking ugly00:51
wallyworld_modern medicine isn't that advanced yet00:52
bigjoolsthe surgeon didn't give you chiselled good looks then00:52
wallyworld_nah, he said something about silk purse, sow's ear00:52
* davecheney hands out 'hoff masks00:52
bigjoolswallyworld_, davecheney: I won't be around for the first day of the sprint, BTW. I have to drive to Maleny.00:52
davecheneybigjools: ok, i don't know where maleny is00:53
davecheneybut it sounds like a long way00:53
bigjoolsabout 100km north00:53
wallyworld_it00:53
wallyworld_'s where he gets his weed from00:53
davecheneybigjools: good, that means you won't be around to tell wallyworld_ and I to not drink wine00:53
bigjools*cough*00:53
bigjoolsI am sure you guys will cope00:53
davecheneybigjools: it would be better if you could bring the choof on the first day00:54
davecheneybut we'll manage somehow00:54
wallyworld_i'll have to leave the hotel early most days to get my kid from school00:54
bigjoolshe'll miss out on the evening festivities then00:54
wallyworld_i also have soccer in the evenings :-( except for tuesday00:55
* davecheney thinks this sprint is a wonderful use of our travel budget00:55
bigjoolsJust send them to the Candy Club00:55
axwprobably should've just left my induction till October :)00:56
davecheneyaxw: it's not your fault00:56
davecheneyand waiting to october would be lame00:56
bigjoolsyeah it's not nice to wait that long00:57
* bigjools not bitter00:57
axwwallyworld_: not sure if you've seen this yet: https://codereview.appspot.com/12926045/01:10
wallyworld_i haven't will look01:10
axwthis fixes the all-machines.log spewing bug, but I don't quite understand why :)01:10
axwthanks01:10
davecheneyaxw: thanks, looking01:11
axwI understand what that line does, but not why its omission would lead to repetition01:11
axwdavecheney: thanks01:11
wallyworld_axw: does this replace the other work that was done in the previous merge proposal?01:13
axwwallyworld_: yes, that was all unnecessary. sorry, I totally misdiagnosed it01:13
axwI scrapped that MP01:13
davecheneyif there are bugs that have been fixed, but not marked as 1.13.201:23
davecheneyhttps://launchpad.net/juju-core/+milestone/1.13.201:23
davecheneycould you please do so01:23
wallyworld_axw: i can't see why adding that break stops the repetition either, since the next rule doesn't match and there's so subsequent contents in the conf file. but if it works, why not i guess01:25
davecheneywallyworld_: lets just hulk smash this fucker01:26
wallyworld_yeah, it seems it fixes the problem, and doesn't do any obvious harm01:27
axwwallyworld_: ok. weird. perhaps an rsyslog bug01:27
wallyworld_maybe, who knows. or maybe i'm missing something obvious, which is more likely01:27
davecheneywhy does juju wait til so late in the piece to tell me the env is already boostrapped02:19
davecheneyhttp://paste.ubuntu.com/6004960/02:19
davecheneyespecially after I have taken destructive action in the bucket02:19
sidneifile a bug?02:20
davecheneysidnei: there is already one one02:20
davecheneyopen02:20
sidneicool, maybe noodles775 can take a look in the morning02:21
arosalesbigjools, back02:26
bigjoolsarosales: hey02:26
bigjoolsI commented on your bug02:26
bigjoolsyou need to use --upload-tools for now02:26
arosalesbigjools, davecheney I see you guys have had the juju release and tools discussion02:26
arosalesdavecheney, I thought you volunteered to upload tools as part of the release process ;-)02:27
arosalesbigjools, so I can create a public container in Azure, no problem02:27
davecheneyarosales: yes sir i did02:27
davecheneyhowever you'll remember the caviet 'to all known clouds'02:27
bigjoolswe don't have a tool to do that for azure yet02:27
davecheneythere is a potentially large set of unknown clouds02:28
* bigjools waits for the Donald Rumsfeld-like speech02:28
arosalesdavecheney, ah I was confused by:02:28
arosales<bigjools> who generally uploads new tools?02:28
arosales<davecheney> bigjools: unknown02:28
davecheneybigjools: you are indeeed correct02:28
arosaleswell lets just star with the clouds we know to work with juju atm02:29
bigjools\o/02:29
davecheneyarosales: the short answer is, i'll upload tools for clouds we know about02:29
davecheneywhich now includes azure02:29
arosaleslets not solve all the worlds problems, just yet02:29
davecheney(note, no credentials to do so btw)02:29
arosalesdavecheney, ack on Azure, we just go the green light to go ahead an test02:29
bigjoolswe need to create a storage account and container on Azure, for public juju tools02:29
davecheneywallyworld_: is there any chance simple streams will save us from this pain ?02:30
davecheneyin the near term ?02:30
arosalesdavecheney, I do think simple streams for juju tools solves the public bucket in all clouds issues02:30
davecheneyarosales: in what timeframe ?02:30
wallyworld_davecheney: i hope to have something done this week, maybe not for release. but CPCs should still have their own tools container02:30
arosalesat least just need to uploads tools to one place, and have juju look there02:30
arosalestime frame is dependent on juju-core and wallyworld_02:31
wallyworld_arosales: and IS02:31
* davecheney ages02:31
wallyworld_they are setting up the repository02:31
arosaleswallyworld_, have you filed an RT?02:31
wallyworld_arosales: yes, you were cc'ed on it :-)02:31
arosaleswallyworld_, cool I'll take you word for it.02:31
wallyworld_you didn't see any email?02:32
arosaleswallyworld_, CPCs need public buckets for faster boots?02:32
davecheneyarosales: they need it for any boot at all02:32
arosaleswallyworld_, I am  just behind on my mail.02:32
arosalesdavecheney, aside from why tools are needed02:32
arosaleswhy they need a local bucket02:32
wallyworld_arosales: yes, a CPC can mirror juju.canonical.com/tools for faster boot02:32
arosaleswallyworld_, so in theory they could go to juju.canonical.com02:33
wallyworld_once this stuff is done, there will be no need for a public bucket02:33
arosalesbut that would require out of cloud access02:33
davecheneyarosales: not following your last comment02:33
wallyworld_arosales: yes, it requires out of cloud access. hence clouds should be able to mirror the tools02:33
wallyworld_if they don't want that02:33
arosaleswallyworld_, so we will still need a public bucket once simple streams is all done02:34
arosalesbut that public bucket would just be a mirror of juju.canonical.com/tools02:34
arosalesdavecheney, sorry I am just trying to get public bucket and simple streams straight in my head.02:35
arosalesbigjools, for azure in the near term (before simple streams is done)02:35
davecheneyarosales: i can't help with that :)02:35
davecheneyi don't have it straight in mine02:35
bigjoolsarosales: --upload-tools is your friend for now02:36
arosalesbigjools, I can create an public container, but it seem uploading tools to a blob requires some azure api foo02:36
bigjoolsarosales: you can use the web ui as well02:37
arosalesbigjools, really?02:37
bigjoolswe have a command line tool but I am not sure of its state at the moment02:37
arosalesbigjools, I didn't find any way to upload a blob via the web, do you have a pointer?02:37
bigjoolsoh wait you're right, you can only download .... gah02:37
bigjoolsI have a tool to upload02:37
bigjoolsas assuming you set up the container in the web ui, we can do it02:38
arosalesI got a container set up via the azure web ui02:38
arosalesbigjools, storage name = jujutools02:39
arosalescontainer name = juju-tools02:39
arosalesblobs endpoint http://jujutools.blob.core.windows.net/02:40
* arosales not sure how to give access to write to it though . . .02:40
bigjoolsif you want me to upload something there for you, I need the storage key02:40
bigjoolsyou can get it from the management ui02:40
bigjoolsPM it :)02:40
arosalesbigjools, ack02:40
davecheneyhttps://bugs.launchpad.net/juju-core/+bug/121420902:41
_mup_Bug #1214209: cmd/juju: add-machine should take a -n param <papercut> <juju-core:Triaged> <https://launchpad.net/bugs/1214209>02:41
bigjoolsbugs with the would "should" in the title need to be instantly marked invalid02:41
bigjoolsword*02:41
bigjoolscan't type today02:41
davecheneybigjools: i think you meant02:43
davecheneybugs witht he word "should" in the title MUST been instantly marked invalid02:43
bigjoolsyou are possibly not wrong02:43
davecheneybigjools: i can edit the titile if required02:44
wallyworld_arosales: sorry, was otp. each cpc would have a repository to mirror the tools, but it won't need to be configured as a public bucket in juju02:44
davecheneyarosales: public buckets in juju are really weird02:45
davecheneythey are a bastardised version of the private (control) bucket02:45
wallyworld_so no more of that public-bucket-url stuff needing to be configured in each client02:45
davecheneyso can't be shared across providers02:45
arosaleswallyworld_, and for cpc we should know what the public bucket is02:45
davecheneyarosales: why ? how would that information be used ?02:46
davecheneyaxw: w1/0:config-changed %02:46
davecheney^ thanks02:46
davecheneyworks well02:46
wallyworld_arosales: there will be a url used to access the tools, but it will be in the simple streams mirrors definitin, not in juju02:46
wallyworld_so users don't need to care02:46
axwdavecheney: cool.02:49
arosaleswallyworld_, so are you saying that juju tools simple stream definition will also list the known cpc public bucket/juju tool mirror locations?02:50
wallyworld_arosales: yes02:50
arosaleswallyworld_, ok, I think the light is slowly staring to turn on02:50
arosaleswallyworld_, so you still need info on where a particular cloud is housing the juju-tools, correct?02:51
wallyworld_you = juju folks maintaining the official tools repo02:52
wallyworld_not users02:52
arosalescorrect02:52
davecheneyarosales: and the code to retrieve the tools from a bucket indicated by simple streams data is provider specific02:52
wallyworld_arosales: so, juju will look locally within a cloud first. failing that it will look at juju.canonical.com/tools and it can always get tools from there. but, if a mirror has been set up, the tools will be fetched locally02:52
arosaleswallyworld_, yes, sorry for not clarifying02:52
arosalesdavecheney, agreed02:53
wallyworld_tgere will also be user config where a dev/user can set up their own tools02:53
wallyworld_and also their own image metadata02:53
wallyworld_so they can experiment/test etc02:53
arosaleswallyworld_, ack on the user config02:53
arosaleswallyworld_, when you say, " juju will look locally within a cloud first."02:53
arosaleswe need to tell juju where to look locally first02:54
arosalesand that is specific for each cloud02:54
arosalesand I think we still need to set that up, correct02:54
wallyworld_arosales: for openstack, the cloud admin has the option of setting up keystone catalog items for the tools/image metadata02:54
wallyworld_so juju will look there02:54
wallyworld_but the end user doesn't know or care02:54
wallyworld_arosales: the keystone thing is how we make canonistack work for example02:55
wallyworld_it's not a cpc, but it just works regardless for users02:55
arosaleswallyworld_, given keystone is configured correct, and given it is an openstack cloud02:55
arosaleswallyworld_, but how would you handle say a joyent or azure cloud02:56
arosalesI am guessing we need to still tell juju where the local juju tools mirror is02:56
wallyworld_arosales: i'm not sure what facilities those have for a central config equivalent. but we will be introducing the idea of central, provider based config which will fill the gap there02:57
wallyworld_that last bit is still beig discussed02:57
wallyworld_as to how it will work02:57
wallyworld_arosales: but yes, in the short term, a url may well still be required for defining where local tools are02:58
wallyworld_the tools will always be available centrally02:58
wallyworld_from juju.canonical.com/tools02:58
arosaleswallyworld_, agreed02:58
wallyworld_for CPCs, no url though :-)02:59
arosalesI was just trying to tease out how you are handing non-openstack clouds that _need_ a local copy of the juju tools02:59
arosalesand how you make that just work for the user.02:59
wallyworld_sure, understood :-)02:59
arosalessounds like that is still be in solved?02:59
arosaleswallyworld_, from what I can decipher I think canonical will still need to host a mirror of the juju tools for non-openstack based clouds.03:01
arosalesand then tell juju what that hosting location is, so juju can look there first before going to juju.canonical.com/tools03:01
wallyworld_arosales: yes, that's what juju.canonical.com/tools will be03:02
bigjoolsok where can I download the current tools03:02
arosalesbigjools, http://juju-dist.s3.amazonaws.com/03:02
wallyworld_http://juju-dist.s3.amazonaws.com/tools03:02
arosaleswallyworld_, I think we went full circle03:03
arosaleswallyworld_,  :-)03:03
wallyworld_lol, yeah03:03
arosaleswallyworld_, so ack on juju.c.c/tools being a central _out_ of cloud location03:03
wallyworld_yep03:04
wallyworld_out of cloud is the key. also, it will be master copy03:04
arosalesbut, for Azure, canonical still needs to host a local mirror of the juju tools somplace03:04
arosaleslocal == in cloud03:04
wallyworld_within azure?03:04
arosalescorrect03:04
wallyworld_yes, you are right. we do that for HP and AWS03:05
arosalesand when simple streams is all done and tidy03:05
wallyworld_that's where the mirror def will point to03:05
arosaleswe will still need to host the juju tools mirrors in these locations03:05
wallyworld_yes03:05
arosalesand then tell juju what these locations are, correct?03:05
wallyworld_so, for hp for example, juju looks at juju.c.com/tools, see there's a local mirror on hp, and gets the tools from there03:06
wallyworld_it's the same public bucket as now03:06
wallyworld_but it's transparent03:06
arosalesgotcha03:06
arosalesbut we specifically told juju what that public bucket is03:06
wallyworld_yes, in the mirror metadata03:07
wallyworld_but not in user env.yaml :-)03:07
arosalesagreed03:07
wallyworld_so we need to know, not the user \o/03:07
arosalesso canonical still needs to setup and host a juju tools in Azure03:07
wallyworld_yeah03:07
arosalesand once we know what that is, the juju tools mirror metadata needs to know the location03:08
wallyworld_and allow for public access without credentials03:08
wallyworld_yes, we can update the mirror files03:08
arosaleswallyworld_, well any writing will needs creds03:08
wallyworld_yes03:08
wallyworld_sorry, i meant read access03:08
arosalesack03:08
arosaleswallyworld_, so a lot of these buckets are a little ad hoc . . .03:09
arosaleswallyworld_, I would suggest that with the juju.c.c/tools RT03:09
arosaleswe open up the conversation to IS maintaining these local  _in cloud_ juju tools mirrors03:09
arosalesIS maintains the CPC mirrors, so perhaps also having a bucket/container in that mirror also host the juju tools mirror may be wise .  . .03:10
wallyworld_agreed. the exact process still needs to be massaged. but yeah03:10
arosaleswallyworld_, also it sounds like we can change the _in cloud_ endpoint and it will be transparent to users03:11
wallyworld_yes indeed03:11
wallyworld_we still need to sort out signing though03:11
wallyworld_did i copy you on my email reply to (scott i think) on that topic?03:12
arosaleswallyworld_, you may have . .  . .03:13
wallyworld_i basically think that signing right now is adhoc and needs to be centralised03:14
wallyworld_managed via a tool chain and not individually on laptops etc03:15
arosalesah I think that was part of the "juju work" thread03:15
wallyworld_maybe, was more than a day ago so my fifo brain has wiped it :-)03:15
arosaleswallyworld_, btw I also see your RT 63925, for juju.c.c/tools03:16
wallyworld_cool03:16
arosaleswallyworld_, I just forgot :-/03:16
wallyworld_np. happens to me all the time :-)03:16
arosaleswallyworld_, exactly on my fifo brain03:16
arosalesvery limited queue03:16
wallyworld_lol03:16
wallyworld_yeah, me too03:17
arosaleswallyworld_, so I am going to work with bigjools on setting up a "public bucket" for Azure03:17
wallyworld_excellent03:17
wallyworld_bigjools *loves* Azure :-D03:17
arosalesI don't think it will be the final one, but I think it will be the one we use until we get an IS official solution.03:17
bigjoolsfuck off03:17
arosaleslol03:17
bigjoolsand when you get there, fuck off again :)03:18
wallyworld_arosales: right now, the mirrors work is still in review, and i need to do tools to generate the files etc. so there's work to do03:18
arosaleswallyworld_, once we have what the "public bucket" is we will let you know so you can tool juju tools metadata03:18
wallyworld_ok03:18
wallyworld_bigjools: ok, i've done that, twice. now what?03:19
arosaleswallyworld_, thanks for the explanation on juju-tools mirroring. I'll work to ensure CPC accounts for some sort of juju-tools _in cloud_ miroring of juju tools and let you know about it.03:20
wallyworld_arosales: np. i'm still in the middle of doing the code for it all. expect a better explanation when it's all done. it's very raw and a wip at the moment03:21
arosaleswallyworld_, no worries. I appreciate your time. I just want to make sure we have all the bits for Azure as it comes online03:21
wallyworld_understood. will be great to have it all working03:22
arosalesdavecheney, I'll follow up with you on uploads once we know what that is for Azure03:22
wallyworld_bigjools says he want to help maintain it to, so keep him in the loop03:22
arosalesdavecheney, did I miss any other of your questions?03:22
bigjoolswallyworld_ said he wanted a full run-down next week so he could understand everything to do with azure03:23
wallyworld_nooooooooo03:23
bigjoolsit will be my duty to make sure he's fully up to speed03:23
* wallyworld_ just puffed on an exploding cigar03:23
bigjoolsis that a euphemism?03:36
wallyworld_you wish03:37
* thumper stabs the convuluted use of shit loads of structures in the juju codebase in the face and goes to make coffee03:48
arosalesbigjools, 2nd failure on " no reachable servers" with, perhaps the 3rd time is the charm04:21
bigjoolswallyworld_: is your coffee machine fixed?04:53
wallyworld_bigjools: no :-(04:53
bigjoolswallyworld_: then I won;t tell you how nice this one is that I just made04:53
wallyworld_i can't have hot liquids for a few days anyway04:53
wallyworld_fo04:53
bigjoolscan you have body temperature liquids?04:54
thumperbigjools: you can tell me how good your coffee is and wallyworld_ will be forced to listen05:03
wallyworld_bigjools: you volunteering?05:04
bigjoolsthumper: it is delicious.  I was even thinking of wallyworld_ when I did a latte art penis05:04
thumperI'm not very good at latte art05:04
bigjoolsit's not hard to do a cock and balls, just ask wallyworld_05:04
* thumper snorts05:05
thumperit is bloody hard to unpick the use of exported structures all over the show05:05
wallyworld_it's hard for bigjools to get them small enough if he wants to do a scale model of hos own05:05
thumperI'm replacing agent.Conf with an interface05:05
thumperoh, smack talk05:05
wallyworld_thumper: you have my pity. i wish we had used interfaces right from the word Go ha ha ha05:05
thumperheh05:06
thumperfunny man05:06
wallyworld_instead of structs everywhere :-(05:06
thumperwe'd be in a better state if we did05:06
thumperno pun intended there05:06
bigjoolswallyworld_: http://tinyurl.com/n2soctz05:06
* wallyworld_ nods sadly05:06
thumperas I think the "state" name and package is bollocks05:06
thumperit is like having a package called "db"05:06
thumperor "ui"05:06
wallyworld_lol. couldn't agree more05:07
thumperbig +1 though to the new "providers" package05:07
thumpernow if we could rename the interface05:07
wallyworld_except the name clashes05:07
wallyworld_with what providers will be in the future05:07
thumpersmall steps05:08
wallyworld_sure, but we could have chosen a better name05:08
wallyworld_only do the rename once05:08
bigjoolswallyworld_: well I was thinking of taking you out for a coffee tomorrow but if you can;t have hot liquids....05:08
wallyworld_bigjools: i'm sure i could manage a warm one05:09
bigjoolsyou could have breakfast with me and jen at the plum?05:09
wallyworld_what would doctors know anyway05:09
wallyworld_\o/05:09
bigjoolsheading there after school drop off05:09
bigjoolsbefore I head into the cbd05:09
wallyworld_most excellent :-D05:09
wallyworld_ah yes, good luck with that05:09
bigjoolswill get told how buggered I am05:10
wallyworld_hopefully no too much05:10
bigjoolshopefully05:10
bigjoolsthink I have made a connection - but ETOPIC05:10
bigjoolshey jam are you around yet?05:10
jambigjools: /wave05:10
jamwallyworld_: good to see you around, I didn't think you'd be fully coherent today.05:11
wallyworld_jam: lots to do :-)05:11
bigjoolshow would we tell the difference?05:11
wallyworld_http://tinyurl.com/n2soctz05:11
bigjoolsahahahaha05:11
bigjoolsjam: hey man, I want to catch up on the tags work you did if you can spare ten minutes?05:11
jambigjools: yeah no problem05:14
jam(just reading backlog right now)05:14
bigjoolsjam: it's more maas specific so I shall switch to #maas05:15
jamnp05:15
axwany particular reason why this isn't closed?  https://bugs.launchpad.net/juju-core/+bug/80624105:15
_mup_Bug #806241: It should be possible to deploy multiple units  to a machine (unit placement) <production> <juju:Confirmed> <juju-core:Triaged> <https://launchpad.net/bugs/806241>05:15
axwisn't this just --to?05:16
axwah sorry, it's --to & -n05:17
axw.. no it's not, the summary is just confusing05:18
=== tasdomas_afk is now known as tasdomas
arosalesbigjools, I added https://bugs.launchpad.net/juju-core/+bug/1214181/comments/405:33
_mup_Bug #1214181: Azure Provider always uploading 1.12 tools <juju-core:New> <https://launchpad.net/bugs/1214181>05:33
arosalesnot sure why that is occuring. . .05:33
arosalesnot sure if 1.12 tools are needed for some reason.05:34
arosalesbut be interesting to see if you hit that by using public-storage-*05:34
* arosales is going to put up for the night05:34
bigjoolsarosales: ok05:35
bigjoolsI'll need to punt to an expert05:35
arosalesbigjools, I had you pegged as the expert :-)05:36
bigjoolsarosales: well I know about azure, but juju not so much :)05:36
arosalesah05:36
arosalesbigjools, would you mind checking if you can reproduce what I am seeing?05:37
arosalesbigjools, I'll check the bug comments in the morning. Thanks for your help.05:40
arosalesbigjools, davecheney, wallyworld_ have a good rest of your day and thanks for the help.05:40
wallyworld_anytime, you too05:40
jamthumper: are you still around?05:43
bigjoolsarosales: sure05:46
axwdoes anyone know if there's a reason why we don't use "write_files" in our cloud-init config? rather than a script that echos the contents... improperly interpreting escape sequences05:49
bigjoolsI can suggest one but it's rude05:55
bigjoolsjam: do all the providers take mem= in MB?05:55
davecheneyaxw: probably no good reason06:02
davecheneyi think we should use cloud init whereever possible06:02
davecheneyscott moser has already solved these problem06:03
davecheneywe shouldn't reimplment his wheel06:03
bigjoolsyeah it's quite amazing the number of formats cloud-init can handle06:03
bigjoolshowever you are tied to the oldest version in old releases06:04
* davecheney sobs06:04
davecheney18 months ago everyone was like 'precise is the best release ever'06:04
davecheneynow they are like' urgh, precise, soooo oooold'06:04
davecheneyaxw: https://code.launchpad.net/~axwalk/juju-core/juju-add-machine/+merge/17984006:06
davecheneyyou have a conflict in your merge proposal06:06
axwdavecheney: I'm not surprised. I was hoping to get some comments before I actually attempted to merge. Are you attempting to test it?06:07
davecheneynah, got as far as reading the diff and saw it was dirty06:07
davecheneyjust merge it to trunk06:07
davecheneythen lbox propose agian06:07
axwyeah ok06:07
davecheneys/to/from06:08
jamaxw: write_files (IIRC) was not available in cloud-init on the versions we wanted to use06:12
jamI could be wrong06:12
jamI think I remember digging into that befgor06:12
jambefore06:12
axwjam: okey dokey. I'll confirm, and add a comment if that's the case06:13
axwthanks06:13
jambigjools: I'm not very familiar with pyjuju, but I'm pretty sure that in go-juju you specify mem directly "juju deploy --constraints=mem=512M06:13
bigjoolsoh that makes more sense06:13
davecheneyjam: correct06:27
davecheney--constraints="arch=amd64 mem=2G" etc06:28
axwjam: yeah, seems write_files was not in cloud-init 0.6.3 :(06:34
axwoh wlel06:34
jamdavecheney, axw: I would think that newer cloud-init might end up in the minor updates (12.10.2, etc). But I don't have any proof of that, we'd have to ask smoser06:36
axwjam: do we stop supporting .1 when .2 is available?06:37
davecheneyjam: i'll not hold my breath06:37
jamaxw: we look in simplestreams to find what "precise" is. I imagine it always points to the latest.06:37
jamThere is still userland and custom images, etc.06:37
axwyeah.. I guess you'd want to give people some warning about doing something like that06:38
jamaxw: the other problem is that we can't really "just detect" what a given image has available, so it is probably most reasonable to stick with least-common-denominator06:39
axwyep06:39
axwI'll add a comment in the code for the next person :)06:39
axwdavecheney: merged and reproposed06:48
bigjoolshey, does a mongo binary need to exist with the public tools even on saucy?06:49
jambigjools: we use the ppa even on precise06:50
jambigjools: we don't use it from the bucket for a while now.06:50
bigjoolsI ask because I am testing on azure and it gets stuck when trying to connect to mongo06:50
bigjoolsconn refused06:51
jambigjools: I think that is because we aren't initializing properly on the machine, and we only detect that because mongo didn't get started.06:51
jambigjools: the jujud binary writes a custom upstart06:51
davecheneybigjools: no, it does not need to exist for any series06:51
davecheneyit is redundant06:51
bigjoolsk06:51
bigjoolsdo we expect it to take a while to get started?06:52
davecheneybigjools: it comes up pretty much instantly06:52
jambigjools: not very long vs the machine being up and running. I think on HP it is < a minute06:52
bigjoolsok, something's buggered then06:52
davecheneybut we can't predict when that instant occurs06:52
jambigjools: can you ssh in to look at log files?06:52
bigjoolsI *could* if I had not already destroyed my env :)06:52
davecheneyjam: yup < 90 seconds for HP06:52
davecheneybigjools: tail -f /var/log/cloud-init-output.log // the goods06:53
bigjoolsyup06:53
bigjoolsta06:53
bigjoolswaiting for a bootstrap to finish is awkward from a UI PoV, are there any plans to improve this experience?06:54
bigjools9 minutes later, still waiting...!06:57
bigjoolsah there it is06:57
davecheneybigjools: there is a discuission on the juju-dev ML i belive07:00
bigjoolsI remember one from a while ago :)07:01
davecheneybigjools: oh, it's still ongoing07:16
rogpeppe1mornin' all07:30
bigjoolswallyworld_: so see you at the Plum any time after 08:45 then?07:31
bigjoolshello rogpeppe107:32
=== rogpeppe1 is now known as rogpeppe
rogpeppebigjools: hiya07:33
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: - | Bugs: 7 Critical, 92 High - https://bugs.launchpad.net/juju-core/
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: rogpeppe | Bugs: 7 Critical, 92 High - https://bugs.launchpad.net/juju-core/
axwdavecheney: 1121914 was released in 1.13.107:44
wallyworld_bigjools: ok, sounds good07:44
dimiternmorning07:46
rogpeppefwereade: i have just come to the conclusion that the collect logic in entityWatcher is entirely redundant - state/watcher can never deliver more than one event for the same document in the same cycle AFAICS.08:09
rogpeppefwereade: (the same is not true of lifeCycleWatcher, of course)08:11
davecheneyaxw: did we call it out in the release notes ?08:13
axwdavecheney: yes08:13
axwunder "Resolved issues"08:13
davecheneyok, removed08:13
davecheneythanks for checking08:13
axwnps08:14
davecheneynight y;all08:14
fwereaderogpeppe, my worry is that the state code doesn't get to control the watcher cycles08:16
fwereaderogpeppe, (and I'd rather keep the watcher implementations in line as far as possible... it will make life easier if we ever get as far as generating some of this code)08:16
rogpeppefwereade: the issue is that it's impossible for me to write a test that will fail08:17
rogpeppedavecheney: g'night dave08:17
fwereadegn davecheney08:17
fwereaderogpeppe, a test that *would* fail, as determined by our best attempt at analysis, is all I'm really looking for here08:18
fwereaderogpeppe, (hmm, if we *were* to force a collect on the initial event that would resolve the double-event ickiness that was causing intermittent failures in the first place... they definitely can fail ;p)08:20
rogpeppefwereade: as i do usually, i try to write a test that actually fails (at least once) when the code is wrong.08:20
fwereaderogpeppe, (but it wouldn't hit the steady state path anyway, so, meh)08:20
TheMuefwereade: ping08:21
fwereaderogpeppe, that is indeed a good thing to do in general and I applaud the approach, but I think it's also occasionally legitimate to write assumption tests that don't fail at the time of writing08:22
fwereaderogpeppe, and entityWatcher is an internal detail anyway -- it's not the only NotifyWatcher out there, and their various implementations do differ08:23
fwereadeTheMue, pong08:23
TheMuefwereade: after syncing I'm now continuing with  https://bugs.launchpad.net/juju-core/+bug/119494508:23
_mup_Bug #1194945: juju set is overloaded <juju-core:In Progress by themue> <https://launchpad.net/bugs/1194945>08:23
TheMuefwereade: the first approach following the "set --default" way showed, that this is not optimal in the sense of usability08:24
fwereadeTheMue, cool, I wasn't that keen  myself -- what's your preference now?08:25
rogpeppefwereade: the only two watchers that call collect are entityWatcher and lifecycleWatcher08:25
TheMuefwereade: it leads to mixed situations like "set foo=bar --default baz" etc08:25
rogpeppeTheMue: i don't think there's any need to allow mixing08:25
TheMuefwereade: I would like an own command for it, like "juju unset ..." or "juju reset ..."08:25
fwereadeTheMue, sweet, so would I :)08:25
TheMuerogpeppe: yeah, now allowance, but people may call it that way08:26
rogpeppeTheMue: if they do that, they'd get an error08:26
TheMuerogpeppe: dimiter and jam mentioned it too, because it is not really clear to the user08:26
rogpeppeTheMue: fair enough. i think i suggested it too originally, but it didn't really seem to justify a whole new command08:27
TheMuerogpeppe: that's what I've done so far, but it doesn't feel good. any reasons against unset or reset as own commands?08:27
fwereadeTheMue, I think I favour unset, because reset seems to be a bit more overloaded08:27
TheMuefwereade: yep, I'm feeling better with unset too08:27
rogpeppeTheMue, fwereade: i'd be ok with juju unset08:28
rogpeppeTheMue: the help text for set should mention the existence of unset08:29
TheMuefwereade, rogpeppe: ok, doing that, makes me feeling better too ;)08:29
TheMuerogpeppe: oh, yes, very good idea08:29
fwereaderogpeppe, my position is that the watchers that don't collect() probably should in general, but that Lax is appropriate for those cases in the meantime; but making every notify checker lax is not a good idea, because it means we can't write tests for alternative notifywatchers that do need to collect08:31
fwereaderogpeppe, the code was written to support being built on in predictable ways08:31
fwereaderogpeppe, ...but, look, from my perspective you made something worse and are fussing about undoing that change because it was less than perfect to begin with08:33
fwereaderogpeppe, can we please just reinstate what we had before?08:33
rogpeppefwereade: i really don't think Lax helps much08:34
rogpeppefwereade: it's smoke-and-mirrors08:34
fwereaderogpeppe, you turned *everything* into a Lax one08:35
rogpeppefwereade: oh, sorry, i forgot the terminology08:35
rogpeppefwereade: if we have some functionality that needs to be tested, we should have a test for that functionality08:35
rogpeppefwereade: rather than pretending that a few nanoseconds of scheduler decisions might give us a test failure more than once a century08:36
fwereaderogpeppe, it doesn't help *much*, I know08:37
fwereaderogpeppe, I added it because it helped a *bit*08:37
fwereaderogpeppe, and you removed it with a clear statement of "I don't understand what it does, but meh"08:37
rogpeppefwereade: i didn't understand what it did, because it didn't do what it was intended to do08:37
fwereaderogpeppe, what do you believe it was intended to do?08:37
rogpeppefwereade: i can easily add a test for aggregation08:37
rogpeppefwereade: i believe it was intended to make some tests fail if aggregation was not working08:38
rogpeppefwereade: but it's not clear which tests are intended to fail in that case08:39
fwereaderogpeppe, and your problem is that those tests do not *always* fail? yes, this is less than ideal -- but you seem to be saying things are better with that behaviour untested?08:39
rogpeppefwereade: my problem is that in all likelihood those tests will never fail08:39
rogpeppefwereade: and i'm not sure which tests they are08:39
jamaxw: do you have any idea of why cloud-init stuff was working for a while but seems to have changed?08:39
rogpeppefwereade: because the "non-lax" code was sprayed around everywhere, as if everywhere was trying to test this issue08:40
axwjam: no, but I never saw it in the working state08:40
jamaxw: k. Is there a reason why 'shquote' doesn't grow the escape '\' code?08:40
fwereaderogpeppe, it was written to test against the NotifyWatcher interface without regard for the underlying implementation08:40
rogpeppefwereade: out of interest, which tests *should* (potentially) fail if the aggregation isn't working?08:41
fwereaderogpeppe, CleanupWatcher at least IIRC -- and the point is that the tests should be somewhat useful canaries against non-obvious underlying implementation changes08:42
fwereaderogpeppe, this is how a NotifyWatcher should act regardless08:42
axwjam: not technically, but I think whomever wrote it intended it to be like http://www.daemon-systems.org/man/shquote.3.html08:42
axwor something similar08:42
fwereaderogpeppe, the fact that the implementation of one particular one is such that it can't fail is just a detail, and not a justification for using less strict testing infrastructure08:43
axwjam: IOW, I didn't want to surprise anyone08:43
rogpeppefwereade: how many tests for impossible happenstances should we write?08:43
TheMuerogpeppe: thx08:44
TheMuerogpeppe: "happenstance" is a new word for me, like it08:45
fwereaderogpeppe, quite a lot, really, because if you use inspection of the implementation as justification not to test we can drop a whole load of tests... but then they *don't fail when someone changes the implementation*08:45
rogpeppefwereade:i did think it should be possible to make a test fail by changing the code it's testing. if that's no longer the case, then i really see no bound on the amount of test code we need to write.08:45
fwereaderogpeppe, then in that case use the Lax watcher explicitly in those cases, because I cannot be bothered to argue that08:46
rogpeppefwereade: the implementation can change in so many possible ways - there are zillions of ways that things could fail if the underlying implementation is free to change completely. i really do find this a difficult issue to deal with.08:47
fwereaderogpeppe, but that is *still* using asusmptions about the implementation in tests for the interface08:47
rogpeppefwereade: yes, i believe it's ok to assume the implementation. otherwise we don't have any idea of how to test the code well.08:47
fwereaderogpeppe, the existence of Lax is a sad acknowledgement that we cannot entriely dodge the issue, but Lax is meant to be the one that's eventually dropped, not the other way round08:48
rogpeppefwereade: Lax==Sync, right?08:48
fwereaderogpeppe, yeah08:48
rogpeppefwereade: StartSync is a total hack08:48
rogpeppefwereade: it shouldn't exist at all really08:49
rogpeppefwereade: (and it didn't, originally)08:49
fwereaderogpeppe, Sync is a total hack, StartSync is the only one that reasonably approximates real conditions08:49
axwjam: now that I read the docstring for utils.ShQuote properly, tho, it seems fair to put it there. I'll update it.08:49
fwereaderogpeppe, sync only helps when you've only got one level do deal with08:49
rogpeppefwereade: StartSync gives you no guarantees of anything happening at all08:49
fwereaderogpeppe, exactly so08:50
fwereaderogpeppe, sync sometimes does and sometimes doesn't08:50
rogpeppefwereade: it at least guarantees that the watcher has received the sync message...08:51
dimiternrogpeppe: https://codereview.appspot.com/13113045 - two more uniter API calls needed I forgot about08:51
rogpeppefwereade: perhaps that's a better way to fix this08:51
fwereaderogpeppe, it guarantees that the message has travelled *some* distance through the plumbing, but not how far it has actually travelled08:51
fwereaderogpeppe, *frequently* it is good enough, but that just encourages faulty assumptions08:52
rogpeppefwereade: it doesn't guarantee, unfortunately, that the sync message has been acted on at all08:52
rogpeppefwereade: oh, Sync, yes08:52
rogpeppefwereade: how about this:08:52
fwereaderogpeppe, StartSync is better because it is reliably unreliable, as opposed to unreliably reliable08:53
axwjam: actually, sorry, last word on this ;)   utils.ShQuote single quotes its input, so '\' characters shouldn't be interpreted08:53
rogpeppefwereade: i change the watcher code so that StartSync actually works properly, and use it everywhere we can.08:53
rogpeppefwereade: in fact, rename StartSync to Sync08:54
dimiternmgz: hey08:54
fwereaderogpeppe, define "works properly", because I'm not following08:54
dimiternjam: ?08:54
rogpeppefwereade: by "works properly" i mean "when you've called (Start)Sync, you know that the underlying watcher implementation is synchronising, and won't act on random other timer events before that"08:55
fwereaderogpeppe, (also, StartSync cannot be blocked by a faulty watcher implementation, and Sync can)08:56
rogpeppefwereade: that is true08:56
dimiternfwereade, rogpeppe: sorry to interrupt, but I'd like to bug one of you for a review08:57
rogpeppedimitern: looking08:57
fwereaderogpeppe, seeking common ground: if we were to call it SyncSoon instead of StartSync, would you agree that was more accurate?08:57
dimiternrogpeppe: ta08:57
rogpeppefwereade: i'd just call it "Sync"08:57
rogpeppefwereade: and make it work08:57
rogpeppefwereade: yeah, SyncSoon is a better description of the current behaviour08:57
rogpeppefwereade: but it isn't very useful behaviour08:58
rogpeppefwereade: because it is inherently difficult to predict its results, which makes it hard to write good tests using it08:58
fwereaderogpeppe, it's hard, and inherently timing-dependent, and that sucks -- but to write useful tests by those lights, we can't depend on anything internal to state/watcher *anyway*, because we're affected by the layers of plumbing between that package and the actual test08:59
fwereaderogpeppe, Sync is useful iff there is a single layer09:00
rogpeppefwereade: yeah, i'm suggesting deleting the current Sync implementation (which simplifies state/watcher a bit too)09:00
rogpeppefwereade: and renaming StartSync to Sync, but making it more deterministic09:00
rogpeppefwereade: my objections to Sync when it was originally written were similar to yours09:01
fwereaderogpeppe, that is to me the problem -- being a bit more deterministic just encourages poor assumptions09:01
rogpeppefwereade: i think it's reasonable to assume that the underlying implementation is using state/watcher09:02
rogpeppefwereade: (that's what (Start)Sync is talking to, after all09:02
rogpeppe)09:02
jamdimitern: I'm around, but I'm not sure what you are asking for09:02
fwereaderogpeppe, I think it's impossible to make state/watcher *clients* more deterministic by tweaking state/watcher -- we're always vulnerable to the client implementations09:02
dimiternjam: sorry :) was just a ping09:03
dimiternjam: you wanted to discuss the api?09:03
fwereadedimitern, what's Resolved() for?09:03
dimiternfwereade: what do you mean?09:03
fwereadedimitern, what calls it?09:03
dimiternfwereade: let me check, just a sec09:03
rogpeppefwereade: if we assume that the clients function as a deterministic result of events received from state/watcher, i think that by tweaking state/watcher, we *can* make the clients more deterministic too.09:04
dimiternfwereade: filter.go:432:if resolved := f.unit.Resolved(); resolved != f.resolved {09:04
rogpeppedimitern: why does Unit.Resolved need to make an API call? can't we get the resolved mode with the rest of the unit params?09:05
rogpeppes/params/fields/09:05
dimiternrogpeppe: how?09:05
fwereadedimitern, forgive my brainfart09:05
rogpeppedimitern: Resolved is in unitDoc, right?09:06
dimiternrogpeppe: since LifeGetter is now common, there's no way to inject stuff that get's refreshed when you call Refresh()09:06
rogpeppedimitern: then surely your problem is that you're using LifeGetter for getting all the fields in a unit?09:06
dimiternrogpeppe: what?09:06
jamaxw: did you dig into cloud-init at all to see what it might be unescaping?09:07
dimiternrogpeppe: LifeGetter reports back the life only09:07
fwereaderogpeppe, life and resolution are essentially orthogonal -- it's an artifact of the current implementation that we get them at the same time09:07
jamaxw: for example: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-run-cmds.txt09:07
fwereaderogpeppe, this gives us the opportunity to move towards less monolithic event filtering at some point in the future09:07
rogpeppefwereade: ah, i thought it's actually quite nice that we get more than one attribute of an object when we ask for the object09:07
rogpeppefwereade: because the overhead is much smaller than making n different API calls09:08
rogpeppefwereade: but if that's no longer considered a good approach, fair enough09:08
fwereaderogpeppe, filter is a beast *because* all these different pieces of information are glommed together09:08
jamthere are no '\' characters there, but it seems to imply that cloud-init won't do anything with '\' but whether you pass a list or a string makes a difference.09:08
dimiternrogpeppe: we changed everything else, so that's a minor thing :)09:08
axwjam: I started to, but didn't get to the bottom of it. I noted that the YAML produced shouldn't be escaped. I didn't want to spend a lot of time on it, but if you prefer I can dig further.09:08
rogpeppedimitern: yeah, i'm just wondering about how many extra round-trips we'll be having09:09
rogpeppedimitern: but since we aren't measuring anything, who knows?09:09
fwereaderogpeppe, I'd like it most of all if we only even bothered to check/watch resolved status when the uniter knew it had to09:09
fwereaderogpeppe, we can't get there in just one step though09:09
dimiternrogpeppe: once we do it, we can measure easily09:10
jamaxw: I mostly want us to understand what is going on rather than just saying "oh, add another \" because that may not be appropriate always, as it might get interpreted as literal "\" + "n"09:10
jamwhich would then fail for a different reason, etc.09:10
dimiternrogpeppe: and optimize as needed09:10
axwjam: yep fair enough. I'll keep digging09:10
jammgz: poke about maas constraints and py-juju09:10
rogpeppedimitern: optimizing will be difficult if we can't change the API (which is the whole rationale for the bulk call thing AFAIR)09:10
jamaxw: for example we also write agent.conf via this mechanism, and some other files, it may be that we should always escape, or never, or write it differently, etc.09:11
rogpeppedimitern: but i agree anyway09:11
jamaxw: what I'm really concerned with is if in precise we need to do it one way, and in saucy it will be different.09:11
dimiternrogpeppe: we're all smart guys, I'm sure we'll find a way when comes to that :)09:11
jamaxw: I think runcmds goes through "shellify" which will escape things if passed in as a list, but *not* escape them if passed in as a string09:11
jamI don't know for sure what cloud-init path we are doing atm09:12
jamaxw: though if I'm reading it correctly, AddFile uses AddScripts which uses RunCmd09:13
rogpeppeaxw: i think i was probably responsible for shquote09:13
rogpeppeaxw: what's the issue?09:13
axwjam: understood. it's a string and not a list, so it will go through the shell09:13
jamaxw: and it looks like we are doing a single string for the whole script to run09:13
axwrogpeppe: nothing specific to shquote. the rsyslog conf written out has its \n interpreted09:13
rogpeppeaxw: ah, so you can't have '\n'09:14
dimiternaxw, jam, rogpeppe: AddFile and AddScripts were recently changed by sidnei IIRC09:14
jamaxw: and we *could* pass it in as "args" rather than "literal" which might be a whole lot more obvious through all the transformation steps09:14
axwjam: sorry I don't follow that last statement09:15
jamaxw: going back to this example: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-run-cmds.txt09:15
jamcloud-config takes a yaml description09:15
jamin that09:15
jamyou can have a list of things to run09:15
jameach item in that list09:15
jamcan either be a literal string09:15
jam"ls -l /root"09:15
jamor a list09:15
jam"[ ls, -l, root ]"09:15
jamif you pass in a literal string09:16
jamthen cloud-init just passes it directly to sh to interpret09:16
jamif you pass in a list of strings09:16
jamcloud-init will escape them for you before passing it to sh09:16
jamaxw: it *looks* like our internal code has a "command" object in cloudinit.go09:16
jamwhich can take either a "literal string" or a "args []string"09:16
jambut all of our AddFile code uses the "literal string" approach.09:16
jamAnd I'm wondering if instead of playing games with escaping on our side09:17
jamwe might just go with09:17
jamthe args []string form09:17
axwjam: indeed, I missed that. I'll look into using that09:17
axwthanks09:18
jamaxw, dimitern, sidnei: it looks like we switched from doing "echo content > filename; chmod MODE filename" to "install -m MODE /dev/null filename; echo CONTENT > filename" any idea why?09:20
jam(I don't think it changes the failure mode here, but I'm trying to understand the logic behind the change)09:20
fwereaderogpeppe, anyway: *WatcherC targets the ideal *Watcher behaviour; some of them suck, and need Lax*WatcherCs, but those should be dropped one day; thank you for tracking down the case where non-lax was not good enough, but please reinstate the stricter versions and use them where they don't cause failures09:21
axwno idea09:21
jamthe commit comment just says "move addFile from environments/cloudinit.go to cloudinit/cloudinit.go"09:21
rogpeppefwereade: how are you with my plan to do away with Sync entirely? (and fix StartSync so it works reliably for us - i've nearly done it, it's not a hard change)09:22
jamI guess the overall commit says "create the file with the correct mode first, so that we slightly increase security". Which is fine, as long as that doesn't actually break this :)09:22
dimiternjam: https://codereview.appspot.com/12352043/ - that's the change09:23
axwjam: using a list isn't really viable here, since it wants to redirect output09:23
fwereaderogpeppe, I still consider it philosophically suspect -- I don't see how guaranteeing that a sync has actually started helps anyone, because what we really care about is that *all* consequences have completed, and we can't affect that at the state/watcher level09:24
rogpeppefwereade: the issue it solves is this:09:24
rogpeppefwereade: a test makes some changes, then wants to see what happens09:24
rogpeppefwereade: it calls StartSync, but the regular timer event happens first, at the wrong time09:25
rogpeppefwereade: if we make StartSync actually guarantee that the watcher is doing a sync *right now*09:26
dimiternrogpeppe: how's the review looking? :)09:26
rogpeppefwereade: then we avoid that possibility09:26
rogpeppedimitern: apologies, getting back on it09:27
fwereaderogpeppe, keep explaining, I'm not seeing how that situation causes problems09:27
fwereadedimitern, can't we determine service name unambiguously from knowing the unit name?09:27
dimiternfwereade: hmm09:27
dimiternfwereade: yes, actually you're right, but we can't be sure we can get it from state perhaps?09:28
fwereadedimitern, the Unit surely knows its own name, though09:28
dimiternfwereade: yes it does09:29
dimiternfwereade: but unit.Service() -> *Service, error09:29
dimiternfwereade: should we always return nil there in the api?09:29
fwereadedimitern, I'm confused, the method I'm looking at is called ServiceName09:30
rogpeppefwereade: there's a good example of the problem in https://codereview.appspot.com/12352044/09:30
dimiternfwereade: i mean, for the error, otherwise we construct and return a proxy uniter.Service object ofc09:30
dimiternfwereade: ah, that one - yeah, you're right there - I should just extract the service name from the unit tag and return it09:30
dimiternfwereade: it's a good candidate for such a call in names09:32
fwereadedimitern, +109:32
rogpeppefwereade: another possibliity, i suppose, would be to raise the watcher period to forever09:32
rogpeppefwereade: in tests09:32
dimiternfwereade: but we still need GetService at the server-side, right?09:32
rogpeppefwereade: so events we get are exactly and always as a result of StartSync calls09:33
fwereadedimitern, yeah, think so09:33
dimiternfwereade: for the other call: func (u *Unit) Service() (*Service, error)09:33
dimiternfwereade: ok09:33
jamaxw: so looking at cloud-init-output.log I see the echo line, but I don't see *any* single quotes09:33
fwereaderogpeppe, that is an interesting thought09:33
jamare they getting interpreted in the yaml maybe?09:33
fwereaderogpeppe, because just guaranteeing that a sync is in progress doesn't say anything about whether or not the last event landed before the sync started09:34
rogpeppefwereade: (if we have any tests that are actually relying on the regular polling, they're broken anyway)09:34
axwjam: just recreating my env, I destroyed it prematurely. sounds feasible though09:34
fwereaderogpeppe, agreed09:34
jamaxw: I'm trying to figure out if cloud-init writes out the script it is going to be running somewhere to more easily inspect iti09:34
jamor I can try querying the metadata server09:35
axwjam: I think it's in /var/lib/cloud/seed/instances or something like that09:35
fwereaderogpeppe, but I'm pretty comfortable as it is with the idea that an initial event may reflect the past, not so much because it's nice behaviour but because it's an insidious sort of assumption that it's hard to spot09:36
fwereaderogpeppe, and I'm concerned about the impact of changing it09:36
fwereaderogpeppe, because I think *that* will also be subtle09:37
rogpeppefwereade: i don't *think* i was suggesting that, was i?09:37
rogpeppefwereade: (it's a pretty fundamental part of the watcher implementation)09:37
jamaxw: /var/lib/cloud/instance/user-data.txt.i09:37
axwjam: there's a runcmd file too09:38
axwjam: I just munted my cloudinit output tho, so need to run it again...09:39
rogpeppefwereade: the random interference of timer events is actually perhaps the underlying problem here, not the StartSync behaviour. still thinking it over.09:39
fwereaderogpeppe, yeah, I'm a bit uncertain myself09:40
fwereaderogpeppe, might go eat something and think it over a bit09:40
rogpeppefwereade: okeydokey09:40
jamaxw: so if I look at instance/scripts/runcmd it uses #!/bin/sh and calls "echo '....'" and that has a %\n" in it.09:41
jamhowever my /etc/rsyslog.d/25-juju.conf doesn't have the \nd09:42
jamI'm noticing we *aren't* using "echo -e" soecho shouldn't be interpreting it.09:42
jambut it might be09:42
jamor #!/bin/sh might be09:42
jamaxw: ugh09:45
jamseems that dash and bash interpret 'echo' differently09:45
axwjam: doh09:46
dimiternrogpeppe: cheers for the review09:46
jamaxw: do 'man dash' and then /echo09:46
jamit always interprets '\'09:46
jamwhile *bash* echo09:46
jamonly interprets '\' characters if you specify "echo -e"09:46
rogpeppejam: there are many different versions of echo09:46
dimiternrogpeppe: but the comment should've been about ServiceName(), not Service(), right?09:47
jam(and you can force echo -E to disable it)09:47
rogpeppejam: even bash has several built in09:47
jamdash's echo -E just prints "-E"09:47
mgzjam: hey09:47
rogpeppejam: that's the original echo (it just interprets -n)09:47
jamaxw: so... if we run on a machine where "/bin/sh" is dash, then we need to escape, if we run on a machine with "bash" then we *shouldn't* escape09:47
jamaxw: fun times :)09:47
axwjam: how about I just pipe it through base64 ;)09:47
jamaxw: hence my "lets understand what is going on"09:47
axwjam: good call.09:48
rogpeppejam: one possibility is to call /bin/echo09:48
rogpeppejam: but that can vary too, of course, sysv vs bsd etc09:48
jamaxw: /bin/echo seems to be sane here09:49
axware we ever going to support anything that uses non-GNU?09:49
* rogpeppe has no idea what portability range we're aiming for09:49
jamaxw: I know we have plans to support centos, but I don't know of any immediate plans for BSD (also maybe windows, but that will be entirely different anyway)09:49
axwCentOS will be fine. We can always change it later anyway09:50
axw/bin/echo it is09:50
jamaxw: I can confirm that write_files is not available on the precise image I'm looking at after bootstrapping.09:52
axwjam: thanks. I also checked in the cloud-init source from 12.04, and confirmed it didn't have the module09:53
* rogpeppe can't work out how to reliably print a string with /bin/echo either09:53
axwrogpeppe: ?09:53
axwyou mean different implementations?09:54
rogpeppeaxw: what if the string is "-E"?09:54
rogpeppeaxw: no, i mean with gnu echo09:54
jamrogpeppe: -- ?09:54
rogpeppejam: nope, doesn't work09:54
axwheheh09:54
rogpeppefrickin' magnificent09:54
jamrogpeppe: so it doesn't matter in this case, but yes, trying to echo things starting with "-" is not very well supported09:55
rogpeppeit's ok if we don't mind an extra space at the start09:55
rogpeppe/bin/echo '' $x09:55
rogpeppe/bin/echo '' $x | sed 's/ //'09:56
rogpeppe:-)09:56
jamrogpeppe: in our particular case we actually start the string with a blank newline, so it really doesn't matter :)(09:56
axw| cut -c2-09:56
axwbut I'm not going to go there09:56
rogpeppeoh G N U we love U09:56
jamaxw: my current vote is for /bin/echo for AddFile09:57
axwjam: yep that's what I'll do09:57
axwwith some comments...09:57
rogpeppeaxw: printf "%s\n" $var ?09:58
rogpeppeaxw: avoids the whole echo minefield09:59
axwoh, I suppose so :)10:00
axwprintf "%s\n" '-E'10:02
axwyay10:02
jamrogpeppe: "printf is a shell builtin" are we sure dash and bash won't interpret it differently?10:09
jamaxw: http://pubs.opengroup.org/onlinepubs/009696799/utilities/echo.html10:10
jamseems to state "echo is not portable across POSIX so use printf"10:10
jamso lets use printf :)10:10
axwjust updating tests now10:11
rogpeppedimitern: my comment *was* actually about GetService10:13
rogpeppedimitern: it just returns the service name currently10:13
dimiternrogpeppe: it returns the service tag, and that'll be used in Service()10:14
rogpeppedimitern: but we know the service tag given the unit name10:14
dimiternrogpeppe: but we need to get it from state, right?10:14
rogpeppedimitern: why?10:14
dimiternrogpeppe: it might be missing or something10:15
dimiternrogpeppe: why is there an error return on state.Unit.Service() otherwise?10:15
rogpeppedimitern: can a service be deleted when it's still got units?10:15
rogpeppedimitern: because state.Unit.Service actually gets other info too10:15
rogpeppedimitern: if we're planning to get that other info as part of the GetService call, then fine10:16
dimiternrogpeppe: i stil think there should be a server-side GetService call10:16
rogpeppedimitern: but it doesn't look like it10:16
rogpeppedimitern: what value does it add?10:16
dimiternrogpeppe: I could change it to get the life as well10:17
dimiternrogpeppe: but we already have Life that works on services10:17
rogpeppedimitern: so is the plan that we don't hold any state locally at all for an entity?10:18
rogpeppedimitern: so we won't have a Refresh call10:18
axwjam: I'll finish this later, going to get my daughter ready for bed10:18
dimiternrogpeppe: I'll propose a follow-up that adds the mentioned ServiceFromUnitTag to names, and can remove GetService then10:18
dimiternrogpeppe: no, we hold the life10:19
rogpeppedimitern: ah, in which case GetService needs to get the life too, right?10:20
dimiternrogpeppe: no, Life already works on service tags10:20
rogpeppedimitern: oh, of course.10:20
dimiternrogpeppe: so perhaps uniter.Unit.Service() should really do a Life call underneath, like uniter.Unit does10:21
rogpeppedimitern: yup, that seems about right.10:21
rogpeppedimitern: it should probably just call Service.Refresh to avoid making assumptions about the fields in a service10:22
dimiternrogpeppe: who should?10:22
dimiternrogpeppe: Life already gets the service afresh10:23
rogpeppedimitern: the client side uniter.Unit.Service should create a local instance of Service, then call Refresh on it, rather than making the Life call itself.10:23
rogpeppedimitern: the Service.Refresh call will make the Life call itself10:24
dimiternrogpeppe: that works as well, yes10:24
rogpeppedimitern: that way if we add more fields to Service, we just need to change that one place (Refresh)10:24
dimiternrogpeppe: yep10:29
dimiternrogpeppe: there it is https://codereview.appspot.com/13112045/10:55
* TheMue => lunchtime10:56
dimiternrogpeppe: included that what talked about above11:02
rogpeppedimitern: i'm halfway through :-)11:02
dimiternrogpeppe: cool11:02
fwereadedimitern, couple of comments11:03
dimiternfwereade: ta11:03
dimiternfwereade: not sure what you mean by the second comment - are you suggesting to drop .ServiceName() ?11:05
fwereadedimitern, if it's not used elsewhere, having a names method for converting unit tag -> service tag seems most helpful here11:05
dimiternfwereade: if what's not used elsewhere?11:06
rogpeppedimitern: reviewed11:07
rogpeppefwereade: i've just suggested a slightly different approach11:07
dimiternrogpeppe: thanks!11:07
rogpeppefwereade: keep the conversion on names only.11:07
fwereaderogpeppe, sgtm11:07
rogpeppefwereade: that way it's only the name-related functions that panic on unexpected things11:07
dimiternrogpeppe: sgtm as well11:08
fwereaderogpeppe, yeah, definitely better, thanks11:08
rogpeppefwereade: cool, ta11:08
jamespagemgz or hazmat: any chance either of you can take a look at these test failures11:22
jamespagehttp://paste.ubuntu.com/6006051/11:22
jamespagejuju-0.7 on saucy - going to re-introduce so as to not kill folks with existing py-juju deployments11:23
dimiternrogpeppe: quick glance before I submit it? https://codereview.appspot.com/13112045/11:24
rogpeppedimitern: glancing11:24
mgzjamespage: I think I filed some bugs for test failures that I got only when run as part of the package build process, not outside it11:24
mgzhm, not those failures though11:25
hazmatjamespage, sure11:25
jamespagehazmat, thanks11:25
natefinchman, testing the azure environ is... less than fun11:26
rogpeppedimitern: one thought: we should implement Unit.String11:27
dimiternrogpeppe: it's easy enough to do if we need it11:27
rogpeppedimitern: (lots of things call printf %q with units and services11:27
dimiternrogpeppe: it also applies to the other proxy objects, like service, etc.11:28
dimiternrogpeppe: only for units and services?11:28
rogpeppedimitern: any types which have String defined on them in state, i guess11:29
dimiternrogpeppe: and they should return names, not tags11:29
rogpeppedimitern: i think so11:29
rogpeppedimitern: reviewed11:30
dimiternrogpeppe: will add them next then, along with other stuff11:30
dimiternrogpeppe: ta11:30
jamdimitern, rogpeppe: https://plus.google.com/hangouts/_/f497381ca4d154890227b3b35a85a985b894b471 standup?11:33
jamespagehazmat, thanks for the licensing update for jujuclient btw - I just uploaded that and juju-deployer + websocket-client to saucy11:36
hazmatjamespage, awesome, thanks11:37
dimiternanybody seen this debug hooks test failure? http://paste.ubuntu.com/6006151/11:57
jamdimitern: I haven't seen it before, though it does look like one of those "it should take 100ms but real-world variability is higher than that". At the least, we need a better message because that is *really* hard to read.12:01
dimiternjam: +112:02
hazmatlooks like its got a skew of 1s12:02
jamwallyworld_: so the bot seems to say "parse-tools-simplestreams" has already been merged12:08
jamwhich is why it isn't merging it again, I'll poke a bit more.12:08
wallyworld_jam: i got a singlr LGTM, so i laned it today12:09
jamwallyworld_: k, looks like LP didn't notice, and I'm not sure why that is.12:10
dimiternrogpeppe: the String()s are here https://codereview.appspot.com/1307904412:26
wallyworld_jam: sorry, just got off the call, is lp sorted out with the merge?12:37
smoserjam, axw there is no plan to put write_files into 12.04. features don't really qualify for SRU, but we could make that happen if it was terribly useful.12:47
axwsmoser: IMHO it's not worth the trouble - it's not a big deal to continue doing what we're doing.12:48
axw(not worth the trouble to update 12.04's cloud-init)12:49
smoserwhat is it that you'ore doing ?12:49
axwjust writing out rsyslog config files12:49
axwthere was a bug in our script that was writing the files; they were using echo, which happened to be the shell's builtin12:50
axwand it was interpreting \n12:50
axwthat shell was dash, evidently. whereas bash's builtin, and /bin/echo don't interpret by default12:50
dimiternrogpeppe: ping12:55
jamwallyworld_: I think I sorted LP out.13:04
wallyworld_great13:04
gemeI'm moving over to juju-core - are there any docs / blogs that explain the key changes ?13:05
TheMuegeme: the repository contains a doc dir with some very helpful files13:06
andrew_w_deanerogpeppe: hi. I'm testing against the api using a local provider and I'm getting "unknown provider type local" errors. I was under the impression that the local provider was supported now.13:08
gemeTheMue: Thanks13:08
natefinchallenap: got a minute for a juju - azure environ question?13:09
rogpeppeandrew_w_deane, dimitern: sorry, was at lunch13:17
allenapnatefinch: Sure, go for it.13:17
andrew_w_deanerogpeppe: no worries.13:17
rogpeppeandrew_w_deane: what are you trying to do, exactly?13:17
rogpeppedimitern: pong13:17
natefinchallenap: I'm trying to test a new method I added to the azure environ to return the addresses (hostname and ips) of instances13:18
natefinchallenap: testing the azure stuff seems to use  gwacl.PatchManagementAPIResponses  to mock out the responses from the server, but I'm having difficulty getting it to do exactly what I want13:19
allenapnatefinch: Yeah, it's rather involved.13:20
natefinchallenap: let me whip up a pastebin so you can see what I'm doing13:20
allenapnatefinch: Cool.13:21
dimiternrogpeppe: did you have a chance to look at it?13:21
rogpeppedimitern: only briefly, will do soon13:21
dimiternrogpeppe: ok, thanks13:21
* dimitern bbiab13:22
natefinchallenap: http://pastebin.ubuntu.com/6006401/13:23
natefinchallenap: the problem is that the test isn't getting the IP address returned from the call to Addresses.... even though I'm creating the deployment with the RoleInstanceList with a RoleInstance that has the IP Address set13:25
natefinchallenap: sorry, that Addresses function was half modified... here's the one that actually compiles ;)   http://pastebin.ubuntu.com/6006409/13:28
smoseraxw, i dont think cloud-init was getting in your way, was it ?13:29
smoserthe message https://code.launchpad.net/~axwalk/juju-core/lp1212148-cloudinit-escapes/+merge/180988 implies cloud-init13:30
allenapnatefinch: I'm going to patch this in and fire it up.13:30
rogpeppesmoser: will cloudinit interpret backslashes inside single quotes ?13:30
natefinchallenap: I haven't actually tested the code against a real azure environment... that was going to be my next question :)13:30
natefinchallenap: and thanks :)13:31
smoserwell, cloud-init reads yaml (through python-yaml).13:31
rogpeppesmoser: hmm, sorry, yes, was thinking of upstart13:31
smoserafaik, cloud-init correctly escapes commands in 'runcmd' or bootcmd when it writes the run script.13:31
smoserrogpeppe, whenever i have to deal with fighting multiple levels of escaping (which i dont think cloud-init is actually giving you pain itself here), i often just base64 encode stuff.13:32
rogpeppesmoser: yeah, that's always an option13:33
smoserecho BASE64_ENCODED_STUFF | base64 --decode > output13:33
rogpeppesmoser: assuming the base64 command is provided by default13:33
axwsmoser: yeah I mistook it to be the culprit, it's not a cloud-init problem at all13:33
smoser$ dpkg -S `which base64`13:34
smosercoreutils: /usr/bin/base6413:34
axwsorry, didn't mean to blemish your good name :)13:34
smoser$ apt-cache show coreutils | grep Ess13:34
smoserEssential: yes13:34
smoserwhat that means is your'e extremely unlikely to find a debian or ubuntu system without coreutils.13:34
rogpeppesmoser: ok, that's good to know, thanks13:34
smoserand iirc busybox even provides 'base64'13:34
axwI did consider base64'ing it13:35
axwno good reason not to I suppose, just didn't want to obfuscate the data and make debugging *slightly* more onerous13:35
rogpeppesmoser: i think the problem was actually in the echo command13:35
rogpeppesmoser: which is, as we know of old, problematic13:36
rogpeppesmoser: so we're going to use printf(1)13:36
smoseryeah, i was going to suggest that too.13:36
smoserprintf "%s" "whatever-wont-get-interpreted-here"13:37
rogpeppesmoser: sad really, but yes, that's the preferred solution13:37
rogpeppesmoser: let's blame SysV13:38
smoserit not sad.13:38
natefinchallenap: I actually have to head out for probably an hour to an hour and a half.  If you need to do something more pressing, please feel free, but any help would be much appreciated.  I'm sure I'm just doing something wrong in setting up the test (whether or not the actual function is correct is another matter).13:38
rogpeppesmoser: it's a bit sad that echo is not fit for purpose. "you only had one job" and all that.13:38
allenapnatefinch: I'll look at in the meantime. It's good actually; you're not blocked while you're out :)13:39
natefinchallenap: thanks, I appreciate it :)13:39
=== natefinch is now known as natefinch-afk
smoserrogpeppe, fair enough :)13:39
=== robbiew1 is now known as robbiew
smoserbut printf just has one job too.13:39
smoserand it does what you want.13:40
axwlong live the mp, the mp is dead13:41
allenapnatefinch-afk: http://paste.ubuntu.com/6006470/ is my solution. Addresses() calls GetDeployment(), which only expects a single deployment to be returned.13:48
allenapnatefinch-afk: Fwiw, where you do `if len(d.RoleInstanceList) > 0 {` I suggest you instead loop. There *should* only be one instance, but my spidey senses think it'll be easier to debug if someone does start another instance in the deployment (by hand, for example). I'm not sure that Azure promises ordering of instance lists.13:51
mgzyeah, I agree with that13:53
jamespagemgz, fwereade: hey - I just uploaded golang 1.1.2 to saucy - I'm guessing that you guys want that in the Juju golang PPA right?14:04
mgzjamespage: that would be ace14:05
fwereadejamespage, yes please :)14:05
jamespagemgz, also fixes up some problems with os/user14:05
jamespagefwereade, mgz: OK _ as soon as it lands in the release pocket I'll do the backports14:06
mrammmgz: jamespage: what do you guys think about a short network addressability sprint in your hometown?14:06
fwereadedimitern, hey, mea culpa for lazy reviewing, but an error on .Count() does not mean "ehh pretend not in scope" it means "something went wrong, report it"14:06
fwereadedimitern, (in InScope)14:06
jamespagemramm, depends when14:06
mrammjamespage: I know you have a "life event" coming14:07
mrammjamespage: but I don't remember when (sorry)14:07
mgzthese events have a habit of not turning up quite when expected as well :)14:08
mgzsounds like a neat idea if james can fit it in though14:09
mrammmgz: jamespage: yea, and I think it would be worth doing even if james can't come14:12
jamespagemramm, probably this week :-)14:13
mrammjamespage: cool.   How much family leave time are you thinking you will take?14:13
jamespagemramm, 2 weeks14:14
mrammcool14:14
mrammso I was thinking about mid september for a network addressability sprint14:14
mrammwell, actually jam suggested it14:14
dimiternfwereade: hmm, then how come all tests pass? :)14:28
fwereadedimitern, because you implemented it to ignore errors14:29
dimiternfwereade: :)14:29
fwereadedimitern, we aren't encountering errors under test conditions14:30
dimiternfwereade: ok, I see your point, will change it to return an error and a bool14:30
fwereadedimitern, thanks :)14:31
=== tasdomas is now known as tasdomas_afk
mrammnatefinch-afk: ping me when you get back RE: windows client14:39
* axw snores14:43
axwnight folks14:43
=== BradCrittenden is now known as bac
=== natefinch-afk is now known as natefinch
natefinchallenap, mramm: I'm back14:56
allenapnatefinch: I replied soon after you afk'ed. Ping again if you didn't get it.14:57
natefinchallenap: saw it.14:58
allenapCool.14:58
natefinchallenap: not sure how we'd handle that... should we return all addresses of all instances?14:58
natefinchallenap: if not, how do we detect which one is "ours"?14:59
mgznatefinch: in practice, there should only be one instance15:00
mgzbut we at least want some handling of len()!=1 in case something is weird15:01
natefinchmgz: right now I handle zero and > 0  :)  for > 0 I just pick the first one.   If there's a better way to handle 2+, I'm all for it15:01
allenapnatefinch: What mgz said. I think it would be better to return all the addresses instead of the first of an unordered list. Maybe better yet would be to return an error.15:02
natefinchallenap: an error sounds good to me. Indicates someone mucked with our environment15:03
fwereaderogpeppe, jam, I'd appreciate your thoughts on https://codereview.appspot.com/12841044 -- particularly the question in the description15:03
allenapnatefinch: Yeah, good point, I agree.15:03
fwereaderogpeppe, jam: it's a step towards https://bugs.launchpad.net/juju-core/+bug/119243315:03
_mup_Bug #1192433: relation-list reporting dying units <jujud> <relations> <juju-core:Triaged by fwereade> <https://launchpad.net/bugs/1192433>15:03
natefinchallenap, mgz: sorry, brb, gotta help with emergency diaper duty15:04
allenapnatefinch: Heh, I know what that's like. Go! :)15:04
fwereaderogpeppe, jam: I'm not quite sure whether this is a good point at which to introduce a provider/requirer asymetry -- can you think of potential issues there?15:05
rogpeppefwereade: that was the first thought that crossed my mind too15:05
fwereaderogpeppe, my heart is saying you should PrepareLeaveScope if you're a peer or a provider, but not if you're a requirer15:07
rogpeppefwereade: i'm not sure15:07
rogpeppefwereade: but it's a difficult call15:07
natefinchmgz, allenap: back, crisis averted ;)15:10
fwereaderogpeppe, my problem is that I'm not seeing a great deal of difficulty in the call, but I feel like I might be missing something ;)15:10
rogpeppefwereade: i can't think of any argument that applies to providers that doesn't apply to requirers too15:10
rogpeppefwereade: a requirer could very easily set up creds for a provider, for example15:11
fwereaderogpeppe, it *could* but that would be very annoying ;p15:11
fwereaderogpeppe, I guess that's the problem with having left it wide open thus far15:11
rogpeppefwereade: how do you mean?15:11
rogpeppefwereade: annoying for us as developers?15:11
fwereaderogpeppe, annoying for anyone who takes the meanings of provide/require at face value really15:12
rogpeppefwereade: i do find the provider-requirer thing a bit odd (you can have a requirer with a relation to several providers, right?)15:12
rogpeppefwereade: but i find it particularly difficult trying to second guess how people might be using our primitives15:13
rogpeppefwereade: maybe we should do this for peer relations only15:15
rogpeppefwereade: the other question is: why does querying a dying unit fail?15:16
rogpeppefwereade: if we made that work, would your CL be necessary?15:16
fwereaderogpeppe, I read that to mean "the dying unit is busy shutting down replication, and an app-level attempt to connect fails"15:18
rogpeppefwereade: ah, i see, yes15:18
fwereaderogpeppe, so I don't think that's something we can affect15:18
rogpeppefwereade: another possiblity would be to add a flag to relation-list to allow it to ask for only alive units15:19
rogpeppefwereade: that way a charm could exclude dying units at its own discretion15:20
rogpeppefwereade: and we wouldn't need to break symmetry15:20
fwereaderogpeppe, I think that'd be somewhat challenging15:20
fwereaderogpeppe, from the charm perspective, it introduces a whole new concept, and I'd need to think pretty hard before I was sure that didn't introduce its own sources of confusion15:21
rogpeppefwereade: i think i'd be inclined to make this happen for peer relations only, addressing the specific use case the bug talks about15:27
rogpeppefwereade: you're assuming that requirer-provider implies client-server, but that's by no means the only possibility15:28
fwereaderogpeppe, I *think* that all I'm assuming is that it means requirer-provider, but I'm probably being dense; expand?15:29
rogpeppefwereade: well, AFAICS, the situation you're concerned about it this:15:31
mgzallenap: any ideas on bug 1214451?15:32
_mup_Bug #1214451: Unable to bootstrap maas environment <juju-core:New> <https://launchpad.net/bugs/1214451>15:32
rogpeppefwereade: provider sees requirer go away, so shuts off access, but requirer is still shutting down and tries to access the provider to flush any final information,  but it can't15:32
rogpeppefwereade: because access is shut off15:32
rogpeppefwereade: is that an accurate representation?15:32
fwereaderogpeppe, yeah15:32
rogpeppefwereade: so, i don't really see how that logic doesn't work the other way around too (with "requirer" swapped with "provider")15:33
fwereaderogpeppe, well, it does indeed all hinge on the naming, that a "provider" is expected to "provide" something that a "requirer" "requires" ;)15:33
dimiternfwereade: quick review? https://codereview.appspot.com/1278204515:34
fwereaderogpeppe, and that it's not a problem if something not required is not present, but the converse does not hold15:34
allenapmgz: My guess is that it's trying to create an empty provider-state, but for some reason MAAS forbids an empty file. Sigh. It's probably our fault, but you never know because Django gets flustered at the difference between "" and None. I'll investigate...15:35
rogpeppefwereade: i think that's perhaps a step too far - it's that a resource that we knew about is going away, and we want to be able to shut down in an orderly manner15:35
rogpeppefwereade: and that seems to apply for both requirers and providers15:36
mgzallenap: a django version difference also sounds like a good bet15:36
mgznatefinch: you have a moment for a quick catchup in a sec? we can do kanbany things while we're at it15:38
fwereaderogpeppe, it still STM that providing something not required is fine; but requiring something not provided is a problem15:38
fwereaderogpeppe, this discussion is I guess orthogonal to the practical question of "how many relations are implemented backwards"15:38
fwereadedimitern, LGTM btw15:39
rogpeppefwereade: ... or "how many charms shut off all access on relation-departed?"15:39
dimiternfwereade: ta15:39
fwereaderogpeppe, well, those that do will I think be helped by this change, so long as they're defined the natural way round; those that don't won't care, AFAICS15:40
natefinchmgz: sure15:40
rogpeppefwereade: i was thinking of applying this change across the board15:41
mgznatefinch: same hangout as this morning?15:41
rogpeppefwereade: and wondering how many things would break15:42
natefinchsure15:42
fwereaderogpeppe, to consciously and deliberately remove a requirer's, er, requirements, seems a little perverse though15:43
fwereaderogpeppe, even if nothing broke it'd cause the relationship to make less sense15:43
rogpeppefwereade: well, the requirements are going away in a moment anyway15:43
allenapmgz: I want Django to die in a lake of burning acid, but that doesn't help here. Raphaël landed a fix in r1540 to allow empty files. jamespage, can you use ppa:maas-maintainers/dailybuilds instead?15:43
jamespageallenap, OK - let me try15:44
fwereaderogpeppe, yeah, the other perspective is "this is the same as an independent breakage of some unit, and we should WONTFIX it, because charms have to handle that anyway"15:44
rogpeppefwereade: the question is whether it's important that we get told that they've gone away *after* they've entirely shut down, or before they start to shut down15:44
fwereaderogpeppe, but I think it's unhelpful to enforce a model that obscures that (IMO) real distinction15:45
rogpeppefwereade: the difficulty is that there's no right answer15:46
rogpeppefwereade: both answers to the question "what are the active units in this relation" are equally valid15:47
mgzhm, nate still isn't in the list of users that I canassign cards to on the kanban board?15:47
mgzfwereade: I don't suppose you have the right kanban access to fix that?15:48
fwereademgz, I don't *think* so but I'll try to poke around15:48
fwereademgz, if I can I can't figure out how15:51
fwereademgz, *but* he seems to be on the list -- just pushed off the bottom of the quick-access submenu15:51
mgzthat worked, thanks!15:51
fwereaderogpeppe, well, we do already define the reality we present to the charm, and what we show doesn't necessarily bear much relation to what's really going on15:52
fwereaderogpeppe, units can be available for a long time before a related unit knows about them, for example15:52
rogpeppefwereade: yeah.15:55
rogpeppefwereade: i think it's worth a post to the mailing list15:57
rogpeppefwereade: interestingly my changes to the watcher logic exposed a bug in cleanupWatcher16:07
rogpeppefwereade: it didn't coalesce events16:07
fwereaderogpeppe, ha :) nicely done then16:08
fwereaderogpeppe, ah, hmm, looking at that code I have something knocking at the back of my mind16:09
rogpeppefwereade: the watcher code?16:09
fwereaderogpeppe, the cleanup watcher16:09
rogpeppefwereade: oh yes?16:10
fwereaderogpeppe, all I can remember is that it was implemented as simply as possible for some specific reason16:11
rogpeppefwereade: hmm16:11
rogpeppefwereade: perhaps we don't really care if we receive two cleanup events16:11
fwereaderogpeppe, that's basically it -- a Cleanup() with nothing to clean up is cheap enough that we don't really care16:11
fwereaderogpeppe, but, that said, there's no reason *not* to coalesce16:12
rogpeppefwereade: hmm, i could just fix the test16:12
fwereaderogpeppe, I'm pretty sure it predated collect anyway, and the introduction of collect was not done across the board16:12
rogpeppefwereade: (although it was no hassle to make it call collect)16:12
fwereaderogpeppe, I think a collect is a good idea tbh16:12
fwereaderogpeppe, ah wait16:13
rogpeppefwereade: (the only awkward bit was that collect assumed string keys, but that wasn't a problem to change)16:13
fwereaderogpeppe, let's go with it16:13
=== teknico1 is now known as teknico
=== TheRealMue is now known as TheMue
TheMueAh, found my silly mistake. *phew*16:30
natefinchrogpeppe: easy one, if you have time - https://codereview.appspot.com/1312604316:41
natefinchmgz: back16:41
rogpeppenatefinch: looking16:42
mgznatefinch, rogpeppe: also an easy one https://codereview.appspot.com/1274104816:44
natefinchrogpeppe: checker was needed for my next couple changesets16:44
rogpeppenatefinch: isn't it O(n³) in fact ?16:44
rogpeppenatefinch: (not that it matters much, probably)16:45
natefinchrogpeppe: yeah with the reslicing, yeah, you're right16:46
natefinchrogpeppe: moral of the story, keep n under 10 and you won't notice16:46
rogpeppenatefinch: i think you can keep the amount of code about the same and still be O(n*log(n))16:47
natefinchrogpeppe: you going to make me sort the slices?16:48
rogpeppenatefinch: nope (you can't do that easily)16:48
natefinchrogpeppe: right, that's why I didn't :)  Sure, lay it on me16:48
rogpeppenatefinch: just sketching it16:48
mgznatefinch: I think I win my bet :)16:49
natefinchmgz: haha yep16:49
natefinchrogpeppe: just saying... you're optimizing something that is only used in tests and likely only ever used on n < ~516:49
mgzbut it's fun!16:50
rogpeppenatefinch: it should make the code more obviously correct too16:51
natefinchrogpeppe: I'm all for more obviously correct. I know it's not the prettiest right now, but the algorithm is pretty simple16:51
rogpeppenatefinch: something like this? http://paste.ubuntu.com/6007096/16:54
rogpeppenatefinch: i *think* that should be sufficient, but i might be missing something16:55
rogpeppenatefinch: i have to go in exactly 3 minutes16:57
natefinchrogpeppe: ok no big deal16:57
natefinchrogpeppe: I'll test that out16:57
rogpeppenatefinch: does that paste make sense?16:57
natefinchrogpeppe: yep.16:57
rogpeppenatefinch: essentially we just need to check that each slice has the same number of each item16:57
natefinchrogpeppe: yep16:58
rogpeppemgz: i think the simpler code is worth it, even despite the performance difference, which i don't really care about.16:58
mgzrogpeppe: I do agree really, also fun is worth it :)17:00
rogpeppemgz: :-)17:00
mgzrogpeppe: go now :)17:00
natefinchrogpeppe: thanks17:00
rogpeppemgz: yes, my wife has a bottle of cold champagne up my back - it's our anniversary and i really must17:01
rogpeppeg'night all17:01
natefinchrogpeppe:  enjoy!17:01
hazmatjamespage, try again re builldd on 0.7 rel branch, i was able to get through a ppa build of the same branch. there is some vagary based on buildd performance and running the test suite reliably. unfortunately its impossible to trigger outside of the buildd envs.17:02
mgzhazmat: that's pretty much what I found previously ;_;17:02
mgzgah!17:13
natefinchmgz: btw, I got azure working with some help from allenap... he had posted a fix that I missed earlier17:25
mgzace17:29
thumpermorning21:00
thumpermramm: ping?21:09
sidneithumper: around?22:05
sidneiah, nm. found https://code.launchpad.net/~themue/juju-core/037-empty-strings-in-charm-config/+merge/17831822:07
thumperhi sidnei22:30
thumperwas putting on slow cook asian pork for dinner in the slow cooker22:31
sidneigot bitten by the bug above22:35
sidneiwith juju-deployer and setting ''22:35
sidneichanged to ' ', which did the trick, but /o\22:35
fwereadesidnei, it's been languishing a bit, TheMue was on holiday for a couple of weeks, but he's back and actively on it AIUI22:40
sidneifwereade: thanks!22:40
thumperfwereade: you wouldn't believe how long it takes to unpick the dependencies on structs across packages22:47
mrammthumper: sorry, lost track of time23:00
fwereadethumper, oh, I would, that's why I tend to give up before I start :(23:02
mrammfwereade: hahaha23:03

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!