thumperwallyworld: ping02:34
thumperwallyworld: hangout?02:38
thumperwallyworld: ?^^02:39
* thumper is trying to work out the least shitty way to do something03:33
thumperthere doesn't seem to be a clean way to get the machine agent data dir from the api server client03:33
thumpersince this is needed to get the ssh identity file location...03:33
thumperI could pass the *agent.Config through about 4 layers03:33
thumperoh, we03:34
thumperI could use a global :-)03:34
thumperaxw, wallyworld, davecheney: opinions?03:34
wallyworldnoooo. not a global03:35
thumperpass the agent config pointer?03:35
wallyworldcould we attach it as a stuct attr somewhere/03:35
thumperit'll need to go machine agent -> api server -> root -> api03:35
thumperwallyworld: it is the three or four somewheres...03:36
wallyworldi know there are other places where agent.Config is passed around03:36
wallyworldso doing it that way at least is consistent03:36
wallyworldmaybe see how it is done elsewhere?03:36
thumperwallyworld: it is passed into the worker03:37
thumperbut it is normally only one level deep03:37
thumperthe api server is somewhat nested03:38
wallyworldah ok. i couldn't recall how it was used03:38
wallyworldwhat about adding it to the state struct of whatever03:38
axwthumper: I wonder if the key should end up getting plonked in the client key dir that I'm introducing, which will automatically get picked up by utils/ssh03:38
wallyworldnot sure, just hand waving03:39
thumperaxw: but the key is on the state server, not the local machine03:39
thumperaxw: two very different ssh bits03:39
axwthumper: the client key dir can be changed. it doesn't have to be ~/.juju/ssh03:40
axwjust a thought, not necessarily a good one03:40
thumperaxw: yeah, but I don't know which directory it is03:40
thumperthat is the problem :)03:40
axwthumper: at startup?03:40
thumperaxw: well the machine agent knows, but that information isn't passed down to the api server client yet03:41
thumperjust trying to work out how to get it down there...03:41
thumperI think just adding the param is a good start03:41
thumperbetter to store a single pointer to an already created structure03:41
thumperthan to pass down a string03:41
axwright, but the machine agent can initialise the client key dir to be whatever it knows about, and then when the api server uses utils/ssh, it'll get picked up. maybe I don't understand correctly03:41
thumperah... wat? how does utils/ssh pick up the client key dir?03:42
thumperaxw: currently we write the generated private key into $datadir/system-identity03:43
axwthumper: it's a global, unique per process thing03:43
thumperwallyworld: there ya go, use a global :)03:44
axwnormally I wouldn't advocate, but in this case it seemed cleanest. hasn't been reviewed yet though :)03:45
* thumper avoids the global05:20
thumperand dances through the apiserver weirdness05:20
* thumper throws in the towel for today05:46
thumperI'm pretty sure I have a way forward now...05:46
thumperat least05:46
thumpernight all05:46
=== OutOfControl is now known as benonsoftware
=== rvba` is now known as rvba
wallyworld_jam: i'm about to head out tonight to a function, so will miss the stand up. i've got about 4 pipes in progress, but none ready to propose yet08:08
jamwallyworld_: k, anything you want to summarize for the group?08:08
wallyworld_i've been adding a bunch of instrafstructure for the charm update work, api service pieces, and splitting it all up, plus refoactoring further the charm store interface and test mock08:09
wallyworld_jam: also,i've had no feedback from martin on that bug and mp we discussed, i'm keen to hear something from hiom about that08:10
* wallyworld_ runs away08:12
=== gnuoy` is now known as gnuoy
rogpeppemornin' all08:13
mgzI may be a few mins late for the standup, but I should be around for it09:42
jamrogpeppe: I think you asked about the fireworks in Dubai: http://www.youtube.com/watch?v=Oo1JwImshw8&list=WLD47DA262F88F149710:41
jamIts a better view than I had, since a lot of it is from above so you get to see the palm shape10:41
rogpeppejam: thanks10:43
rogpeppejam: that's quite a lot of fireworks...10:43
jamwhere we were, we only saw the primary palm island on edge.10:43
jamso a couple low shots is what it looked like to uss10:43
jamwelcome natefinch, how did you get here in the end ?11:09
jamrogpeppe: do you know if that was 1.16.3 against a 1.17 server, or a 1.17 against a 1.16.3 server?11:11
natefinchjam: ip address... something must be wonky with my DNS11:11
rogpeppejam: the latter11:11
jamIt doesn't reproduce for me just yet, but it looks like my /usr/bin/juju got updated over the vacaiton11:11
jamsudo apt-get install juju-core=1.16.3-0ubuntu0.13.10.1~ctools011:14
jamand now 'juju status' using 1.17 looks as described11:14
jammgz: http://paste.ubuntu.com/6708528/11:15
jammgz: ^^11:15
mgzjam: ta11:17
jammgz:  but I also see it after doing "juju upgrade-juju -e local --upload-tools" with juju 1.16.511:23
mgzokay, so it's probably not fixed for them11:23
jammgz: well "its fixed" was that they reverted to the 1.16.5 client, I bet11:24
jammgz: but you're going to file the bug, right, or shall I do it and assign it to you ?11:24
jamits certainly considered a critical bug that juju status in the 1.17 series is incompatible with the 1.16 series, just like add-machine, etc.11:25
jamI'm currently finishing add-machine11:25
jamwe removed a struct I was using :(11:25
mgzjam: if you could file it, I will assign to me11:34
jammgz: https://bugs.launchpad.net/juju-core/+bug/126673411:39
_mup_Bug #1266734: juju status incompatible in 1.17.0 against 1.16 <compatibility> <juju-core:Triaged by gz> <https://launchpad.net/bugs/1266734>11:39
jamthat's a bug in _mup_, Triaged by gz11:39
jamI assigned it to you, and it is triaged, but it certainly isn't triaged *by* you :)11:39
jam_mup_: make me a sandwich11:39
jamah, mup doesn't respond to pokes anymore :(11:40
jamaxw: I'm getting "environment is no longer alive" trying to add a machine to a 1.16 system.11:57
jamDid we mess up the default value for environment life?11:57
jammgz: ^^ that *might* be a hint as to what is going wrong? If juju 1.17 is saying "oh this env is dying, I won't probe any deeper", or something like that11:58
axwjam: I had tested when there was no life in the document... but it does sound like it points to that12:27
jamaxw: it is the assertAliveOp()12:35
jamafter debugging some more12:35
jamaxw: Life() still returns Alive12:35
jambecause 0 is the default value12:35
jamwhich is the enum Alive12:35
jambut assertAliveOp() asserts that "life == 0" when it "== nil)"12:35
axwjam: yeah, I thought it was working in the db level too. sounds like I messed up my testing12:36
jammgz: sounds like unrelated to status12:36
mgzyeah, that's what I was coming to with adding logging in status here12:36
jamaxw: well, I created a bug and assigned it to you already :)12:36
axwthanks, will get onto it in the morning12:37
_mup_Bug #1266748: Environments.Life() in juju-1.17.0 incompatible with juju-1.16.* <compatibility> <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1266748>12:37
mgzeveryone can triage it!12:37
jammgz: more importantly *I* can make anyone triage it :)12:37
mgzI'll try a hack fix and see if that make status happy12:38
jamaxw: ugh, and your latest change to auth keys also conflicts with the changes to add machine... sigh12:40
jamguess I'll look more tomorrow12:40
jamI refactored that code so it operated in a "poll for information, then set that information"12:41
jamrather than interleaving it12:41
jam(so that when we go to set it via the API we can fallback to directly setting it in the DB)12:41
jamaxw: note, what is your compatibility with 1.16 here with the ssh keys stuff?12:41
axwjam: I thought it was polling first to be honest - there were some late changes when merging with rogpeppe's updates, maybe it got broken around then. will have to investigate some more later12:43
axwjam: are you referring to the CL I put up today?12:43
jamaxw: you just landed code today that breaks my patch for bug #125363112:43
_mup_Bug #1253631: juju add-machine in trunk is incompatible with 1.16 <compatibility> <regression> <juju-core:Triaged by jameinel> <https://launchpad.net/bugs/1253631>12:43
jamwhen I mean "poll", I just mean, gather up the information we need to send in the add-machine request and do all the gathering before we start talking to the db.12:44
axwyeah, I thought that was being done12:44
* axw looks at what he merged today12:45
jamaxw: I created a new function "gatherMachineParams" that did all of the NewUUID, ParseIP, hostAddresses, checkProvisioned, etc12:45
jamand Arch, and etc12:45
jamit was a bit interleaved in the function before12:46
jamanyway, I can sort it out12:46
axwjam: ah, sorry12:46
jamjust one more time that this stuff got broken (ian did it last time by getting rid of State.AddMachine)12:46
axwjam: as for 1.16 compat, the stuff I've done is all client side so far12:46
jamaxw: I just need to get this landed so it is on everyone else to keep it working :)12:46
jamaxw: sure, but changing where SSH keys are found is going to break if someone uses 1.17 and then 1.16, right?12:47
jamaxw: InitUbuntuUser sounds a bit "mutating state" while we are "gathering information" (might be necessary, I'm not sure exactly yet)12:48
axwjam: sorry, I don't understand. if they use 1.17 and then 1.16...?12:48
jamaxw: from what I can tell, if I use 1.17 client in a 1.16 environment, and do "juju add-machine" it is going to set up that machine in a way that 1.16 may not understand12:48
axwjam: all that InitUbuntuUser does is ensures that ubuntu exists on the machine, and if not creates it and makes it a passwordless sudoer12:50
axwso that's not going to break anything older - this is only done at add-machine/bootstrap time anyway12:51
axwthe CL I put up today is about using additional keys from the client side; the additional public keys are added to authorized-keys in the env config12:51
axwso no changes needed on the server side there either12:52
=== gary_poster|away is now known as gary_poster
axwjam: I realised what happened with my testing. The assertion used to be !dead, then it got changed during review13:43
axwand I didn't retest13:44
rogpeppenatefinch: i've been wondering about IP address changes with respect to replica sets14:37
rogpeppenatefinch: do you know how we can deal with that?14:37
rogpeppenatefinch: (at the moment i don't think it's possible to change the IP address of a member, right?)14:37
rogpeppenatefinch: ping?14:43
natefinchrogpeppe: here now14:46
rogpeppenatefinch: what do you think of the above?14:46
natefinchrogpeppe: I don't think you can change the ip address, but you can remove and re-add14:46
rogpeppenatefinch: you *should* be able to change the ip address, at least according to this: https://groups.google.com/forum/#!topic/mongodb-user/_EUC63MJfQA14:46
rogpeppenatefinch: unless i'm misinterpreting14:47
=== hatch_ is now known as hatch
nate_finchrogpeppe: sorry, stupid USB ethernet adapter keeps making my computer hang up.  Didn't see anything after what I posted14:48
rogpeppe[14:46:58] <rogpeppe> natefinch: you *should* be able to change the ip address, at least according to this: https://groups.google.com/forum/#!topic/mongodb-user/_EUC63MJfQA14:49
rogpeppe[14:47:07] <rogpeppe> natefinch: unless i'm misinterpreting14:49
nate_finchrogpeppe: you're right.  I thought I had seen somewhere that you oculdn't change the host on a member, but maybe I'm thinking of something else. Looks like you can: http://docs.mongodb.org/manual/tutorial/change-hostnames-in-a-replica-set/14:52
rogpeppenate_finch: i think that means, unfortunately, that our current strategy of keying by address is not valid14:53
rogpeppenate_finch: jam also brought up the point that it would probably be better if we didn't denormalise the replica set voting info14:54
rogpeppenate_finch: which means, i think, that we want to be able to map from replicaset Members to *state.Machine14:54
rogpeppenate_finch: i think we can probably use the tagging functionality for that14:55
rogpeppenate_finch: we'd associate a tag holding the machine id with the replica set member14:55
nate_finchrogpeppe: so, there is a problem with replicaset.Set, if you modify the address of a member, yes.   I thought I had mostly copied that function from the javascript code, but maybe there's a subtle difference.  I can double check.14:57
nate_finchrogpeppe: I'm not sure what you mean by denormalizing the voting info14:58
rogpeppenate_finch: do the ids have to be strictly numbered from zero?14:58
nate_finchrogpeppe: I don't believe so, I was just following the way the javascript code works.14:59
nate_finchrogpeppe: so you want the id to be the same as the machine id?15:00
rogpeppenate_finch: i'm not sure if that's a good idea15:00
rogpeppenate_finch: but we could provide access to the Tags functionality15:00
rogpeppenate_finch: and state something like: if the id is greater than zero, it is treated as explicitly set15:01
rogpeppenate_finch: otherwise it will be automatically generated15:01
rogpeppenate_finch: then a client of the package can inspect the tags and determine which members match, according to the tags, and when updating, it can reuse the original indexes for previously existing members15:03
rogpeppenate_finch: does that make some kind of sense?15:03
nate_finchrogpeppe: what are we gaining?  Just being able to figure out what replica member corresponds to what machine, if its address changes?15:04
rogpeppenate_finch: it means we can use the replicaset member list as the canonical source of what machines are voting15:04
rogpeppenate_finch: rather than needing to trust that state reflects that correctly15:05
nate_finchrogpeppe: definitely using the information in the replicaset member list as the canonical source of who is voting is the way to go, since it *is* the canonical source :)  However, I'm trying to figure out when the host name in the member list would be wrong, and how this would help in that situation.  obviously if the hostname is wrong in mongo, then that member isn't voting anyway, right?15:07
rogpeppenate_finch: consider what will happen if a machine's address changes. the address updater will change the machine's address in state, and we will need to change the appropriate replica set member to reflect the new address15:09
rogpeppenate_finch: that member can't vote while it has the wrong address, but it's still a valid member (it's analagous to a network error cutting off that machine, i think)15:09
nate_finchrogpeppe: can't that just be another thing the address updater does, change it in the replica config?15:09
rogpeppenate_finch: i don't want two things both changing the replica config15:10
rogpeppenate_finch: seems like a recipe for race conditions15:10
nate_finchrogpeppe: hmm.. true.  I'm just not a fan of having things out of sync like that.15:12
rogpeppenate_finch: well, it's just one part of the address propagation15:13
rogpeppenate_finch: the machine's address was out of date for a while too15:13
nate_finchrogpeppe: I'm not averse to having the machine id be the replica id.  That's not a terrible solution, though maybe tags are more flexible in case of weird edge cases15:14
rogpeppenate_finch: i don't want to tie us to having integer machine ids15:14
nate_finchrogpeppe: ahh, as opposed to random strings?15:15
nate_finch"random"  = arbitrary15:15
=== nate_finch is now known as natefinch
rogpeppenate_finch: yeah15:15
natefinchrogpeppe: good point.15:15
natefinchrogpeppe: I don't know that tags is the right solution.  Couldn't we just add a value to the machine document in state to hold replica set member id?15:18
rogpeppenatefinch: using tags seems like a direct reflection of what we want to do15:20
rogpeppenatefinch: the other way around isn't so clearly correct to me15:20
rogpeppenatefinch: because the replica set must be updated independently from the machine doc15:21
rogpeppenatefinch: so there's a potential for things to get out of step15:21
rogpeppenatefinch: what are your reservations about using tags here?15:21
natefinchrogpeppe: tags are human-readable information used for selecting machines.  I don't think replica set id is really anything anyone will ever want to see in tags.15:22
rogpeppenatefinch: i don't think it's incompatible with that15:23
rogpeppenatefinch: { "juju-machine-id": "12" }15:24
rogpeppenatefinch: they don't look like they're for human-readable info to me, BTW15:25
rogpeppenatefinch: mostly it looks like they're used for write- and read-concern configuration15:25
rogpeppenatefinch: http://docs.mongodb.org/manual/tutorial/configure-replica-set-tag-sets/15:25
natefinchrogpeppe: crap, we were talking about different things.  I thought you meant tags in state on the machine docs15:26
natefinchrogpeppe: yes, that sounds fine15:26
rogpeppenatefinch: ah, cool15:26
rogpeppenatefinch: sorry for the misunderstanding15:27
rogpeppenatefinch: i'd forgotten state.Machine had tags :-)15:27
natefinchrogpeppe: haha no big deal15:27
mattywfwereade, we should probably have a chat this week about the next stage of user ids in juju15:30
rogpeppenatefinch: would you be able to move forward with that?15:40
rogpeppenatefinch: or i could if you're otherwise busy15:43
natefinchrogpeppe: I can do that.15:46
rogpeppenatefinch: thanks!15:47
rogpeppenatefinch: just to be clear, i'm thinking of this:15:48
rogpeppe// Set changes the current set of replica set members.  Members will have their15:48
rogpeppe// ids set automatically if they are zero.15:48
rogpeppeabentley: and a change to a Tags field to Member15:48
abentleyrogpeppe: That's good, 'cause I had no clue...15:49
rogpeppeabentley: :-)15:49
natefinchrogpeppe: yep, sounds good15:50
rogpeppenatefinch: and the automatic ids we'd choose would start from 1, not 015:51
natefinchrogpeppe: right15:51
fwereademattyw, for sure15:58
fwereademattyw, I'm sprinting today and flying tomorrow15:58
mattywfwereade, no problem, shall I put something in the diary for thursday or friday or would you prefer to wait till next week?15:59
fwereademattyw, if I come online tomorrow pm before you go, grab me -- otherwise it may have to be thursday15:59
fwereademattyw, let's do thursday, it's a meetingy day for me anyway15:59
mattywfwereade, ok cool - I'll put something in the diary15:59
mattywfwereade, thanks very much15:59
fwereademattyw, a pleasure, sorry it can't be sooner16:00
mattywfwereade, not at all - I have plenty to do in the meantime16:00
fwereademattyw, cool16:00
* rogpeppe grabs a late lunch16:05
=== dames is now known as thedac
rogpeppenatefinch: important thing i've just noticed, (from http://docs.mongodb.org/manual/tutorial/configure-replica-set-tag-sets/) the type of the Tags field should be map[string]string17:31
rogpeppenatefinch: do you have a strong idea as to what we should do if we find extra members in the peer set that don't correspond to machines?17:42
natefinchrogpeppe: yeah, I was making the Tags map[string]string .    Not sure what to do if we find extra members.   Honestly, I think if we just ignore that condition it'll basically take care of itself, which is to say, normally we'll just ignore them, if the normal process is <get config> <change node in config> <set config>18:05
rogpeppenatefinch: it's not quite that straightforward18:05
rogpeppenatefinch: for the time being i'm just going to return an error if there extra members18:06
rogpeppenatefinch: i've added a TODO comment with various options18:06
natefinchrogpeppe: that's probably fine... it shouldn't really ever happen unless people go messing with the replicaset by hand, in which case, not mucking with it is probably the right thign to do18:06
rogpeppenatefinch: BTW i just wanted the replica set id from the CurrentStatus results - it's not there and not documented, but it appears in the actual results18:07
rogpeppenatefinch: do you think it's reasonable to add it as a field in the Status type?18:08
natefinchrogpeppe: sure, no harm18:09
rogpeppenatefinch: one mo, i'll just check that it is the expected id18:09
natefinchrogpeppe: probably we should just export a ReplicaSetStatus struct that includes the data from replSetGetStatus18:10
rogpeppenatefinch: isn't that what Status is?18:11
natefinchrogpeppe: status just returns the members right now18:11
natefinchrogpeppe: the member statuses that is18:11
rogpeppenatefinch: ah yes18:11
natefinchrogpeppe: brb diaper18:12
rogpeppenatefinch: i don't care much at the moment18:12
natefinch(not mine ;)18:12
natefinchrogpeppe: back18:18
rogpeppenatefinch: confirmed that _id is the replica set id18:18
rogpeppenatefinch: i thought it was strange that it wasn't there, otherwise there's no way of cross-correlating statuses with members18:19
rogpeppenatefinch: unless you assume that there's a one-to-one correspondence in the arrays, but that's not documented either18:20
natefinchrogpeppe: you mean between the members and their statuses in replSetGetStatus?  I presume there's a 1:1 relationship.  I'd be surprised if there were not18:23
rogpeppenatefinch: i would too, but i can't find anything that documents that fact18:24
rogpeppenatefinch: and the fact that we can't fetch them both at the same time makes me naturally wary18:24
rogpeppenatefinch: so given that the _id field is provided, i'd rather latch onto that18:24
natefinchrogpeppe: that's fine18:25
=== _bjf is now known as bjf
* rogpeppe is done for the day19:01
rogpeppeg'night all19:01
natefinchrogpeppe: g'night19:02
=== hatch_ is now known as hatch
thumpermorning folks19:53
thumpero/ natefinch20:44
thumpernatefinch: care for a catch up call?20:45
natefinchthumper: sorry, just got surprise acquisition of the kids for a bit.  When my wife gets back, I'll be able to catch up, but kind of occupied at the moment, unfortunately20:53
thumpernatefinch: np20:53
natefinchthumper: still want to catch up?21:24
thumpernatefinch: yeah21:24
thumpernatefinch: https://plus.google.com/hangouts/_/7acpicc4tp7juqvoctqj0i9qb4?hl=en21:25
thumpernatefinch: you have frozen21:49
* thumper goes to tell the dog to be quiet...21:50
=== gary_poster is now known as gary_poster|away
thumperdecisions... decisions...23:35
thumperdo I hook up my crazy shit and try it23:35
thumperor write more tests first23:35
thumperdavecheney: you around?23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!