/srv/irclogs.ubuntu.com/2013/02/25/#juju-dev.txt

davecheneyping thumper00:02
fwereade__davecheney, thumper, whoops, I should sleep00:15
thumperfwereade__: I guess so :)00:15
davecheneythumper: are your tests still broken ?00:15
fwereade__davecheney, thumper: if either of you is inclined to go on a trawl through my +activereviews I have a whole bunch and they should all be relatively easy00:16
davecheneyi have some suggestions that you can apply from my experiences making the tests work under Q00:16
thumperdavecheney: I'm running a quantal virtual box00:16
thumperI've not tried today with raring00:16
fwereade__davecheney, thumper: they're all in a pipeline because they're all things I need to do to connect constraints up to environs00:16
* fwereade__ waves and departs00:16
thumperfwereade__: ok00:16
thumperdavecheney: do you know where lbox stores the google auth token?00:17
davecheney~/.lpad_oauth00:18
thumperta00:18
thumperdavecheney: it seems that a whole bunch of ubuntu pastebins were deleted over the weekend00:21
thumperno idea why00:21
thumperdavecheney: also, ~/.lpad_oauth seems to only be the launchpad credentials00:22
thumperdavecheney: I deleted it and tried lbox propose00:22
davecheneythumper: good point00:23
thumperdavecheney: but it used the google credentials I was trying to change00:23
davecheneylet me look again00:23
davecheneythumper: ~/.goetveld_*00:26
thumperdavecheney: ta00:26
davecheneyone per host00:27
davecheneyshould only be one00:27
* thumper nods00:27
* thumper deletes it00:27
thumperand tries again00:27
thumperwow: error: Failed to load data for project "juju-core": use of closed network connection00:28
thumpertried again... and it does something00:28
davecheneyyup, nobody is sure if that is LP, or something weird in the go http client00:29
davecheney2013/02/25 11:28:41 JUJU environs: searching for tools compatible with version: 1.9.10-quantal-amd6400:29
davecheney2013/02/25 11:28:45 JUJU juju bootstrap command failed: cannot start bootstrap instance: cannot allocate a public IP as needed00:29
davecheneyand the public IP shortage continues to make it impossible to test canonistack00:29
thumperdavecheney: I'm tempted to just leave the raring test issues until atlanta, as I expect the round-tripping will be a lot faster00:43
davecheneythumper: i'll propse a branch00:43
davecheneyi know most of the places that are broke00:43
thumperoh, ok00:43
thumperI am happy to test00:43
davecheneyfrom memory it is the test mocks00:44
davecheney/s/mock/fixtures00:44
davecheney        <h1>Please try again</h1>00:47
davecheney        <p>00:47
davecheney          Sorry, there was a problem connecting to the Launchpad server.00:47
davecheney        </p>00:47
davecheney        <p>00:47
davecheney          Try reloading this page in a minute or two.00:47
davecheney          If the problem persists, let us know in00:47
davecheney          <a href="irc://chat.freenode.net/#launchpad"00:47
davecheney          >the #launchpad IRC channel on Freenode</a>.00:47
davecheney        </p>00:47
davecheney        <p>Thanks for your patience.</p>00:47
davecheneylove this00:47
davecheneythumper: Proposal: https://code.launchpad.net/~dave-cheney/juju-core/097-environs-raring-fixtures/+merge/15025400:49
davecheneyRietveld: https://codereview.appspot.com/738004700:49
thumperdavecheney: what's with the 097 branch prefix?00:52
davecheneydunno, just something we all started doing00:52
davecheneypersonally, i got lost in all the open branchs I had00:52
davecheneynumbering them at least let me know where they were temporarlly00:52
thumperdavecheney: hmm, still fails a lot, I'll pastebin the errors00:56
davecheneyta00:57
davecheneygonna have to do this blind, i'm not upgrading to raring00:57
thumper:)00:58
davecheneysorry, i need my machine to work00:59
thumperdavecheney: http://paste.ubuntu.com/5563524/01:01
thumperdavecheney: I've managed to keep working using a quantal VM I had lying around :)01:01
davecheneyyou need to do the git dance01:01
davecheneyarguably we should pass a faux gitconfig01:02
thumperI have done that...01:02
davecheneydance harder, monkey boy01:02
thumperhow to I get git to tell me what it has configured for user and email?01:03
davecheneycat ~/.gotconfig01:03
davecheneycat ~/.gitconfig01:03
thumper$ cat ~/.gitconfig01:04
thumper[user]01:04
thumperemail = tim@penhey.net01:04
thumpername = Tim Penhey01:04
thumperand it still fails01:04
davecheneyle sigh01:04
thumperso I'm wondering what else has changed01:04
davecheneyeither way, the test shouldn't assumed you've done the git dance01:04
thumperagree there 100%01:04
davecheneythumper: am fixing now01:05
davecheneycan you be a sport and raise an issue01:05
thumperdavecheney: for juju-core?01:06
davecheneysure01:06
thumperfor the entire test failure, or just the git dance?01:06
davecheneyeither01:06
thumpersure01:06
davecheneyraise as many or as few as you like01:06
thumperthe tests pass with quantal01:06
thumperand I hadn't done the git dance there (at least I don't think I had)01:07
thumperor if I had, something else has changed between quantal and raring about where git looks01:07
davecheneyyeah, they didn't used too, which is why i'm wary of upgrading to raring01:07
* thumper files a bug about tests failing on raring01:07
davecheneyok, raise an issue about the git parts failing01:07
davecheneythat is the biggest part of the failure atm01:07
thumperhmm...01:09
thumperfiled bug 1132585 just now01:09
_mup_Bug #1132585: Tests fail on raring <juju-core:New> < https://launchpad.net/bugs/1132585 >01:09
davecheneyta muchly01:09
thumperdavecheney: boo...01:19
thumperI thought I was going to be so clever01:19
thumperbut go didn't like it01:19
thumperI was thinking I'd be able to inject a couple of methods into the cmd.Context function set from a testing module01:20
thumperbut it ddn't work01:20
thumperwas wanting something like:  func (c *cmd.Context) StdoutString() string01:20
thumperbut go said no01:20
* thumper decides to use testing.stdout(c *cmd.Context) instead01:21
thumperwith a capital S01:21
thumperI think that is nicer than bufferString(c.Stdout)01:21
thumpertesting.Stdout(c) instead01:22
thumperdavecheney: what do you think?01:22
* davecheney doesn't really follow01:22
davecheneypaste ?01:22
thumperdavecheney: the guts of it is: we have two identical bufferString implementations01:22
thumperand I was wanting to not01:22
thumperalso considered changing the names somewhat01:23
thumperas I thought it read better01:23
thumperam I being too anal?01:23
* davecheney doesn't really follow, has three people talking to him at the moment01:23
thumper:)01:23
* davecheney is confused, i've hacked the uniter tests to supply a completely impossible value for GIT_CONFIG, and the tests still pass01:24
* thumper shelves the change for an anal day01:24
davecheneyu gonna submit your cmd aliases branch, or what ?01:24
thumperI did01:31
thumperI thought I did it last week01:31
thumperbut for some reason it didn't go through01:31
thumperfixed and resubmitted today01:31
thumperugh01:35
thumpergnuflag FTL01:36
* thumper realises that a bunch of refactoring wasn't needed01:36
* thumper relocates01:38
davecheneythumper: i can't figure it out, i've removed my own .gitconfig and the tests still pass04:59
davecheneythumper: def a version change for git in raring05:05
davecheneyhttps://launchpad.net/ubuntu/+source/git05:05
davecheneyNFI what has changed05:05
davecheneycan you post the source for goroutine 10 [select]:05:43
davecheneygithub.com/wharf/tuplespace.(*TupleSpaceImpl).run(0xc20015f000) /Users/alec/.go/src/github.com/wharf/tuplespace/tuplespace.go:164 +0xb8805:43
davecheneycreated by github.com/wharf/tuplespace.NewTupleSpaceImpl /Users/alec/.go/src/github.com/wharf/tuplespace/tuplespace.go:58 +0x142 ?05:43
jamwallyworld_: poke06:44
wallyworld_hi06:44
wallyworld_mumble?06:44
jamyep06:44
jamjust got on06:44
dimiternfwereade__: ping08:09
rogpeppedimitern, fwereade__: morning!08:11
dimiternrogpeppe: morning!08:11
dimiternrogpeppe: how was your holiday?08:11
rogpeppedimitern: very good, thanks. nice weather, made it to the top of a few hills, had some nice food.08:12
dimiternrogpeppe: cool :)08:13
jamfwereade__, rogpeppe: Maybe one of you guys know. How do you compile just the test binary? I want to profile it, and I can run "go test -cpuprofile" but "go tool pprof" needs the build binary as a reference to interpret the profile output08:18
rogpeppejam: hiya08:18
jamBut I can't find a flag for "build just the test binary"08:18
rogpeppejam: go test -c08:18
jamrogpeppe: anyway to actually find the documentation for that?08:18
rogpeppejam: go help test08:18
jamgo help testflag doesn't seem to have it08:18
rogpeppejam: testflag gives help on the flags for the test binary itself08:18
jamrogpeppe: there are 4-different ways to get help, and I missed that one... :(08:18
jamgo test -h no good08:18
jamgo help08:19
jamgo help testflag08:19
jamgrep around online for a while08:19
jamrogpeppe: thanks for at least pointing it to me. I really expected to find it somewhere else08:19
rogpeppejam: go help <command> should be your first stop08:19
rogpeppejam: the testflag documentation is unusual08:19
rogpeppejam: because it's not really documentation on a go subcommand08:20
rogpeppejam: but it is referred to by the go test help08:20
rogpeppejam: (how else did you know about testflag?)08:20
jamrogpeppe: well 'go <command> -h' is my natural first stop, but that ==> 'go help' which is, unhelpful IMO08:20
jamwhy would 'go help <subcommand>' not match 'go <subcommand> -h'08:20
jamrogpeppe: note that "go tool command -h" *is* the way to get help for individual commands as part of tools.08:21
rogpeppejam: i dunno. but to be fair that (and go help) both say "Use "go help [command]" for more information about a command."08:21
jamThough "go help tool" *doesn't* show you what tools are available.08:21
rogpeppejam: they might well accept a patch making go command -h work08:22
jamrogpeppe: and "go tool -h" also == "go help tool" which is a bit inconsistent.08:22
jamrogpeppe: as is 'go build -h'08:23
jam(I'm not planning on trying all of them, though)08:23
rogpeppejam: oh yes, so it is08:23
rogpeppejam: that's definitely an inconsistency i'd never noticed08:24
rogpeppejam: (because i've always used "go help x" - as i do in bzr and hg etc)08:24
jamrogpeppe: also, there is no way to get the flags for the test command, you have to do some weird invocation of "go test -invalidflag" and then the binary fails and reports its options08:24
rogpeppejam: (but not juju!)08:24
jamthough those *don't* include the ones to 'go test' itself, because it is a different binary that interprets them.08:24
rogpeppejam: go help testflag, no?08:25
jamrogpeppe: And I've generally always used "bzr command -h" :)08:25
rogpeppejam: ah, except not quite08:25
jamrogpeppe: go help testflag doesn't do the -gocheck.* etc.08:25
jamI tried reporting an issue on it08:25
jamthat there was no way to get the flags for the test suite08:25
jamand it was sort of punted.08:25
rogpeppejam: you might be able to do "go test . -help"08:26
jamrogpeppe: nope, that '-help' gets interpreted by 'go test' and just get 'go help' output08:26
rogpeppejam: yeah, that seems wrong08:28
jamrogpeppe: anyway thanks, -c helped me get what I wanted, and I can use the 'go help foo' style08:28
rogpeppejam: np08:28
rogpeppejam: it'd be worth raising an issue i think. now's the time if we want it fixed by go 1.108:29
jamrogpeppe: care to raise it? Might get more attention from you or dave08:30
rogpeppejam: i'd appreciate a review of https://codereview.appspot.com/7390043 if you have some time at some point, BTW.08:30
rogpeppejam: ok08:31
rogpeppejam: FWIW, go test is the *only* subcommand for which -h doesn't work08:39
rogpeppejam: it's probably something to do with the fact that it mixes flags between top level and compiled-binary level08:39
TheMueMorning all.08:41
dimiternTheMue: morning08:42
fwereade__TheMue, dimitern, rogpeppe, jam: mornings08:42
rogpeppefwereade__: yo!08:42
rogpeppefwereade__: i'm angling for reviews of https://codereview.appspot.com/7390043, but you're probably aware of that :-)08:43
dimiternfwereade__, jam: morning08:51
dimiternmgz: hey08:51
mgzhey dimitern08:55
fwereade__rogpeppe, just sent a slightly bewildered one :)09:04
rogpeppefwereade__: ok09:04
fwereade__rogpeppe, can you think of any reason for the ec2 tests to depend on the dummy provider?09:04
rogpeppefwereade__: i don't *think* so09:04
rogpeppefwereade__: do they?09:05
fwereade__rogpeppe, can't run them without fixing the dummy provider, it seems, but I can't quite figure out the connection09:05
rogpeppefwereade__: ah, i know09:06
rogpeppefwereade__: the tests build the tools when testing PutTools09:06
fwereade__rogpeppe, oh, ffs, ofc09:07
rogpeppefwereade__: and i guess the tools depend... hmm, they shouldn't09:07
fwereade__rogpeppe, ah!09:09
fwereade__rogpeppe, juju/testing uses the dummy provider09:09
rogpeppefwereade__: ah, of course09:09
fwereade__rogpeppe, btw: why does StartInstance take apiinfo/stateinfo?09:09
* rogpeppe tries to remember09:10
fwereade__rogpeppe, fwiw I'm also planning to drop the tools param09:10
rogpeppefwereade__: oh yes, the idea was that the new instance needs to know how to contact the state server09:11
fwereade__rogpeppe, yeah, but the environment already knows that09:11
fwereade__rogpeppe, the only thing that's actually needed is the password09:12
fwereade__rogpeppe, (btw are passwords for stateinfos or just for apiinfos? it's not clear why we save them for both when bootstrapping and only for state when startinstanceing)09:12
=== negronjl` is now known as negronjl
rogpeppefwereade__: i'm not sure what you're referring to by that last sentence09:14
fwereade__rogpeppe, when you start a machine with a api info, what uses that API info?09:16
rogpeppefwereade__: i think i was trying to avoid making the assumption that the environment always knows where the state server is.09:16
fwereade__rogpeppe, why?09:16
fwereade__rogpeppe, that's the one core thing we depend on the environment being able to do for us ;p09:16
rogpeppefwereade__: we might want to store the current set of state server addresses in the state itself.09:17
rogpeppefwereade__: in fact for the state server (as opposed to the API server) i think that's going to be the case09:18
rogpeppefwereade__: hmm, but actually09:18
rogpeppefwereade__: we're going to be able to drop state.Info eventually09:19
rogpeppefwereade__: when you start a machine with an api info, the api info is used by the agents on that machine to connect to the API server.09:20
rogpeppefwereade__: one advantage of passing a state info into StartInstance is that the environ doesn't have to query the bucket and resolve the DNS name each time. but perhaps that's a trivial cost.09:22
rogpeppefwereade__: and it could be cached.09:22
jamrogpeppe: so I'd like to ask a bit about possibly caching the 'bundleTools' output during the test suite. Right now it takes about 2.5s for it to compile jujud and then tarball.gz it, and it happens far more times than I would expect.09:26
jam(eg, "BootstrapMultiple" test actually rebuilds the tools at least 2 times for that one test, but it also happens between tests.)09:27
rogpeppejam: yeah, it would be nice to get rid of that overhead09:27
dimiternfwereade__: I think waitHooks{} is not working correctly09:27
dimiternfwereade__: or maybe I'm not using it right09:27
rogpeppejam: i could never think of a sufficiently clean way of doing it, but i didn't think about it that hard.09:28
fwereade__rogpeppe, ok, long-term, I agree that the state info will probably be stored elsewhere and distributed separately, but for now it seems that the state info is the one we use and the api info is just kinda hanging on alongside without any agent actually using it yet -- is this correct?09:28
fwereade__dimitern, hmm, ok... would you paste me your output again and I'll see if any inspiration strikes?09:28
rogpeppefwereade__: agents will use it very soon. they do actually connect to it, i believe, currently, even if they don't actually use it.09:28
rogpeppefwereade__: actually... maybe they won't use it so soon.09:29
rogpeppefwereade__: hmm.09:29
dimiternfwereade__: I'm augmenting the logging to see what's going on, just a moment09:29
rogpeppefwereade__: yes, the API info isn't currently used. but it will be - it's a placeholder for what's to come.09:34
dimiternfwereade__: I found it!09:34
* fwereade__ cheers at dimitern09:34
fwereade__dimitern, go on09:34
dimiternfwereade__: matchLogHooks does not accept u/0 db2:009:34
fwereade__dimitern, grar, sorry09:35
dimiternfwereade__:  `u/0(| [a-z-]+:[0-9]+)` that's the hook pattern09:35
fwereade__dimitern, yeah, all  makes sense09:35
dimiternfwereade__: I'll change it to `u/0(| [a-z0-9-]+:[0-9]+)`09:35
fwereade__dimitern, sgtm09:37
dimiternfwereade__: now the db2-relation-joined mysql/0 db2:0 is matched, but still somehow ignored09:38
fwereade__dimitern, discraded by test id badge checking?09:38
dimiternfwereade__: still checking..09:39
dimiternfwereade__: :) the other regexp - donePattern needs changing as well09:41
fwereade__dimitern, ah, ok09:41
dimiternfwereade__: now it works :) yay!09:44
fwereade__dimitern, w00t!09:44
dimiternfwereade__: well, it fails, but for a different reason09:44
dimiternfwereade__: matching seems to pick up all hooks now09:45
dimiternfwereade__: this is what I got before the failure: [LOG] 0.53984 JUJU actual: []string{"install", "config-changed", "start", "upgrade-charm", "config-changed", "db2-relation-joined mysql/0 db2:0"}, ctx.hooks: []string{"install", "config-changed", "start", "db2-relation-joined mysql/0 db2:0"}09:45
dimitern[LOG] 0.54002 JUJU expected []string{"install", "config-changed", "start", "db2-relation-joined mysql/0 db2:0"}; never got anything09:45
dimiternfwereade__: upgrade-charm hook is missingm hence the failure09:46
fwereade__dimitern, yeah, I think you should specify the exact sequence of hooks09:46
fwereade__dimitern, I think if anything it makes the test clearer actually09:47
fwereade__dimitern, "do these things as close together as possible; check everything happens in the correct order"09:47
dimiternfwereade__: yeah, I commented out the waitHooks{"upgrade-charm", "config-changed"} before the db2-r-j one, now I uncommented that and run it again09:47
dimiternfwereade__: and guess what - OK: 1 passed09:48
dimiternfwereade__: cool! I'll propose it now09:48
dimiternfwereade__: I could debug it effectively by replacing c.Logf() with log.Printf() - we should think about this probably (otherwise these are 2 separate logs and are out of sync)09:51
fwereade__dimitern, expand please... I'm pretty sure that log.Printf is the one that produces apparent sequence breaks09:51
dimiternfwereade__: well juju uses log.Printf/Debugf and all lines are in a nice, expected order when printed09:53
fwereade__dimitern, gaah sorry I totally misread you09:54
dimiternfwereade__: c.Logf on the other hand is a separate log, so the combined output is not sequential09:54
fwereade__dimitern, they're happening on different goroutines anyway, though...09:55
dimiternfwereade__: yes, and we you look at the output it's not helpful (smth happened here and before or after is happened this other thing in the other log)09:56
dimiterns/we//09:56
dimiterns/is//09:56
dimitern:) I'm too fast to be readable today09:56
fwereade__:)09:57
dimiternfwereade__: running a couple of times without the log hacks, to make sure the timing wasn't screwed somehow09:57
fwereade__dimitern, well, you can see the two streams of independent events going on, right, they're just interspersed -- how would that change if they used the same log?09:58
dimiternfwereade__: well, you can tell that all lines after X happened after X happened, while now some things before X also happened after X09:59
dimiternfwereade__: X = juju log09:59
dimiternfwereade__: it'll be really nice if we separated parts of the uniter tests in smaller chunks (they can still use the same table format) - maybe on logical boundaries ("// XYZ tests" + NL)10:02
fwereade__dimitern, yeah, I think that'd be a definite win10:03
dimiternfwereade__: there it is https://codereview.appspot.com/738504910:04
fwereade__dimitern, cool, will take a look shortly10:04
fwereade__dimitern, thanks :)10:05
dimiternfwereade__: cheers, I left some questions in comments10:06
fwereade__dimitern, reviewed, LGTM, some comments10:32
dimiternfwereade__: thanks!10:32
fwereade__rogpeppe, dimitern, jam: I would very much appreciate your thoughts on https://codereview.appspot.com/7394048 which is still WIP10:42
dimiternfwereade__: I'll take a look in 2m10:42
rogpeppefwereade__: chunk mismatch. please reupload.10:42
fwereade__rogpeppe, what does that complaint actually imply? what will change if I repropose?10:43
rogpeppefwereade__: i've no idea, but reproposing usually seems to fix the problem.10:43
fwereade__rogpeppe, haha, ok :)10:44
fwereade__rogpeppe, better?10:45
rogpeppefwereade__: yup10:45
rogpeppefwereade__: thanks10:45
fwereade__rogpeppe, cool, wish I understood what was up there10:45
rogpeppefwereade__: me too. let me know if you find out!10:46
rogpeppefwereade__, dimitern: i've responded to your comments on this, and made some changes: https://codereview.appspot.com/739004310:50
rogpeppefwereade__: (i've also identified one significant omission, which i will be addressing shortly)10:51
fwereade__rogpeppe, sorry, it was clearly monday morning... I think I read the goroutine as a defer or something10:52
rogpeppefwereade__: ah, i wondered where the comments were coming from :-)10:53
dimiternrogpeppe: cheers, I'll take a look10:53
dimiternfwereade__: can we assume my CL closes bug 1101140 ?10:56
_mup_Bug #1101140: uniter does not skip unimplemented relations <upgrade-charm> <juju-core:In Progress by dimitern> < https://launchpad.net/bugs/1101140 >10:56
fwereade__dimitern, yep10:56
dimiternfwereade__: sweet!10:57
fwereade__:D10:57
dimiternfwereade__: one down, 7 to go (wrt upgrade-charm bugs)10:58
fwereade__dimitern, yay :)10:58
fwereade__dimitern, SetCharm is the next goal, but I'm not sure whether you should be handling the config changes first or the sanity checks first10:59
dimiternfwereade__: I'll revisit the notes and we may need to have a g+ later perhaps?11:00
fwereade__dimitern, I suspect doing the config changes is your best bet... maybe go with that, not worrying about the refcounting for now11:00
fwereade__dimitern, but, yeah, reread the notes and let me know11:00
fwereade__dimitern, I'd really appreciate thoughts on the Environ change first though :)11:01
dimiternfwereade__: I'm on it now11:01
fwereade__dimitern, <311:01
dimiternbtw i'll need one more LGTM on this guys https://codereview.appspot.com/7385049/ - anyone willing?11:02
fwereade__rogpeppe, what's the other oversight? can't spot it, so LGTM, but you've made me nervous ;p11:07
rogpeppefwereade__: i never tear down the watchers when the connection is dropped11:07
fwereade__rogpeppe, ha! I remember that thought crossing my mind and then flitting away before I captured it in the first read11:08
fwereade__rogpeppe, ty, good catch11:08
rogpeppefwereade__: it requires a little more more mechanism, and i'm not quite sure of the best method yet11:08
rogpeppefwereade__: am reviewing your bootstrap refactoring branch BTW11:09
fwereade__rogpeppe, comments always welcome but it's far from complete11:09
fwereade__rogpeppe, it's totally focused on getting ec2 working, nothing else11:10
rogpeppefwereade__: i'm trying to think about implications for future providers11:10
fwereade__rogpeppe, I have made an effort to consider that but am glad for the backup scrutiny11:12
fwereade__rogpeppe, LaunchInstance might actually be something more like RunMachineAgentOnInstance if you're bothered by the local provider11:13
rogpeppefwereade__: how does the new environs.Bootstrap square with the local provider?11:15
fwereade__rogpeppe, what does it use that any provider doesn't provide?11:15
rogpeppefwereade__: i'm not sure that the local provider provides a working StartInstance, does it?11:16
fwereade__rogpeppe, no reason it can't do it, though, is there? the "instance" is always the local machine, but it gets "launched" when the machine agent is started11:16
fwereade__rogpeppe, same deal as the SSH provider11:17
rogpeppefwereade__: and what about future environments that have no bootstrap instance?11:17
fwereade__rogpeppe, I don't see that you'd use bootstrap at all in that case11:18
fwereade__rogpeppe, there's no actual bootstrapping going on there, is there?11:19
rogpeppefwereade__: yeah - you'd tell some other machine to create an environment for you.11:19
rogpeppefwereade__: that's still "bootstrapping" even though it doesn't require a new machine11:19
rogpeppefwereade__: in general, i like the way that the refactoring looks (and it's necessary) but i'm not sure about its exact positioning11:20
fwereade__rogpeppe, well, it performs tasks that have *some* degree of overlap with those in Bootstrap11:20
TheMuedimitern: You've got a review.11:21
dimiternTheMue: thanks!11:21
fwereade__rogpeppe, go on11:21
TheMuedimitern: yw11:21
rogpeppefwereade__: i'm still mulling it over11:21
dimiternfwereade__: reviewed11:25
fwereade__dimitern, cheers11:25
rogpeppefwereade__: i'm wondering about leaving environs.Bootstrap as it is, but providing another function, say environs.BootstrapInstance, that contains the bulk of what ec2.Bootstrap does now (the tools uploading and the instance starting). then ec2.Bootstrap (and other similar providers) would just call that, but stranger providers would be free to do as they wished.11:29
fwereade__rogpeppe, is your concern specifically about starting instances? because that feels to me like the only behaviour that should differ11:30
rogpeppefwereade__: it's not just about starting instances. it's also about the way that your proposal makes Bootstrap into a fundamentally racy thing when called concurrently.11:31
rogpeppefwereade__: perhaps i should publish my current comments on the CL11:31
fwereade__rogpeppe, was it not so beforehand?11:31
fwereade__rogpeppe, sgtm11:31
rogpeppefwereade__: (they're superceded by the above thought, but still pertinent to the CL)11:32
dimiternmgz: standup?11:32
mgzdimitern: yup11:32
rogpeppefwereade__: no, Bootstrap was only racy because its implementations were racy, not because the operation is fundamentally racy, i think.11:33
rogpeppefwereade__: as a trivial example, dummy.Bootstrap isn't racy11:34
rogpeppefwereade__: i published my comments  BTW11:35
allenapHi. In Python Juju, how does Juju get onto a machine that I'm deploying to? It is fetched from Launchpad? I'll put it another way. I make some code changes to a machine provider in my local branch of lp:juju. I then deploy a charm using this code. Does the allocated machine use my updated code? Does a machine need the machine provider code? (MAAS in this instance.)11:35
* TheMue enjoys lunch, biab11:35
fwereade__rogpeppe, cheers11:35
fwereade__allenap, depends on juju-origin in the config11:35
fwereade__allenap, IIRC you can set it to your own lp: branch11:35
rvbajuju-origin: ppa11:36
rvbaallenap: ^11:36
fwereade__allenap, you'll need to push before it'll use it though11:36
fwereade__allenap, btw, I have a maas question11:36
fwereade__allenap, can a process on a maas node easily find out the maas instance id?11:36
allenapfwereade__: Cool, thank you, I think that'll be useful.11:36
allenapfwereade__: rvba is checking that for you.11:38
fwereade__allenap, rvba, cheers11:38
allenapfwereade__: Doesn't look like it. What's the use case?11:39
fwereade__allenap, there's this foul InstanceIdAccessor hack that's persisted all the way back since orchestra and is uglying up every provider11:40
* allenap looks it up11:40
rvbafwereade__: I think the only thing you have is the 'name' of the node.11:41
fwereade__allenap, the bootstrap node needs to know its own instance id and set it before its provisioning agent starts trying to provision an instance for itself11:41
fwereade__rvba, and that's not the same as the instance id, I guess?11:41
rvbaNo11:41
rvbaThe 'name' is just the node's hostname.11:42
fwereade__rvba, ok, thanks, no worries11:42
fwereade__allenap, it's not a big deal, I can certainly make it less evil regardless11:43
allenapfwereade__: The MAAS provider (Python) does call `cloud_init.set_instance_id_accessor(instance_uri)` so I assume it's getting over to the machine somehow.11:44
rvbaI'll check that it's there as soon as my node comes up…11:45
rvba(I've just destroyed my environment in the lab)11:45
rvbafwereade__: actually, allenap was right, the instance id is in /var/lib/cloud/instances/11:52
rvba$ ls lib/cloud/instances/11:52
fwereade__allenap, rvba, wow, really? cool11:52
rvbanode-74c10d5c-7f34-11e2-9a9a-52540012345611:52
allenapRight, so the directory names is the instance ID?11:53
rvbaYes11:53
allenaps/names/name/11:53
rvbaContains cloud-init stuff.11:53
allenapIs it also in a file within?11:53
fwereade__rvba, allenap: in that case I think we can just straight-up drop InstanceIdAccessor and ask providers to implement InstanceId just as they do PrivateAddress/PublicAddress11:54
fwereade__rvba, allenap: we can always hack it in via provider-specific machine config for functionality-challenged clouds in the future11:54
mgzfwereade__: that would make more sense than the chunk of bash method used currently11:55
fwereade__mgz, yeah, it's total crack11:55
fwereade__mgz, it's only there for orchestra11:55
allenapfwereade__: I haven't got my head around what InstanceIdAccessor does yet, so I'm not confident to say, but if all we need is the instance ID on the node then I guess we're okay.11:56
fwereade__allenap, that is all it does11:56
allenapCool.11:57
fwereade__rogpeppe, repsonded12:15
rogpeppefwereade__: looking12:20
rogpeppefwereade__: there's something about the new structure that doesn't feel quite right to me. a combination of a few things, i think12:25
rogpeppefwereade__: for instance, why does LaunchInstance now depend on a cloudinit data structure, when cloudinit is really just an artifact of one particular way of starting instances?12:26
fwereade__rogpeppe, that data structure *defines* how we configure an instance -- AFAICT it's only in cloudinit because we don't have a more general place for it yet12:27
fwereade__rogpeppe, the important bit is the "MachineConfig", not the "cloudinit.", I think12:28
rogpeppefwereade__: i don't think so. it defines the information that the generated cloudinit script needs to start stuff up, but that's a slightly different thing.12:28
fwereade__rogpeppe, please expand upon the distinction12:28
fwereade__rogpeppe, AFAICT MachineConfig is a representation of what needs to be done to get a machine agent running, and the other stuff in cloudinit is resposible for transforming that into  cloudinit data12:30
fwereade__rogpeppe, I anticipate that other providers will maybe have different ways of transforming fundamentally the same data12:31
rogpeppefwereade__: it's got other stuff too. AuthorizedKeys isn't about getting a machine agent running, for example.12:32
jammgz: poke12:32
rogpeppefwereade__: i think that using cloudinit.MachineConfig is just fine as part of the bootstrap process, but i don't see it as fundamental to the Environ interface12:33
fwereade__rogpeppe, surely that's only there because nobody bothered to do it properly? :)12:33
rogpeppefwereade__: which i still think is a reasonable abstraction12:33
fwereade__rogpeppe, what if MachineConfig were to be defined somewhere else?12:35
rogpeppefwereade__: and i think that taking the guts of ec2.environ.Bootstrap and putting into a new function in environs, specialised for starting a cloudinit-oriented environ, is a reasonable way forward12:35
rogpeppefwereade__: rather than making Environ.Bootstrap more specific and meaning that providers that don't fit the mould will need to simulate stuff in order to "pretend" to start instances etc12:36
mgzjam: hey, you're back?12:37
jammgz: yep12:37
jamcare to do a 1:1 mumbling?12:37
fwereade__rogpeppe, why do you think MachineConfig is connected with cloudinit? is there anything other than the simple accident of placement?12:38
rogpeppefwereade__: because it was fashioned entirely to hold the things that cloudinit needs to do its job.12:38
fwereade__rogpeppe, that job being..?12:38
rogpeppefwereade__: there's no point in passing half of that stuff into the Environ, because it already knows it12:39
rogpeppefwereade__: the fact that the data dir is in /usr/lib/juju is something that the Environ should decide, not the caller of the environ.12:40
rogpeppefwereade__: (for example)12:40
fwereade__rogpeppe, yes... some fields are filled in by provider-specific code, and some by non-provider-specific code12:40
dimiternfwereade__: just to double check - in order to handle config stuff properly in SetCharm, these are all related: http://paste.ubuntu.com/5564588/12:41
rogpeppefwereade__: i would expect every parameter to Environ methods to be filled in a non-provider-specific code12:41
rogpeppes/in a non/in by non/12:41
fwereade__rogpeppe, like yuo did with stateinfo and tools, eh?12:41
rogpeppefwereade__: those types *are* filled in by code external to the environ, no?12:42
fwereade__rogpeppe, they're a crazy frankenstein patchwork of could-come-from-anywhere AFAICT12:43
rogpeppefwereade__: at least they each represent a coherent resource (the state we need to connect to; the tools we want to use)12:43
fwereade__rogpeppe, both of which are totally meaningless and inappropriate from the caller's POV12:44
fwereade__rogpeppe, how did tools get into the interface in the first place?12:44
rogpeppefwereade__: i thought the process of finding the tools could be portable.12:45
rogpeppefwereade__: but i think you're right that it should probably be internal to Environ12:46
rogpeppefwereade__: and perhaps the same with state.Info as passed to Environ.StartInstance12:46
fwereade__rogpeppe, it could never be portable12:47
rogpeppefwereade__: no?12:47
rogpeppefwereade__: perhaps not, but it seemed not too far from a possibility to me12:47
dimiternfwereade__: have a look pls when you have time12:47
* dimitern lunch12:47
fwereade__rogpeppe, it always depended on the arch, which could only ever be determined after contrsint handling12:47
fwereade__dimitern, will do, cheers12:48
rogpeppefwereade__: anyway, all of that is making the Environ interface less, not more, specific. but your proposal makes it much *more* specific.12:48
rogpeppefwereade__: yeah, all of this stuff was always going to change with the advent of constraints.12:48
fwereade__rogpeppe, it makes explicit the fact that you need to run a machine agent on an instance, and it takes the non-provider-related job of configuring that machine agent out of the hands of the environ12:50
rogpeppefwereade__: there's nothing explicitly mentioned about a machine agent in MachineConfig, is there?12:51
fwereade__rogpeppe, ISTM pretty clear that a MachineConfig holds exactly the information necessary to get a machine agent running12:52
fwereade__rogpeppe, please explain again the distinction you are seeing between a machine config and a ...MachineConfig ;)12:52
rogpeppefwereade__: there's lots more in there - StateServerCert/Key, StateServer, MongoPort, APIPort, ProviderType, MongoURL, AuthorizedKeys, Config12:54
fwereade__rogpeppe, how do you propose to run a machine agent without state to run it against?12:54
rogpeppefwereade__: maybe the state is available elsewhere12:55
rogpeppefwereade__: that's up to the provider, surely?12:55
fwereade__rogpeppe, in that case it doesn't need to be specified... and indeed it is not specified12:55
rogpeppefwereade__: what if someone passes a MachineConfig to the local provider's LaunchInstance with StateServer=true ?12:56
fwereade__rogpeppe, what it they do?12:57
rogpeppefwereade__: i'm interested though: what objection do you have to factoring out the ec2-like bootstrap stuff in a similar way to how the cloudinit stuff was factored out before?12:58
fwereade__rogpeppe, who's going to be doing that in the first place?12:58
rogpeppefwereade__: the (new) Environ interface looks like it's a viable possibility12:58
rogpeppefwereade__: i'd like to keep environs.Environ as simple as possible, and i think we can do that without loss of generality12:59
rogpeppefwereade__: and as a bonus that means that simpler providers don't need to mess with any of that stuff.13:00
fwereade__rogpeppe, so what of the stuff that I do in bootstrap do you believe to be optional?13:00
rogpeppefwereade__: LaunchInstance, for one13:01
fwereade__rogpeppe, it's basically RunMachineAgent, though, right?13:01
rogpeppefwereade__: upload tools also13:01
rogpeppefwereade__: no, it's more than that. running a machine agent is considerably more simple.13:02
fwereade__rogpeppe, er, you made upload-tools a command line param, and part of the interface of environ in the first place13:02
fwereade__rogpeppe, how do you imagine that to be optional?13:02
rogpeppefwereade__: there's nothing that states that the tools need to be uploaded to the environ's Storage AFAICS. the local provider could simply copy the binaries to the new container, for example.13:03
fwereade__rogpeppe, running a machine agents *once you have all the requirements in place* is simple13:03
fwereade__rogpeppe, why would we fuck up the local provider like that? because more code paths are a Good Thing?13:04
fwereade__rogpeppe, because it's Fun to implement providers that randomly break when you want to upgrade?13:05
rogpeppefwereade__: because copying something to the storage just to copy it out again in the next line isn't very useful?13:06
rogpeppefwereade__: the only code path that depends on the tools being available in the storage is the shell script in cloudinit13:07
rogpeppefwereade__: apart from later when we upgrade13:07
rogpeppefwereade__: which is different.13:08
fwereade__rogpeppe, oh really?13:08
rogpeppefwereade__: yeah, i think so13:08
=== gary_poster|away is now known as gary_poster
rogpeppefwereade__: istm that this might be better in G+13:08
fwereade__rogpeppe, yeah, maybe, just a mo, not sure if lunch is coming13:09
jammgz: apparently hdmi really is needed :)13:11
fwereade__rogpeppe, https://plus.google.com/hangouts/_/f13861a7f65b5a679aa07c1d94088d17c4e8330b?authuser=0&hl=en13:13
benjibac: let me know when you have a minute to sync up13:26
bacbenji: ok, how about in 5 minutes?13:26
benjibac: sounds great13:26
bacbenji: i'll make a hangout13:37
benjirogpeppe: I am getting test failures running "go test ./state/api/" on trunk; is that a know issue?14:09
benjiin fact, trunk doesn't look like it even builds: worker/firewaller/firewaller.go:319: undefined: state.IsNotAssigned14:13
rogpeppebenji: on a call, sorry, will be with you in a bit14:24
benjirogpeppe: thanks14:24
=== wedgwood_away is now known as wedgwood
rogpeppebenji: out of call finally. trunk at least compiles all tests for me currently. just checking that all tests run ok.15:22
benjirogpeppe: I blew away my entire launchpad.net tree and re-got it and it works now <shrug>15:23
benjiperhaps I am missing some part of the dev process; is there some sort of re-fetch-and-build-all-dependencies command that people use?15:23
rogpeppebenji: hmm, i wonder what the problem was. in future, you might try blowing away just $GOPATH/pkg instead and see if that makes a difference15:23
benjik15:23
rogpeppebenji: not really. the only thing i usually have to update is launchpad.net/goose15:24
benjihow do you go about doing that?15:24
rogpeppebenji: other than that, all dependency management *should* be done by the go tool15:24
rogpeppebenji: go get -u launchpad.net/goose15:24
benjik, thanks15:24
rogpeppebenji: i update it only when some openstack stuff starts failing15:25
benjik15:25
fwereade__dimitern, ping15:26
dimiternfwereade__: pong15:33
dimiternfwereade__: I decided to propose what we discussed in the mean time - split the uniter tests15:33
dimiternfwereade__: will be ready shortly15:35
fwereade__dimitern, awesome, just going t have a really quick piece of toast15:40
fwereade__dimitern, hey, sorry, actually here now15:55
dimiternfwereade__: np15:55
fwereade__dimitern, starting a hangout15:56
dimiternfwereade__: cool15:56
fwereade__dimitern, https://plus.google.com/hangouts/_/4e9d42088fdb90f3cdce77923346ef9c1a62a89e?authuser=0&hl=en15:56
fwereade__rogpeppe, don;t suppose I can interest you in a quick review of https://codereview.appspot.com/7396056/ ?16:50
rogpeppefwereade__: looking16:51
dimiternfwereade__: there it is https://codereview.appspot.com/7378066/16:55
dimitern:)16:55
fwereade__dimitern, cheers16:55
fwereade__dimitern, that's awesome, LGTM16:56
fwereade__dimitern, do they get any faster? :)16:56
dimiternfwereade__: thanks!16:56
dimiternfwereade__: haven't check, but if they did is not much noticeable16:57
dimiternfwereade__: I'll run them now to compare with older times16:57
fwereade__dimitern, I think it ought to save a couple of seconds at least, but maybe that's all eaten up in extra test setup/teardown16:58
rogpeppefwereade__: reviewed16:59
dimiternfwereade__: nah.. probably they're *slightly* slower due to extra call overhead16:59
dimiternfwereade__: WOW they're actually faster!16:59
fwereade__dimitern, wanna know why?17:00
dimiternfwereade__: old time: 149.123 s, new time: 102.571s17:00
fwereade__dimitern, waitHooks is O(len(testlog))17:00
dimiternfwereade__: yes!17:00
dimiternfwereade__: oh, I see17:00
fwereade__dimitern, my money's on that, anyway17:00
dimitern:)17:00
dimiternfwereade__: can we consider this trivial with one LGTM17:03
fwereade__dimitern, if they're all passing, yeah, I think so -- no logic changes, so go for it :)17:05
dimiternfwereade__: yeah, all passing17:12
dimiternfwereade__: cheers17:12
rogpeppefwereade__: if len(testlog) is a significant contributor to the test runtime, we should fix it. it surely wouldn't be hard.17:18
dimiternTheMue: ty17:19
fwereade__rogpeppe, it only became so gradually as the test table got seriously long17:19
rogpeppefwereade__: yeah, maybe it's insignifant again.17:19
rogpeppeicant17:19
fwereade__rogpeppe, I'm not sayng it *shouldn't* be fixed, but I think the bulk of its contribution has been wiped out17:20
fwereade__rogpeppe, btw, good catch on init-time version.Current; ty17:20
rogpeppefwereade__: np17:20
rogpeppefwereade__: i thought uniter spent all its time making changes to mongo and waiting for changes.17:21
rogpeppefwereade__: (the uniter tests, that is)17:21
fwereade__rogpeppe, I had a closer look the other day and only then spotted that the testlog-scanning had become significant17:22
fwereade__rogpeppe, and was starting to consider rewriting the whole lot as separate test cases but didn't quite have the energy ;p17:22
rogpeppefwereade__: you could easily avoid using the whole test log by pre-filtering for particular log messages, no?17:22
rogpeppefwereade__: (i remember doing something similar once)17:23
fwereade__rogpeppe, a target that collects the ones I care about and stashes them somewhere? yeah, might be nice17:24
rogpeppefwereade__: yeah. a simple prefix regexp worked ok for me.17:24
fwereade__rogpeppe, so long as I don;t disturb the rest of the log, which does actually contain useful stuff sometimes17:24
rogpeppefwereade__: yeah - i logged as well as stashing17:25
fwereade__rogpeppe, cool, didn't think of that at the time17:25
rogpeppefwereade__: for our copious spare time :-)17:25
fwereade__rogpeppe, indeed :)17:26
dimiterni'm off guys17:29
dimiternhave a good evening17:30
fwereade__dimitern, gn, take care17:30
rogpeppefwereade__: a small CL that enables cleaning up of watchers in the other one: https://codereview.appspot.com/739805017:46
rogpeppefwereade__: perhaps "Abort" might be better than Kill, but i thought it's similar enough to tomb's Kill that the same name is probably appropriate.17:47
rogpepperight, that's me for the day. see y'all tomorrow.18:42
=== wedgwood is now known as wedgwood_away
=== wedgwood_away is now known as wedgwood
* thumper stabs go in the face20:56
thumpergo not having any form of real inheritance sucks monkey balls20:57
=== _thumper_ is now known as thumper
thumpermorning davecheney22:21
thumperdavecheney: why does go make it so hard to do something like virtual dispatch?22:21
davecheneythumper: do you want the long answer or the short answer22:26
thumperdavecheney: I want to know how to do what I want to do22:26
thumperas each thing I try, I get stuck by go's limitations22:27
davecheneythumper: not limitations, design decisions22:27
thumper:)22:27
thumperdavecheney: so... lets see if you can help me out22:27
thumperright now in the cmd package, there is a help method on the Info object22:28
thumpers/object/struct22:28
davecheneymmm22:28
thumperand a hard coded function call when needing help that takes (c *Command) -> c.Info().Help(f) where f aresome flags22:28
thumperwhat I want however is to override that behaviour when c is a SuperCommand22:28
thumperand I feel that looking at the underlying type is a horrible crutch22:29
thumperand was trying to find a better way22:29
thumperincluding having a "Base struct member" like CommandBase22:29
thumperhowever that doesn't work22:29
thumperas it can't call something it doesn't know about22:29
davecheneyi think the problem is we're trying to make SuperCommand a subclass of Command22:30
thumperhmm...22:30
thumperperhaps22:30
davecheneythe first thing that comes to mind is makeing Command an interface that both22:30
davecheneysubcommand and supercommand satisfy22:31
thumperright... I started down that path22:31
thumperactually, that is how it works right now22:32
thumperbut there is no concept of a "subcommand" as an entity in its own right22:32
thumperand I don't want to have to implement default behaviour in every subcommand22:33
thumperwhat I really want is some equivalent to the "pure virtual method" issue with an intermediate default implementation22:34
thumpersurely go has some answer to that design problem22:34
thumperI just don't know what it is22:34
davecheneyembedding22:36
davecheneyit is also that go does not have an answer for this22:36
davecheneythe authors made some deliberate decisions to not include type inheritence or function polymorphism22:36
davecheneyso sometimes it isn't possible to directly transpose from one language to another22:37
thumperdavecheney: I tried embedding, but it fails in one critical aspect22:44
thumperdavecheney: there isn't a way to define a method that doesn't yet exist22:45
thumperdavecheney: where you can get the result from a more-derived class22:45
thumperdavecheney: and you can't have a method on an interface, nor can you have function overloading22:45
davecheneyyeah, there is only composition, not virtual dispatch22:45
thumperdo I'm stuck trying to work out a solution for the problem in go22:46
davecheneywut "you can't have a method on an interface" ?22:46
thumperthat is trivial in several other languages22:46
thumperand I'm venting22:46
thumperas the target of the method22:46
thumperlike22:46
* davecheney pats thumper on the shoulder22:46
thumperfunc (i *SomeInterface) Foo() {}22:46
thumperwhich would work fine if it worked22:46
davecheneynup, can't do that, intefaces only define the contract, not satisfy them22:46
thumperright22:47
thumperhmm... time to collect my daughter for lunch22:48
thumperbbl22:48
davecheneythumper: or, you could use the type assertion and move on with your life :)22:48
thumperI may just do that, bit it feels horrible!22:49
thumpergoes against a whole lot of my professional life22:49
thumperit is like saying, "just shoot that person, and get on with your life"22:49
thumperI've spent 20 years learning not to do that22:49
davecheneysure22:49
davecheneythen lets fix it properly22:49

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!