/srv/irclogs.ubuntu.com/2013/05/24/#juju-dev.txt

=== gary_poster|away is now known as gary_poster
niemeyerthumper: Yo01:38
thumperniemeyer: oh hai01:39
thumperI had a go question...01:39
thumperif you have a select and each of the cases are waiting on a channel01:40
thumperwhat is happening under the covers?01:40
thumperand that select is inside a for { }01:40
thumperlike the provisioner01:40
niemeyerthumper: An epoll01:40
niemeyerthumper: Not sure if that's your question, though?01:41
thumperniemeyer: I was just wanting to make sure it wasn't doing stupid looping when nothing is happening01:41
thumperI considered wrting a test app to check, but got busy thinking of something real01:42
niemeyerthumper: Ah, no, it will block01:42
niemeyerthumper: Unless you have a default in the select loop.. then it would spin like crazy01:42
* thumper nods01:42
niemeyers/select loop/select clause/01:43
wallyworld_thumper: if you have a moment at some point, https://code.launchpad.net/~wallyworld/juju-core/create-machine-command/+merge/16551802:19
thumperwallyworld_: sure02:19
wallyworld_thumper: i didn't update machine state to include a Clean attribute. still not sure if we want that or not02:20
thumperwallyworld_: I'm about to email the group about my proposed machine state changes as there are a lot of them02:20
wallyworld_i'm thinking if we do add it, we can change the default assignment policy02:21
wallyworld_from AssignNew to AssignUnused02:21
wallyworld_since we can look for a "clean" machine02:22
thumperright...02:22
thumperthere is more to it though :)02:22
thumperwhich I'll be putting in the email02:22
wallyworld_sure, containers complicate things02:22
wallyworld_what i say above gives us the same thing as we have now02:23
wallyworld_since without it, create-machine just wastes resources02:23
thumperwallyworld_: well, you sould be able to use --force-machine02:23
thumperwallyworld_: does that work?02:23
wallyworld_ah,true02:23
wallyworld_should work02:24
wallyworld_the machine is created, spun up, and is just sitting there02:24
wallyworld_with no principal units or anything running02:24
wallyworld_i'll check it out02:24
wallyworld_thumper: all works as expected :-)02:34
thumperwallyworld_: good02:46
wallyworld_thumper: i think juju status should print out the constraints for the machine02:48
wallyworld_right now it doesn't seem to do that02:48
thumperperhaps a flag02:48
thumperwe are getting too much info by default02:48
thumpermuch more than is usually useful02:49
wallyworld_you think?02:49
wallyworld_i sorta like what's there now02:49
wallyworld_there's only 5 lines for eacdfh machine02:50
wallyworld_adding one more for constraints doesn't seem too bad02:50
=== wedgwood is now known as wedgwood_away
thumperwell, that is me about done for the day05:25
rogpeppe1mornin' all06:32
kyhwanaevening06:34
dimiternmorning06:51
rogpeppe1kyhwana: :-)06:51
rogpeppe1dimitern: hiya06:51
dimiternrogpeppe1: hey06:53
dimiternrogpeppe1: i couldn't finish yesterday because just as i was nearing the end and only a few more test cases were needed, i started getting test failures or panics when running the apiserver tests; but not all the time, like 1 in 10 cases06:54
rogpeppe1dimitern: hmm, that's not entirely surprising for that kind of logic.06:54
rogpeppe1dimitern: have you run it with race detection on?06:55
dimiternrogpeppe1: yes, multiple times - no races06:55
rogpeppe1dimitern: well, not a memory race anyway, eh? :-)06:56
dimiternrogpeppe1: so there are 2 kinds of failures - timeouts for the watchers (always the same - next() returns a stopped error before stopping either the server or the watcher - and in the logs i can see it is indeed getting stopped somehow)06:56
rogpeppe1dimitern: is that one kind of failure or two?06:58
dimiternrogpeppe1: another issue is the pretty long and somehow not very legible panic, usually mentioning PutCharm some 50 frames down and having mgo stuff close to the top06:58
rogpeppe1dimitern: could you paste the panic here?06:58
dimiternrogpeppe1: running them again06:59
dimiternrogpeppe1: http://paste.ubuntu.com/5696111/07:01
rogpeppe1dimitern: interesting, looks like a runtime bug07:01
rogpeppe1dimitern: you can reproduce this ok?07:01
dimiternrogpeppe1: trying more times07:02
dimiternrogpeppe1: how can you tell is likely a runtime bug?07:03
rogpeppe1dimitern: it's dying during garbage collection07:03
dimiternrogpeppe1: wow :) these kind of things always happen to me :D07:04
rogpeppe1dimitern: so it seems.07:04
rogpeppe1dimitern: please could you try and see if you can reproduce the behaviour using gustavo's new Go-only yaml implementation07:05
dimiternrogpeppe1: it happened again - almost the same, but trace order is changed a bit07:05
dimiternrogpeppe1: http://paste.ubuntu.com/5696122/07:06
dimiternrogpeppe1: so how to get it?07:06
rogpeppe1dimitern: i'm just looking for it07:08
dimiternrogpeppe1: hmm now it happens on every run - yesterday happened less often.. also it doesn't happen if I run individual tests07:10
rogpeppe1dimitern: https://code.launchpad.net/~niemeyer/goyaml/go-port07:10
dimiternrogpeppe1: so what - delete the goyaml in gopath and get this instead?07:11
rogpeppe1dimitern: yeah - although you'll need to manually branch it into launchpad.net/goyaml07:12
dimiternrogpeppe1: or just remove the files and replace them07:12
rogpeppe1dimitern: yeah, that's probably easier07:12
TheMuemorning07:14
rogpeppe1dimitern: fairly trivial CL: https://codereview.appspot.com/968104507:14
rogpeppe1TheMue:  hiya07:14
dimiternTheMue: hey07:15
dimiternrogpeppe1: looking07:15
dimiternrogpeppe1: LGTM07:17
rogpeppe1dimitern: ta07:17
=== rogpeppe1 is now known as rogpeppe
dimiternrogpeppe: ha! i happened at first with goyaml-go, but wasn't sure it's clean, so I went in pkg/linux_*/launchpad.net/ and deleted goyaml.a07:18
dimiternrogpeppe: now there's no panic, just some failures07:18
rogpeppedimitern: hmm. keep trying07:19
dimiternrogpeppe: http://paste.ubuntu.com/5696153/07:19
rogpeppedimitern: it may still be a runtime bug07:20
rogpeppedimitern: just that you're not triggering it any more for some reason07:20
dimiternrogpeppe: oops there is it again: http://paste.ubuntu.com/5696157/07:20
rogpeppedimitern: because i'm not sure the code path you're testing uses yaml much07:20
rogpeppedimitern: ah, good. (well, kinda :-])07:21
rogpeppedimitern: could you push the branch - i'll see if i can reproduce07:21
dimiternrogpeppe: sure07:22
dimiternrogpeppe: ha! now all passed07:22
rogpeppedimitern: run it in a loop07:22
TheMue*: any probs with the current goyaml?07:23
dimiternrogpeppe: lp:~dimitern/juju-core/041-provisioner-api-calls07:23
dimiternrogpeppe: how do you mean in a loop?07:23
dimiternrogpeppe: it seems it happens in the TestOperationPerm ... except now it happened later + a failure: http://paste.ubuntu.com/5696171/07:25
rogpeppedimitern: while true; do go test; done07:26
dimiternrogpeppe: running07:28
rogpeppedimitern: i've just reproduced it07:31
dimiternrogpeppe: cool07:31
dimiternrogpeppe: w/o goyaml-go?07:32
rogpeppedimitern: no - i'll just try with non-cgo goyaml07:33
dimiternrogpeppe: ok, if it's indeed a runtime bug now begins the tedious hunting for a minimal case to reproduce i guess..07:34
rogpeppedimitern: yeah.07:34
dimiternrogpeppe: while you're looking at this, i'll finish the rest of the tests, so hopefully if there's a way around the panic it'll be ready for proposing07:41
dimiternrogpeppe: if you don't mind07:41
rogpeppedimitern: you could work against 1.0.3 for the time being07:42
dimiternrogpeppe: does it work like that?07:42
rogpeppedimitern: it certainly should do07:42
rogpeppedimitern: we're trying to keep 1.0.3 compatibility after all07:43
rogpeppedimitern: if it doesn't, it's a bug07:43
dimiternrogpeppe: i'm getting 1.0.3 again to try07:43
rogpeppedimitern: if you're using go from source, you can just do hg update go1.0.307:43
rogpeppedimitern: then run all.bash07:44
dimiternrogpeppe: I'll put it in a separate dir and just wipe out the gopath/pkg/*, i messed it up last time i tried to setup multiple go versions + gopaths so i can switch between them easily, i'll be more careful this time07:45
rogpeppedimitern: i have an entirely separate tree for working with separate go versions07:45
rogpeppedimitern: (which is actually very useful because i can be testing against one branch while developing another)07:46
dimiternrogpeppe: well, I tried having ~/go/<ver> for goroot, and ~/work/go/<ver> for gopath, but then it didn't work with emacs somehow after the raring upgrade07:46
dimiternrogpeppe: (come to think about it, it still doesn't work - i need to run emacs from a terminal rather than from the launcher, otherwise the gopath/root i set in .bashrc are not visible07:47
rogpeppedimitern: that's not surprising. emacs won't run .bashrc07:47
rogpeppedimitern: did you try putting the settings in your .profile ?07:48
dimiternrogpeppe: but it worked in quantal! now it cannot find gofmt etc. unless run from the shell07:48
dimiternrogpeppe: i tried .profile, .Xsessionrc, /etc/profile, everything.. no joy07:48
rogpeppedimitern: try removing it from your .bashrc, adding it to your .profile, logging out and logging in again07:49
rogpeppedimitern: .bash_profile might work too07:50
dimiternrogpeppe: tried that as well, just emacs somehow defies all known ways it should cooperate with my environment :)07:50
rogpeppedimitern: i'd be surprised. this is an environment-variable/shell issue, not emacs, i think07:50
dimiternrogpeppe: well, that very well maybe, but haven't seen a problem with another app so far07:51
rogpeppedimitern: that's probably because no other app needs $GOPATH and $GOROOT07:51
rogpeppedimitern: or relies on any setting in your .bashrc, i suspect07:52
dimiternrogpeppe: btw - if i just rm -fr $GOPATH/pkg/ and then run should work, or I should leave pkg empty?07:52
rogpeppedimitern: that should work fine.07:52
rogpeppedimitern: directories are created on demand07:52
dimiternrogpeppe: hmm apparently goyaml-go uses go1.1 features (function ends without a return statement)07:55
dimiternrogpeppe: switching to goyaml-c07:55
rogpeppedimitern: ha07:55
dimiternwallyworld_: ping07:59
wallyworld_hi07:59
dimiternwallyworld_: hey, looking at create-machine08:00
wallyworld_thanjs08:00
dimiternwallyworld_: why do we need this at all? why not have an argument to deploy instead? don't we want it to be atomic?08:00
wallyworld_it's part of the containerisation work08:00
wallyworld_maybe read tim's email08:01
dimiternwallyworld_: i know; i read it08:01
dimiternwallyworld_: but still now convinced it's a good idea to have a separate command08:01
wallyworld_the next step is to allow containers to be created inside a machine08:01
dimiternwallyworld_: i see that, ok, but still - using 2 steps to deploy inside a container seems wrong08:02
wallyworld_ultimately the common operation ie deploy will be atomic. this is a step along the way08:04
dimiternrogpeppe: with go 1.0.3 i see no panics so far, but still some tests failures with the watchers (again, not on every run)08:04
wallyworld_just constructing the jigsaw peices08:04
dimiternwallyworld_: so you're saying create-machine isn't there to stay?08:04
wallyworld_not sure yet08:04
wallyworld_all up for prototyping and discussion08:05
wallyworld_will aid in testing during the development etc08:05
wallyworld_plus it ma turn out to be useful longer term, not sure yet08:05
dimiternwallyworld_: yeah, that's just the thing - adding stuff we're not sure we need right now will surely bend the direction in the future to finding ways to use them08:06
dimiternwallyworld_: anyway, just thinking out loud - will take my concerns to tim's email :)08:07
wallyworld_fair questions. i started this work before the email. it's all very much in a state of flux08:08
wallyworld_but as a developer, the ability of create machines/containers will be very usrful08:08
dimiternwallyworld_: sure when starting on a big new feature there's always uncertainty where to begin08:08
wallyworld_eg may want to spin up a few containers ahead of time08:09
dimiternwallyworld_: i agree, but i doubt a command is the necessary for that08:09
wallyworld_and then scale out later08:09
wallyworld_how else would it be done if there were no command to so it08:09
dimiternwallyworld_: you see "juju's way" of doing things afaiui is telling it what you need and letting it do it for you08:10
dimiternwallyworld_: what mark (sabdfl) said scaling out a service + containers could be automated08:10
wallyworld_sure. but having some manual control would be nice too08:11
dimiternwallyworld_: having a service deployed on 1 machine + some containers and this is 1 unit, adding a unit could replicate that and the containers08:11
dimiternwallyworld_: could be, not sure yet, i think it's worth some careful consideration08:12
wallyworld_think of it like being able to add an existing running machine/vm to the env08:12
wallyworld_it's just another tool in the kit08:13
dimiternwallyworld_: you could do that with manual provisioning (eventually), but this is just a special case - why complicate things adding more special cases like that?08:13
rogpeppedimitern: https://code.google.com/p/go/issues/detail?id=555408:14
wallyworld_dimitern: for me it's a useful tool. ymmv. the command bit is a small portion of the logic. i like having manual control08:16
dimiternrogpeppe: cool! thanks for filing a bug08:17
rogpeppedimitern: i'm currently testing against tip. it's possible the problem is already fixed.08:18
dimiternrogpeppe: so with go 1.0.3 more often than not tests pass (1 in 5 failure so far)08:18
rogpeppedimitern: i'm sure there's a problem you need to fix too :-)08:19
rogpeppedimitern: i'm not looking into it any further for the moment, because there's lots of stuff i want to get proposed today before i go away next week08:19
dimiternrogpeppe: it'll be helpful if i manage somehow to reproduce it consistently08:19
dimiternrogpeppe: sure08:19
dimiternrogpeppe: but I want to propose a split of apiserver.go into multiple submodules08:20
rogpeppedimitern: different packages?08:20
rogpeppedimitern: if you're just talking about splitting the file, +108:21
dimiternrogpeppe: the same package state/apiserver/ but having things like root.go, client.go, machine.go, etc.08:21
dimiternrogpeppe: ok, i'll do it after i finish this08:21
rogpeppedimitern: i'd name all the ones implementing the actual api to start with "api"08:21
rogpeppedimitern: e.g. apiroot.go, apiclient.go, apimachine.go08:22
rogpeppedimitern: so it's easy to see which files implement the core logic08:22
dimiternrogpeppe: also i'd like to split api_test.go into perm_test.go, machine_test.go, client_test.go, etc.08:22
rogpeppedimitern: seems reasonable08:23
dimiternrogpeppe: i'm fine with api*.go08:23
dimiternrogpeppe: cool08:23
rogpeppedimitern: if you do it, make it a branch which is entirely mechanical - no code changes at all, please.08:23
rogpeppedimitern: i'm not seeing any panics against tip, BTW08:24
dimiternrogpeppe: sure, but after i'm done with this; don't want to make it harder for me to merge later08:24
rogpeppedimitern: yeah08:24
dimiternrogpeppe: will it impede the run time too much if we split agent and client perm tests into 2 files and 2 tables?08:25
rogpeppedimitern: i'm not sure i see the point08:26
rogpeppedimitern: the point of those tests is to test permissions against the entire API surface area08:27
rogpeppedimitern: you could split the table, i suppose08:28
dimiternrogpeppe: hmm, ok - but only perm checks should be in there; error conversion tests, bad logins, etc should be separate08:28
rogpeppedimitern: agreed08:28
rogpeppedimitern: i have some vaguely formed thoughts about how we can make the perm checks much smaller and faster08:28
dimiternrogpeppe: oh?08:30
dimiternrogpeppe: (while i'm at splitting stuff i might as well split apiclient to multiple files as well)08:30
rogpeppedimitern: that's a fair bit smaller - i wonder if it might be better just as two files - one for the client api and one for the agent api08:32
rogpeppedimitern: or actually08:32
rogpeppedimitern: it's perhaps good to have a 1-1 file mapping between apiserver and api08:32
dimiternrogpeppe: exactly my point08:32
rogpeppedimitern: for the pieces implementing the api08:32
dimiternrogpeppe: good, will do then08:32
rogpeppedimitern: i've run the apiserver tests 17 times so far against tip and no panic08:35
dimiternrogpeppe: good! so it seems fixed08:36
rogpeppedimitern: or the bug just isn't being tickled any more08:36
dimiternrogpeppe: well, it might also mean someone will recognize it and say it's fixed08:37
rogpeppedimitern: yeah. i'm hoping someone will say "ah, we fixed it then", and then they'll issue a new point release08:38
rogpeppewallyworld_: ping08:38
wallyworld_rogpeppe: hi, i'm just about to leave for soccer08:39
rogpeppewallyworld_: v quick question08:39
wallyworld_ok08:39
rogpeppewallyworld_: in environs/cloudinit, you did syslogConfigRenderer, right?08:39
wallyworld_i think so, let me check08:39
rogpeppewallyworld_: is there a particular reason why the config is rendered after bootstrap-state is called?08:40
rogpeppewallyworld_: because i'm thinking about moving it earlier in the script08:40
wallyworld_rogpeppe: i think it will be ok to move. i think it was put where it is because that's where some previous logging stuff was initialised08:41
wallyworld_but i'm not entirely sure08:41
rogpeppewallyworld_: ok, cool08:41
wallyworld_all it does anyway is write a conf file08:41
wallyworld_so that can go anywhere really08:41
rogpeppewallyworld_: that's what i thought; just wanted to check there wasn't anything subtle going on.08:42
rogpeppewallyworld_: thanks08:42
wallyworld_np.08:42
=== ehg_ is now known as ehg
dimiternrogpeppe:09:12
rogpeppedimitern:09:12
dimiternrogpeppe: oops, I have a question :)09:12
rogpeppedimitern: go on09:13
dimiternrogpeppe: in this log: http://paste.ubuntu.com/5696381/09:13
dimiternrogpeppe: is it normal to have an Error response to the Stop request?09:14
rogpeppedimitern: i don't see that you are09:15
rogpeppedimitern: which line are you thinking is an error response to the stop request?09:15
rogpeppedimitern: i am seeing that i need to sort out the rpc logging - it's logging each message twice09:16
dimiternrogpeppe: ah, right - [LOG] 94.86663 DEBUG rpc/jsoncodec: -> {"RequestId":4,"Error":"watcher has been stopped","ErrorCode":"stopped","Response":{}}09:16
dimiternthis is the Next response09:16
rogpeppedimitern: (i know why)09:16
rogpeppedimitern: that's a response to a Next09:17
rogpeppedimitern: note the RequestId09:17
dimiternrogpeppe: yeah09:17
dimiternrogpeppe: i think we have a problem with the current watcher implementation in the client, but only the environ config one fails because it needs to do extra work on marshaling/unmarshaling the AllAttrs when receiving a response09:18
dimiternrogpeppe: it seems next should quit as soon as stop was called09:18
dimiternrogpeppe: (the next goroutine in the client, that is)09:18
rogpeppedimitern: on the server side?09:18
rogpeppedimitern: why's that?09:19
dimiternrogpeppe: because otherwise we have this issue09:19
dimiternrogpeppe: and arguably it's an error to try reading a stopped watcher anyway09:19
rogpeppedimitern: perhaps you could try describing the issue and why it's happening09:19
dimiternrogpeppe: well, i *think* as soon as stop is called the next goroutine does not exit immediately and tries to read, getting the error, and then exits, but the error is dispatcher to the client outside09:20
dimiternrogpeppe: but really it shouldn't report that error if it knows for sure the server-side watcher was stopped09:21
rogpeppedimitern: isn't that what the ErrStillAlive check is about?09:22
dimiternrogpeppe: if it worked i shouldn't have received anything on the in channel, right?09:23
dimiternrogpeppe: when next encounters a stopped state09:23
rogpeppedimitern: ha, yes09:23
rogpeppedimitern: it should return after the Kill09:24
dimiternrogpeppe: ah, I think I see the error - after w.tomb.Kill(err) i should return rather than trying the select09:24
dimitern:)09:24
dimiternyeah09:24
dimiternrogpeppe: that fixed it09:26
rogpeppedimitern: cool09:26
dimiternrogpeppe: only one things bothers me - i think it's maybe worth panicking when i cannot unmarshal the environ config i got on the in channel, rather than logging an error09:28
rogpeppedimitern: i think the watcher should die if that happens09:28
dimiternrogpeppe: it does, but if we got that far something is very wrong, hence a panic is in order i think09:29
rogpeppedimitern: can't that happen if the server sends some dubious data back?09:29
dimiternrogpeppe: and where did it get it from? state - so it think it's still worth panicking09:30
rogpeppedimitern: definitely not09:30
dimiternrogpeppe: explain?09:30
rogpeppedimitern: in general we don't panic when an external data source produces something unexpected09:30
dimiternrogpeppe: hmm09:30
rogpeppedimitern: it might be talking to a version of the API with a subtly different interface09:30
dimiternrogpeppe: ok then, i'll leave just the error log09:31
dimiternrogpeppe: fair point09:31
rogpeppedimitern: you don't need to log the error - you can kill the watcher with an appropriate error09:31
rogpeppedimitern: then it will be picked up as usual by the rest of the logic09:31
rogpeppedimitern: causing the agent to die, or whatever09:31
dimiternrogpeppe: i'm killing it, and logging the error09:31
rogpeppedimitern: i'm not sure that the log is necessary, but i guess09:32
dimiternrogpeppe: it won't hurt when debugging incompatible api server/client connections, i think09:32
dimiternrogpeppe: btw you copied my branch when you filed the golang issue, right?09:33
rogpeppedimitern: yes09:36
dimiternrogpeppe: cool, i'm proposing it soon09:36
dimiternrogpeppe: so with the newly gained understanding of the api internals hopefully, i'll be able to help the other guys next week while you're gone09:46
rogpeppedimitern: that would be great, thanks09:46
rogpeppedimitern: another small CL: https://codereview.appspot.com/971004409:47
* dimitern looking09:47
dimiternrogpeppe: what's "bootstrap-state" ?09:50
rogpeppedimitern: jujud bootstrap-state09:50
rogpeppedimitern: the thing that initialises the mongo state09:50
rogpeppedimitern: and sets the initial password for the machine agent09:50
dimiternrogpeppe: i see (never heard of it before)09:50
dimiternrogpeppe: LGTM09:52
dimiternrogpeppe: do you think you'd manage adding the agent API connection stuff today?09:59
rogpeppedimitern: that's the aim09:59
rogpeppedimitern: i have a branch that does it10:00
rogpeppedimitern: but there are one or two branches that need to go in first10:00
dimiternrogpeppe: that will be awesome10:00
rogpeppedimitern: well, let's sat i have a branch that does it *in theory* :-)10:00
dimiternrogpeppe: i was thinking once i started the api stuff i might as well do most (if not all of it), so others can concentrate on replacing state calls with api calls next week, once we have the connection10:01
rogpeppedimitern: that sounds like a good plan10:01
rogpeppedimitern: BTW the API connection stuff involved refactoring the machine agent, and it's turned out really quite nice10:01
dimiternrogpeppe: good to hear10:02
rogpeppedimitern: did you look at the Runner CL?10:05
rogpeppedimitern: i don't think you reviewed it, which was a bit surprising :-)10:06
dimiternrogpeppe: this one? https://codereview.appspot.com/9643043/10:17
rogpeppedimitern: yeah10:18
dimiternrogpeppe: I looked at it briefly, but by the time I got the time to review it it already landed :)10:18
dimiternrogpeppe: https://codereview.appspot.com/9721043/ is ready10:18
dimiternrogpeppe: will look at yours once more more carefully10:19
rogpeppedimitern: np. more eyes always appreciated - it's a moderately complex piece of code.10:19
rogpeppedimitern: i'm slightly surprised you've done all those api calls in one CL. i thought they might work well as many separate ones.10:20
rogpeppedimitern: it's not a particular problem, but i might not get it reviewed today10:20
dimiternrogpeppe: they're closely related - all needed by the provisioner and really most are trivial10:21
dimiternrogpeppe: np, go as far as you can10:21
rogpeppedimitern: i guess i just prefer separable changes to be separate :-)10:22
dimiternrogpeppe: i guess watchers and machine calls can be separated10:22
rogpeppedimitern: that might be good, if you don't mind too much.10:22
dimiternrogpeppe: uhh.. you're right actually - just now glanced at the diff - it's in excess of 1000 lines10:23
rogpeppedimitern: yeah, it's pretty big10:23
dimiternrogpeppe: didn't look like that when i was doing it :) anyway - will try bzr pipes to see if i can split it easily10:24
rogpeppedimitern: you might not need pipes too much10:24
rogpeppedimitern: most of the changes are orthogonal10:24
rogpeppedimitern: if you do the commonWatcher thing first (making the EntityWatcher use it only) then all the rest are likely to be independent10:25
dimiternrogpeppe: good point, will do10:26
dimiternrogpeppe: ok, disregard that CL then, I'll propose it after splitting10:27
rogpeppedimitern: thanks10:28
* dimitern hates bzr diff!!!!10:34
mgz?10:35
mgzyou know it takes arguments, right?10:35
dimiternmgz: so i've been in this situation before10:35
dimiternmgz: i have trunk and a branch off it10:35
dimiternmgz: i need to see changes in a particular path between the two branches10:36
dimiternmgz: but none of the suggested args help10:36
dimiternmgz: tried --old trunk while on the new branch; tried -r-2, but i need a path as well, not the changes in the whole tree10:36
dimiternmgz: tried --old trunk --new feature-branch - still frustratingly no results!10:37
dimiternmgz: please help :)10:38
dimiternmgz: (if it makes a difference - both branches are clean - no uncommitted changes)10:39
mgzhm, I'd expect --old to work10:39
mgzhave you got qbzr intalled?10:40
dimiternmgz: well it doesn't; yes I have qbzr10:40
dimiternmgz: with qdiff is still the same (No changes)10:41
dimiternmgz: all i need here is diff trunk:tip and feature:tip and show me the changes in state/api/ only10:41
mgzso, it works for me with standalone branches10:42
dimiternmgz: so i can cherry pick some of them and split the feature into multiple smaller ones10:42
mgzbut something seems to be unhappy with native colo10:42
mgzbut, I'd expect if you cd ~ then do `bzr diff --old .../src/real/branch/location/trunk --new .../src/.../feature state/api` it'll work10:43
dimiternmgz: here's what bzr reports: http://paste.ubuntu.com/5696558/10:43
dimiternmgz: ah, i'll try that10:43
mgzright, so try --old ~/src/juju-core/trunk --new ~//src/juju-core/041-provisioner-api-calls10:44
mgzthere might be an issue with checkouts, I'll see if we have a bug filed10:45
dimiternmgz: hallelujah! it works like that10:45
dimiternmgz: although i'd expect it to work in a LW checkout as well..10:46
mgzlooks like bug 596785 plus bad location resolution code in the diff command10:47
_mup_Bug #596785: 'bzr diff --old|new PATH' does silently exists if arg does not exist <diff> <patch-needswork> <Bazaar:Confirmed> <https://launchpad.net/bugs/596785>10:47
mgzcan you file a new bug with your symptoms please dimitern?10:48
dimiternmgz: a new one or amend this10:48
mgza new one10:48
dimiternmgz: sure10:48
mgzthe location code being wrong I don't see any entry for, can do both issues in one merge possibly, but might make sense to do them seperately10:49
dimiternmgz: bzr diff --old/--new does not report and changes with a lightweight checkout of a native repo?10:51
dimiternmgz: as summary10:51
mgzs/and/any/ and seems reasonable10:51
mgzand just end after "checkout"10:52
dimiternmgz: not sure i get you - where's the "and" ?10:52
dimiternmgz: ah10:52
mgz:)10:53
dimiternmgz: bug 118377611:01
_mup_Bug #1183776: bzr diff --old/--new does not report and changes with a lightweight checkout of a native repo <amd64> <apport-bug> <bzr> <diff> <raring> <bzr (Ubuntu):New> <https://launchpad.net/bugs/1183776>11:01
mgzthanks!11:01
dimiternmgz: any idea how to make my life easier and cherry pick interactively what to merge? rather than having to manually edit the diff and run patch with it11:04
danilosdimitern, mgz: I need to step out to sign some documentation in the court, likely to not make it for the stand-up11:09
dimiterndanilos: ok11:09
rogpeppedimitern, TheMue: another small CL: https://codereview.appspot.com/972304311:19
mgzdimitern: use shelve11:20
dimiternrogpeppe: LGTM11:21
rogpeppedimitern: ta!11:21
dimiternmgz: i'll try11:21
dimiternmgz: how will shelve help with committed trees?11:22
mgzcherrypick the feature branch across to your new integration branch11:22
mgzthen set EDITOR and use shelve interactive to edit out the bits you don't want11:23
dimiternmgz: it seems way too complicated.. i'll just branch the old feature and vistually merge into the new one with directory comparison11:24
mgzit's two commands...11:24
mgzbut depending on how hard the changes are to unpick, interactive mode may be useful or not11:25
mgzoh, and I'm not actually sure what `bzr merge -i` does11:26
mgzwhat you want, it seems11:26
mgzso, you have feature, which you want to split out11:27
mgzdimitern: so, in your checkout, on trunk, `bzr switch -b prefeature` then `bzr merge -i .../feature`11:27
dimiternmgz: i'll try next time, thanks; spent too much time already fiddling around with tools, so i'm back to good old manual methods + meld for visual merging11:29
TheMuerogpeppe: done11:29
dimiternmgz, wallyworld_: are we doing the standup?11:34
mgzlets11:34
mgzcan just say hi at least :)11:34
dimiternrogpeppe, mgz: https://codereview.appspot.com/9667048 first part11:48
rogpeppedimitern: that's more like it :-) :-)11:48
dimiternTheMue: ^^ reviews appreciated11:52
dimiternthere are 3 more CLs coming in this pipeline11:52
TheMuedimitern: done11:58
dimiternTheMue: cheers11:58
dimiternTheMue: actually making them arguments to init() will make the code less readable11:59
dimiternTheMue: and I don't see an advantage in doing that12:00
dimiternTheMue: could you explain your reasoning?12:00
TheMuedimitern: having them as arguments would let you check if they are set. that's the only reason.12:04
dimiternTheMue: it won't compile anyway if they're not set12:04
dimiternTheMue: i'll add a check in init() to ensure they are set, but not as arguments12:07
TheMuedimitern: it won't compile?12:07
dimiternTheMue: no, you're right - it will compile, but panic at runtime12:09
dimiternTheMue: i'll add a check, thanks12:09
TheMuedimitern: you set them after init(), so a check there would fail.12:09
dimiternTheMue: i moved init after setting the,12:12
TheMuedimitern: so please also comment that those values have to be set before calling init().12:13
dimiternTheMue: sure12:13
TheMuedimitern: great, thx12:14
dimiternthanks guys, submitting with the changes12:19
dimiternrogpeppe, mgz, TheMue: next in line: https://codereview.appspot.com/971404412:51
dimiterndanilos: you as well? ^^13:12
rogpeppedimitern: you have a review13:35
dimiternrogpeppe: tyvm13:35
rogpeppedimitern: much better split into bits like this!13:35
dimiternTheMue: calling init() in commonLoop() is not quite the same, because it's needs to be called before the go routine spins up13:36
TheMuedimitern: due to the created channel?13:38
dimiternTheMue: yeah, that as well13:39
TheMuedimitern: ok, could lead to a race otherwise. what else?13:41
dimiternTheMue: actually I think this is it - couldn't find other reason13:42
TheMuedimitern: ok, fine13:46
=== wedgwood_away is now known as wedgwood
=== BradCrittenden is now known as bac
dimiternthanks for the reviews, submitting soon; 2 more to go14:00
dimiternrogpeppe: kanban14:05
dimiternrogpeppe, TheMue: last one about the provisioner: https://codereview.appspot.com/9708044/14:51
TheMue*click*14:54
TheMuedimitern: done15:05
dimiternTheMue: thanks!15:05
rogpeppedimitern: reviewed15:18
dimiternrogpeppe: cheers15:18
dimiternrogpeppe: we you're saying the agent shouldn't use SetMongoPassword at all, and use SetPassword instead?15:19
rogpeppe dimitern: yes15:20
dimiternrogpeppe: ok, even better15:20
rogpeppedimitern: the API server knows when SetMongoPassword is appropriate, and does it15:20
dimiternrogpeppe: it's getting smarter then :)15:21
rogpeppedimitern: that's kinda the idea of the API :-)15:21
dimiternrogpeppe: indeed15:22
dimiternrogpeppe: does the same apply for unit.SetMongoPassword?15:37
rogpeppedimitern: yes. hmm, that's an interesting point actually15:37
rogpeppedimitern: currently SetPassword should be doing SetMongoPassword for all agents15:37
dimiternrogpeppe: so for unit.SetMP i'll use the same logic  as for a machine15:38
rogpeppedimitern: yeah - i.e. it should always be set.15:39
rogpeppedimitern: presumably you've merge trunk?15:39
mrammanybody know how sync-tools works15:39
dimiternrogpeppe: ok15:39
rogpeppemerged?15:39
dimiternyeah15:39
rogpeppemramm: i've never used it15:39
dimiternrogpeppe: it's used to sync ec2 public tools to your private bucket15:39
mrammand have a minute to walk danwest through getting tools into MAAS?15:39
rogpeppedimitern: yeah i know roughly what it does15:40
rogpeppedimitern: but i've never actually *used* it15:41
dimiternrogpeppe: ops, sorry - that was for mramm actually15:41
dimiternrogpeppe: got too used to addressing you lately :)15:41
rogpeppedimitern: np - i do that too sometimes15:41
mrammcan you use sync-tools as an argument to bootstrap15:43
mrammand IIRC you need to put your EC2 credentials somewhere to make it work15:44
mrammis this documented anywhere that we can just point danwest too?15:44
dimiternmramm: no15:44
mramm:(15:44
dimiternramit's a separate command15:44
mrammok15:44
dimiternmramm: there was a mail from jam about how it works15:45
mrammso bootstrap and then sync tools?15:45
mrammok, will search for that15:45
dimiternrogpeppe: do you think unit tests for agents/workers in general should also only use the api?15:46
rogpeppedimitern: that's an interesting question.15:46
rogpeppedimitern: i'm not entirely sure it's possible15:46
rogpeppedimitern: you'd need to juggle at least two API connections and send the right commands to the right one15:47
dimiternrogpeppe: yeah15:47
dimiternrogpeppe: and if we leave them using state directly how bad will it be?15:47
rogpeppedimitern: i'm not sure that's bad at all15:47
rogpeppedimitern: the actual code being tested will need to use the API15:47
rogpeppedimitern: it's just the setup code which will talk directly to mongo (and probably some verification code too)15:48
dimiternrogpeppe: yeah, exactly, the tests can still access it directly15:48
rogpeppedimitern: yup15:48
dimiternrogpeppe: and btw this will save a *huge* amount of work15:48
rogpeppedimitern: yeah15:48
rogpeppei hadn't seen this error before: cannot make S3 control bucket: A conflicting conditional operation is currently in progress against this resource. Please try again.15:54
dimiternrogpeppe: it seems like an ec2 error15:54
rogpeppedimitern: yeah it is. just another one i hadn't seen before15:55
rogpeppedimitern: to join the collection of sporadic live test failures.15:55
rogpeppedimitern: getting live tests to pass reliably is gonna be really hard15:55
dimiternrogpeppe: yup, unfortunately15:55
danwestdimitern: mramm pointed me your way for juju-core/MAAS bootstrap issues15:56
dimiterndanwest: hm, ok - what do you need?15:58
danwestdimitern: new raring install with apt-get juju-core15:59
danwesttrying to bootstrap against MAAS provider and get the following:16:00
danwest~$ juju -v bootstrap16:00
danwest2013/05/24 11:27:45 INFO environs: reading tools with major version 116:00
danwest2013/05/24 11:27:45 INFO environs: falling back to public bucket16:00
danwest2013/05/24 11:27:45 ERROR command failed: no tools available16:00
danwesterror: no tools available16:00
dimiterndanwest: can you try juju-core from source and see if it helps?16:01
mrammI expect that nobody has tested the sync tools path with MAAS16:02
mrammit landed at the very last second for the raring cycle16:02
mrammI know I have not tested it myself :/16:02
dimiternmramm: me too, and maas landed at the last moment as well, so it might not work from the released version16:02
danwestdimitern: mramm: I can try from source but that might take me a couple to get setup16:03
mrammdimitern: dan is working on the "cloud installer" and any help we can give him would be good16:03
danwestthanks mramm16:03
mrammdimitern: we want to keep him unblocked ;)16:03
dimiternmramm: sure, i'll help if i can, but have never tried the maas provider16:04
mrammyea, but william is out16:04
dimiterndanwest: we fixed a bunch of issues with tools and bootstrapping after the raring release, hence my suggestion to run from source16:05
danwestdimitern: k16:05
mrammdanwest: once you have a working source install16:05
mrammthe upload tools thing you were pointed at a while ago will probably work16:05
danwestdimitern: mramm: grabbing source now16:07
mrammdanwest: sorry this is not a better path yet16:07
mrammI have to run out and get some food, back in a few16:07
danwestsame here - just grabbing source first16:08
mrammwilliam will also be back on monday, and I know he has tested this16:08
danwestk16:09
MakyoI'm seeing http://pastebin.ubuntu.com/5697229/ in trunk, running in a precise lxc.  Is this a known thing?16:37
dimiternMakyo: don't tell me you never updated from lucid? :)16:37
Makyodimitern, Haha, nah, running raring, but tests were failing there, thus the precise lxc dev setup16:38
dimiternMakyo: well, lucid is definitely not among the expected series iirc16:38
dimiternMakyo: can you reproduce it consistently?16:39
Makyodimitern, Every time.16:39
MakyoLet me run with -gocheck.v16:39
MakyoNo more info.16:40
dimiternMakyo: running with -gocheck.v in environs/imagemetadata/16:41
Makyodimitern, Yeah, no extra info on the failure, others pass.16:43
Makyodimitern, lucid is specified in the test, environs/imagemetadata/simplestreams_test.go:46316:43
dimiternMakyo: file a bug; maybe they decided to drop lucid from the cloud archive just today16:44
Makyodimitern, got it.  Thanks.16:44
dimiternMakyo: thanks16:44
dimiternrogpeppe: ping16:45
rogpeppedimitern: pong16:45
dimiternrogpeppe: actually why do you insist on having api prefix to filenames when split?16:45
rogpeppedimitern: because i like having error.go and server.go stand out as files related to core api server functionality, not the actual API methods16:46
dimiternrogpeppe: hmm16:47
dimiternrogpeppe: well, ok16:47
dimiternrogpeppe: so apiserver.go will be gone, to be replaced by apiroot.go, apistate, apiclient, apimachine, etc, right?16:48
rogpeppedimitern: there is actually a possibility that the api* files could be moved into their own package actually (in the api server anyway)16:48
rogpeppedimitern: yeah16:48
dimiternrogpeppe: that sounds better - so apiserver/api/*16:48
rogpeppedimitern: yeah, but it's only a possibility - i'd prefer not to do it for now16:49
dimiternrogpeppe: what are your concerns about it?16:50
rogpeppedimitern: it feels unnecessary16:50
dimiternrogpeppe: why not rename server to apiserver, errors to apierrors and leave the rest uprefixed?16:51
dimiternrogpeppe: they's stand out even more16:51
rogpeppedimitern: that's not a bad idea16:51
dimiternrogpeppe: cool, i'll make it so then16:52
rogpeppedimitern: sgtm16:52
dimiternrogpeppe: and the same for the client?16:52
rogpeppedimitern: yeah16:52
dimiternrogpeppe: nice16:52
danwestdimitern: any tricks to building juju-core from source?17:50
dimiterndanwest: so if you do go get -v -u launchpad.net/juju-core/... (note the ...)17:51
dimiterndanwest: once done, you'll just need to do go install launchpad.net/juju-core/cmd/juju17:51
danwestk17:52
dimiterndanwest: and then use juju <command> as usual17:52
dimiternrogpeppe: still there?18:16
rogpeppedimitern: aye18:17
dimiternrogpeppe: i'm done with the splitting, proposing in 5m, and i'd like if you take a look18:18
rogpeppedimitern: ok. i'm gonna be gone quite soon though, and i'm really trying to get this branch proposed18:18
dimiternrogpeppe: if you can; it's mechanical, but mostly if it seems ok as an arrangement https://codereview.appspot.com/974604318:21
rogpeppedimitern: well remembered about the copyright notices :-)18:23
dimiternrogpeppe: oh yeah :)18:23
rogpeppedimitern: i get "upload in progress" at https://codereview.appspot.com/9746043/diff/1/state/apiserver/apiserver.go?column_width=9018:24
rogpeppedimitern: i think that's because codereview is confused though.18:24
dimiternrogpeppe: hmm.. well it's just what server.go used to be18:25
dimiternrogpeppe: probably the rename + modify somehow confuses rietveld18:25
rogpeppedimitern: utils.go rather than helpers.go ?18:25
dimiternrogpeppe: np18:26
rogpeppedimitern: i'm wondering if each test file should have its own suite18:26
dimiternrogpeppe: we can do that, but let it be a follow up18:27
rogpeppedimitern: but i don't mind that much18:27
rogpeppedimitern: yeah, definitely18:27
dimiternrogpeppe: i think it should be considered a trivial CL18:27
dimiternrogpeppe: and would really want to land it today, to save me some trouble tracking moving trunk18:28
rogpeppedimitern: where have the auth* functions ended up?18:29
dimiternrogpeppe: in root.go18:29
rogpeppedimitern: ah, that seems goo18:30
rogpepped18:30
rogpeppedimitern: it all seems great actually18:30
dimiternrogpeppe: cool enough to land?18:30
rogpeppedimitern: LGTM trivial18:31
dimiternrogpeppe: great, tyvm!18:31
rogpeppedimitern: i think that's the largest "trivial" change ever :-)18:31
dimiternrogpeppe: yeah :)18:31
=== wedgwood is now known as wedgwood_away
=== wedgwood_away is now known as wedgwood
rogpeppedimitern: in case you're still around, you might want to take a look at this: https://codereview.appspot.com/973804319:14
dimiternrogpeppe: will do19:16
dimiternrogpeppe: LGTM19:30
rogpeppedimitern: thanks. i won't submit until the next one's ready though19:30
dimiternrogpeppe: ok19:30
dimiternrogpeppe: btw "biggest diff ever": Diff against target:7640 lines (+3825/-3612) 26 files modified19:30
rogpeppedimitern: what's that in?19:32
dimiternrogpeppe: the one I just landed19:32
rogpeppedimitern: :-)19:32
rogpeppedimitern: not as big as the error one which may happen sometime, probably19:33
dimiternrogpeppe: in lp - doesn't show the full diff, just the first 5000 lines, but the summary at the top shows the info above19:33
dimiternrogpeppe: oh, yeah - we should do that19:33
rogpeppedimitern: i didn't get it working yet - i was spending too much time on it, so put it on hold19:34
dimiternrogpeppe: yeah, it'll come to that19:35
* dimitern is still not quite sure i didn't screw up trunk with that rename19:35
dimiternI pulled and run all tests pass19:36
dimiternnow i'm doing the same in a fresh clone of trunk, just to be shure19:36
dimitern*sure19:36
* dimitern is relieved.. all tests pass on trunk.. *whew*19:44
dimiterntime to go19:46
dimiternrogpeppe: good night and have a nice holiday!19:46
=== wedgwood is now known as wedgwood_away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!