/srv/irclogs.ubuntu.com/2013/04/18/#juju-dev.txt

* thumper wonders if os.Rename is stable under stress00:05
thumperit is the only think that would be causing this to fail imo00:05
thumperah fuk00:05
davecheney??00:06
thumperI wasn't checking errors00:07
* thumper has a screed of them00:07
davecheneybzzzt00:07
thumperdirectory not empty...00:07
thumperthat is an error I didn't expect00:08
davecheneyis there is a .turd in there ?00:08
davecheneyor an editor file, or something ?00:08
thumper    c.Check(err, IsNil)00:13
thumper... value *os.SyscallError = &os.SyscallError{Syscall:"readdirent", Err:0x2} ("readdirent: no such file or directory")00:13
thumperhuh?00:13
thumperthis is on Unlock00:13
thumperah.. I think I know what this is...00:15
thumpermaybe00:15
thumper\o/00:20
thumperstress test passes now00:20
* thumper ups the stress00:20
thumperdavecheney: what is a reasonable amount of stress in your opinion?00:21
thumper3*100 showed the problem, which is now fixed00:21
bigjoolsobvious joke is obvious00:21
thumperhad to make unlock atomic at fs level too00:22
thumperand rename returned more errors than just ErrExists00:22
thumperwhich caught me out00:22
thumperit comes down to time00:22
thumper1000 iterations with 10 concurrent locks takes about 2.5 seconds00:22
thumperwow, or 7s without the max procs00:23
thumperI really don't want 7s of time added to the test :(00:23
thumper200 and 10 is 1.5s, which is bareable00:24
thumperjust00:24
thumperok, that has taken longer than I wanted...00:25
thumperbut I'm off for lunch00:25
thumperwhich is really heading into town to go to the supermarket and buy the new device CD00:26
bigjoolsdavecheney: does juju core have any kind of automated integration testing?00:43
davecheneybigjools: i think the best answer to that is the charm testing harness that m_3 has built01:02
davecheney2013/04/18 01:06:45 INFO environs/openstack: started instance "1517935"01:07
davecheney2013/04/18 01:06:45 NOTICE worker/provisioner: started machine 45 as instance 151793501:07
davecheney2013/04/18 01:06:45 INFO worker/provisioner: found machine "46" pending provisioning01:07
davecheney2013/04/18 01:06:45 INFO worker/provisioner: found machine "47" pending provisioning01:07
davecheney^ we need to log when the PA reloads01:07
bigjoolsok thanks01:08
jtvHi there bigjools01:08
bigjoolsjtv: you're doing an awesome impression of someone who has the week off :)01:08
jtvI get the hint01:09
jtvI'll be off later, through a region with spotty GSM coverage let alone internet.01:09
jtvI just wanted to pop online for a moment, and then had a long fight with bluetooth tethering in Raring.01:09
jtv(I checked out of the resort earlier this morning)01:10
jtvApart from bluetooth tethering still not working except once just after installation, raring is working out pretty well so far.01:10
jtvWhy does the Ubuntu Software Center now have a big A on it?01:11
bigjoolsjtv: you can re-enable virtual desktops in settings btw01:11
jtvYeah, already did thanks.01:11
jtvI tried to say it on IRC yesterday, but I think my network connection was in a bit of a limbo state at that point.01:11
bigjoolsit seemed so!01:15
jtvOh, gotta go01:19
jtvYou may not get this message because my IRC ping time just now was 44 seconds.01:20
davecheneyhttps://bugs.launchpad.net/juju-core/+bug/117017601:42
thumperthat's nasty01:44
davecheneythumper: i think a nil in instance is being stored in that map01:54
davecheneychecking now01:54
davecheneythumper: ubuntu@juju-hpgoctrl2-machine-0:~$ juju bootstrap -v --upload-tools01:57
davecheney2013/04/18 01:57:18 INFO environs/openstack: opening environment "goscale2"01:57
davecheney2013/04/18 01:57:22 INFO environs/tools: built 1.9.15.1-precise-amd64 (2193kB)01:57
davecheneywhy does upload tools append a build number to the tool ?01:57
thumperdavecheney: I don't know01:58
davecheneythat kind of sucks01:58
davecheneyi wanted to use those numbers01:58
thumperask fwereade01:59
davecheneythumper: https://bugs.launchpad.net/juju-core/+bug/1170176/comments/102:02
thumperhmm, that would explain it :)02:03
thumperjam: hi there, I've munged all my fslock branches into one, and addressed all the comments (I think)04:49
thumperjam: spent most of the day writing tests actually :)04:49
thumperand fixing the fallout when something failed...04:50
davecheney  "17":05:59
davecheney    instance-id: "1520273"05:59
davecheney    dns-name: 15.185.165.3505:59
davecheney    agent-version: 1.9.15.105:59
davecheney    agent-state: started05:59
davecheney  "18":05:59
davecheney    instance-id: "1520275"05:59
davecheney    dns-name: 15.185.165.8105:59
davecheney    agent-version: 1.9.15.105:59
davecheney    agent-state: down05:59
davecheney    agent-state-info: (started)05:59
davecheneywhut ?05:59
* davecheney away til 18:00h06:45
* thumper just realised that the meeting is not in 15 minutes06:46
danilosjam: heya, let me know when you can pair up07:16
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: dimitern | Bugs: 2 Critical, 61 High - https://bugs.launchpad.net/juju-core/
dimiternmorning all07:25
rogpeppe1dimitern: hiya07:25
rogpeppe1mornin' everyone else too!07:25
rogpeppe1darn internet is still down07:25
dimiternrogpeppe1: hey, what happened to status?07:26
rogpeppe1dimitern: sorry, it changed07:26
dimiternrogpeppe1: I like it :)07:26
rogpeppe1dimitern: glad to hear it :-)07:26
TheMuerogpeppe1: me too, even i wondered this morning07:29
TheMuerogpeppe1: i know ctx as abbrev for context, so i wondered here too. the context argument for the command execution btw has the name ctx ;)07:30
rogpeppe1TheMue: now not a single occurrence of .(someType) or interface{} in sight?07:30
rogpeppe1TheMue: yeah, i'd forgotten that. maybe i should change ctxt to ctx to fit in07:31
rogpeppe1TheMue: i hadn't even realised that i'd got "ctx" and "ctxt" in the same function scope...07:32
TheMuerogpeppe1: would be nice. the introduction of a context (or a similar type) as brace for all status operations would have been the next refactoring step, as well as getting rid of this pseudo generic structure07:33
TheMuerogpeppe1: but i thought due to other needs with respect to the code freeze we could do this later07:33
rogpeppe1TheMue: i figured it was easier to do the refactoring than to get the subord and relation stuff to work nicely07:34
rogpeppe1TheMue: i'm afraid i found processService entirely opaque. and i think there was a problem with it too.07:35
rogpeppe1thumper: i don't think you've pushed your latest changes to https://codereview.appspot.com/8602046/07:36
rogpeppe1TheMue: oh, i see, you've made the changes in the final branch only07:37
rogpeppe1oops07:37
rogpeppe1thumper: ^07:37
TheMuerogpeppe1: which changes do you refer to?07:37
TheMuerogpeppe1: ah, you meant thumper07:38
rogpeppe1TheMue: yeah, sorry, tab malfunction :-)07:38
TheMuerogpeppe1: i know that, happens often to me with ro<tab>bbiew, *shit* ^H^H^H^H^Hg<tab> *lol*07:41
* TheMue just got the order to write an article about rust. will be interesting to see the differences to go07:43
davecheney'sup ?07:47
fwereadedavecheney, heyhey, sorry late07:48
fwereadedavecheney, it appends a build number so that uploaded tools always have dev versions07:48
davecheneyfwereade: right07:49
davecheneybut we never made any use of the concept of dev version07:49
davecheneyso now it just sticks out there like a sore thumb07:49
davecheneyfwereade: the reason I wish to complain is I need to use the build version, as I will discuss in the hangout in T-1007:49
fwereadedavecheney, ok, cool07:50
davecheneyfwereade: https://codereview.appspot.com/8648045/07:52
davecheneylooking for a 2nd lgtm07:52
* fwereade looks07:52
davecheneythis turned up today in laod testing07:53
fwereadedavecheney, I don;t think that's correct07:53
fwereadedavecheney, why hide state info if the instance is missing?07:54
fwereadedavecheney, (this is not to say that panicing is correct either)07:54
davecheneyfwereade: ok, that is fine, but this is just making, https://codereview.appspot.com/8842043 work07:55
davecheneyi'm not changing the behavior07:55
davecheneyhttps://docs.google.com/a/canonical.com/document/d/1bSiicbYOV25fq73dZqXz738OU5l96QjkY3mqnXFdLlE/edit07:57
thumperhi rogpeppe107:59
rogpeppe1thumper: hiya07:59
thumperrogpeppe1: yes, changes all in fslock-mashup07:59
dimiternrogpeppe1: thanks for doing this07:59
rogpeppe1thumper: am just reviewing. a few more comments on the fslock code after some pondering.08:00
fwereadewell, it's making it *slightly* less broken, and mea culpa for not spotting that in the review yesterday, but it's not really following intent08:00
dimiternwhere did the g+ link from the even go?08:01
dimiternevent08:01
dimiterncan anybody send me the link?08:01
dimiternfwereade, rogpeppe1: ?08:02
dimiterndavecheney, thumper: link please?08:04
rogpeppe1dimitern: https://plus.google.com/hangouts/_/calendar/bWFyay5yYW1tLWNocmlzdGVuc2VuQGNhbm9uaWNhbC5jb20.gdt9rkp5uspih9n3db6b95kccc08:04
dimiternrogpeppe1: cheers08:04
rogpeppe1dimitern: i'm probably not going to be able to make it.08:04
rogpeppe1dimitern: will try through my phone connection, but i have my doubts08:04
davecheney1:05 AM08:06
davecheneyThursday, April 18, 2013 (PDT)08:06
davecheneyTime in Portland, OR, USA08:06
fwereadeum09:49
fwereadedo we have any expectation that the packaged version will work? we don't know what the critical differences were in 1.9.1409:50
fwereade(other than that they shouldn't exist)09:50
fwereadejam, rogpeppe2, dimitern: ^09:50
rogpeppe2fwereade: we should try the packaged version and see whether we see the same problems09:51
dimiternfwereade: not sure i understand your question09:51
rogpeppe2fwereade: it looked like the problem was client-side, so shouldn't be too hard to diagnose09:51
* fwereade is trying to remember what the hell he saw going wrong when he tried ap-southeast-209:53
fwereadeok, rogpeppe2 and dimitern, you should not be worrying about this09:53
fwereadeyou have code to land :)09:53
dimiternfwereade: I'll try with the ppa and one random region09:53
dimiternfwereade: (after I land my stuff)09:54
fwereadedimitern, awesome09:54
fwereadeTheMue, would you pick a region, let us know what it is, and get bootstrapping from the PPA please?09:54
* TheMue just prepares a test image with ppa09:54
fwereadeTheMue, AFAICT us-east-1 works09:55
TheMuefwereade: hehe, just wrote when you sent09:55
fwereadeTheMue, except... hmm, no, I think that maybe even that does not10:00
rvbaHi guys, I just put up for review a branch which adds constraints support in the MAAS provider: https://codereview.appspot.com/8842045/10:04
rvbaPlease have a look.10:04
dimiternrvba: i'm glad to see you got lbox working!10:05
fwereadervba, cool10:05
rvbadimitern: yeah, I must admit the problem was my fault, wrong bzr config.10:05
dimiternrvba: please share your findings with the rest of the red squad, so they can set it up too :)10:06
rvbadimitern: already done :)10:06
dimiternrvba: great, thnaks!10:07
fwereadervba, I will take a look at it but can I please ask you not to land anything today, while we try to handle the release frenzy10:07
rogpeppe2fwereade: what do think about reporting agent-state=pending when instance-state==pending or missing?10:07
rvbafwereade: sure, no problem.10:07
fwereaderogpeppe2, I'd rather not pretend we can give an instance-state when we can't10:07
rogpeppe2fwereade: oops, sorry, i meant instance-id not instance-state10:07
fwereaderogpeppe2, the list appeared sanguine about the prospect of (temporarily) missing agent-state10:07
dimiternrvba: I'll review it shortly10:08
rvbata10:08
rogpeppe2fwereade: the current tests assume no agent-state when instance-id is pending or missing10:08
fwereaderogpeppe2, while an instance id is pending I'm fine with reporting only machine series10:08
fwereaderogpeppe2, missing is a different matter10:08
rogpeppe2fwereade: my changes are pushing towards reporting it always10:08
rogpeppe2fwereade: otherwise i have to special-case10:09
rogpeppe2fwereade: which seems kinda unnecessary.10:09
fwereaderogpeppe2, that's probably simplest -- the downside is that not-yet-provisioned machines are nicely visually distinct today10:09
fwereaderogpeppe2, and this would work against that10:09
rogpeppe2fwereade: yeah, maybe i'll make pending the only special case10:10
fwereaderogpeppe2, when you say special-case... ISTM that it's just one branch in one place10:10
rogpeppe2fwereade: sure10:10
fwereaderogpeppe2, if it has tentacles that's a different matter10:10
fwereaderogpeppe2, +1 on early exit on pending10:10
rogpeppe2fwereade: every if statement doubles the number of reachable states :-)10:10
fwereaderogpeppe2, but I thought dimitern was doing that?10:10
rogpeppe2fwereade: it's actually "if id==pending {agentstate=""}.10:11
rogpeppe2fwereade: it meshed too closely with what i was doing already10:11
dimiternfwereade: we agreed rogpeppe2 to take life and I'll do series10:11
fwereaderogpeppe2, agreed, but we are trying to report on a very large number of states ;p10:11
fwereadedimitern, rogpeppe2: isn't that just asking for conflicts?10:11
rogpeppe2fwereade: i was already mucking with processMachine10:11
fwereaderogpeppe2, you have one to land and one to write and get reviewed already though10:12
rogpeppe2fwereade: one to land?10:12
fwereaderogpeppe2, https://codereview.appspot.com/8821043/ ?10:12
dimiternrogpeppe2: cmd logging10:12
fwereaderogpeppe2, that one on top10:13
rogpeppe2fwereade: oh yeah; will do10:13
fwereadedimitern, it's a significant reduction in logspam when the allwatcher's running, been approved for a day or 210:13
fwereaderogpeppe2, dimitern: regardless, rogpeppe2 is messing with exactly the method dimitern needs to change10:14
dimiternfwereade: what's that?10:14
fwereadedimitern, processMachine10:14
rogpeppe2fwereade: that is true10:14
fwereaderogpeppe2, I do not think this work is sanely parallelisable10:15
dimiternthe changes shouldn't conflict (or not badly)10:15
rogpeppe2fwereade: the tests are the main part of the work10:15
rogpeppe2fwereade: i don't care about conflicts in the code - it's all trivial10:15
fwereaderogpeppe2, so dimitern needs to modify every one of your tests, and also your code10:15
rogpeppe2fwereade: hmm. dimitern shall i pass over my WIP to you?10:16
fwereaderogpeppe2, I appreciated the ninja-rewrite last night very much but I think this is decidedly less convenient tbh10:16
rogpeppe2fwereade: i think that all the changes we agreed to do will clash10:17
dimiternrogpeppe2: I'm proposing mine now10:17
rogpeppe2dimitern: cool10:17
fwereaderogpeppe2, dimitern: cool, I think that is a much cleaner direction to merge10:17
fwereaderogpeppe2, dimitern: objections withdrawn10:18
fwereaderogpeppe2, please focus on the other branches while waiting on dimitern's to land though10:19
rogpeppe2fwereade: will do10:19
rogpeppe2fwereade: only one branch, right?10:19
fwereaderogpeppe2, cheers -- well, 290 to land if not already done, command output to propose with the logging just in SuperCommand10:20
rogpeppe2fwereade: ah, thanks for reminding of that one. i should've made a list!10:20
dimiternI'm having trouble authenticating on rietveld while proposing :( tried 10 times, sign out/in from the web site works, I authorized the app (again), still no joy - probably related to the recent google apps / ubuntu sso change?10:22
fwereadedimitern, np, it's pretty easy to review on LP10:36
fwereadedimitern, you have an LGTM, but considering the current circumstances I'm not keen to call it a trivial10:37
fwereaderogpeppe2, would you glance at https://code.launchpad.net/~dimitern/juju-core/034-status-shows-machine-series/+merge/159589 briefly please?10:38
dimiternfwereade: sure10:38
fwereadeTheMue, fwiw my current findings are that eu-west-1 works perfectly but I *think* the us-east-1 issues were down to ec2 not us10:42
rogpeppe2fwereade: looking10:42
rogpeppe2fwereade, dimitern: i might remove the omitempty, 'cos if we do have a blank series for some reason, we'll want to know10:43
dimiternrogpeppe2: ok10:43
dimiternrogpeppe2: can you pull my branch and merge it for me please?10:44
fwereaderogpeppe2, you can't create a machine without a series, and you can't set a machine's series once it's created10:44
rogpeppe2fwereade: so it can never be empty?10:44
fwereaderogpeppe2, yeah, I think that is a guarantee that state makes10:44
fwereaderogpeppe2, hence no ,bool or ,error10:44
rogpeppe2fwereade: in which case we don't need the omitempty, right?10:44
dimiternrogpeppe2: i'm still struggling to get lbox/lpad working - auth failing10:44
fwereaderogpeppe2, ha, that is true10:44
rogpeppe2fwereade: i don't care much though10:44
rogpeppe2dimitern: ok, i'll merge it for you10:45
fwereaderogpeppe2, AFAICT omitempty is entirely academic10:45
dimiternrogpeppe2: tyvm10:45
fwereaderogpeppe2, follow your heart10:45
rogpeppe2fwereade: yeah. i might leave it there for consistency10:46
fwereaderogpeppe2, +110:46
fwereadeTheMue, ok, yes, us-east-1 problem confirmed as an unhappily-timed connection loss to s3 causing apparent lack of tools10:47
fwereadeTheMue, where are you looking?10:48
rogpeppe2fwereade: Put needs to retry :-)10:48
fwereaderogpeppe2, it's List actually10:48
fwereade;10:48
rogpeppe2fwereade: hmm. i thought List did10:48
dimiternwtf is this: 2013/04/18 12:48:49 RIETVELD 0xf840000150 client.Get returned (*http.Response)(nil), &url.Error{Op:"Get", URL:"http://example.com/marker", Err:(*errors.errorString)(0xf8400a26d0)}10:49
TheMuefwereade: wanted to look at us-east-1, but will now choose a different one. had troubles with my test image. :(10:49
fwereadeTheMue, what's this test image?10:49
fwereadeTheMue, you can just install from the ppa, can't you?10:50
TheMuefwereade: have an extra vm for it10:51
dimiternGet http://example.com/marker: redirect blocked ???10:52
dimiternreported by net/http/client.Get10:52
fwereadedimitern, have you ever visited example.com?10:52
fwereadedimitern, it's a placeholder basically10:53
dimiternfwereade: i *know* what it is, but why is lbox misbehaving?10:53
dimiternfwereade: interestingly, searching for that message in google gave me a #juju-dev log where I complained about the same thing on 2012/11/21 :)10:54
dimiternhttp://irclogs.ubuntu.com/2012/11/21/%23juju-dev.txt10:54
fwereadedimitern, I was taking the use of example.com to be evidence of lbox hitting the crack pipe pretty hard today, but I don't know *why*10:54
fwereadedimitern, does nuking your various relevant .files  and reauthing help?10:55
dimiternfwereade: tried that already10:55
fwereadedimitern, sorry, out of ideas then :(10:55
dimiternfwereade: yeah... drawing knowledge from my earlier self in that conversation - it seems it's a go 1.0.3 issue, which was fixed on tip, and I need to rebuild lbox with go tip10:59
rogpeppe2cmd/juju tests almost take 3 minutes!11:00
fwereadeFFS this is altogether too eventual for my liking11:02
fwereadeand I need food11:02
fwereaderogpeppe2, I'm aware of those tests, they are for now the price we pay for coverage11:03
fwereaderogpeppe2, I will be trying to figure out what the hell is taking so long very soon11:03
rogpeppe2fwereade: me too :-)11:03
fwereadebut for now, lunch is an absolute necessity11:03
fwereademaybe the instance will have shown up next hour11:03
fwereadedimitern, rogpeppe2: if you can find someone else to review everything before I return, I would encourage one of you to set the build number and follow up on my juju-dev email11:04
fwereadedimitern, rogpeppe2: but please, whoever does that, make sure it works live ;p11:05
fwereadedimitern, rogpeppe2: if not I'll bbiab11:06
rogpeppe2fwereade: ok; i'm currently doing the manual diff thing on DeepEqual output11:06
dimiternit worked!11:10
dimiternso, for the record: goetveld is broken before a patch from wallyworld_ (https://code.launchpad.net/~wallyworld/goetveld/auth-cookie-fix/+merge/147585)11:12
dimiternnow it works with go 1.0.3, the "redirect blocked" issue is gone and I can use it normally11:13
rogpeppe2everything crashes11:34
wallyworld_jam: dimitern: ping11:34
* rogpeppe2 thanks thumper for passing on the nm-applet hack11:34
dimiternI cannot bootstrap on any region with 1.9.14 from the ppa: http://paste.ubuntu.com/5718528/11:35
dimiternfwereade: any idea?11:36
TheMuecurrently it looks as i can bootstrap but status doesn't return :(11:38
TheMueah, now, a pending machine 011:39
dimiternTheMue: how did you manage? with the ppa version and just "juju bootstrap" on ec2?11:39
dimiternTheMue: no --upload-tools or --series, right?11:40
dimiternrogpeppe2: you managed to get my branch?11:41
rogpeppe2dimitern: my phone went down, and i've run into unexpected difficulties with the Life change.11:41
rogpeppe2dimitern: will submit your branch now11:41
TheMuedimitern: with the ppa version11:42
TheMuedimitern: and a pure juju bootstrap11:42
dimiternTheMue: which region?11:42
dimiternTheMue: ah, you're running precise!11:43
TheMuedimitern: i wanted to start from west to east, so us-east-1 now11:43
TheMuedimitern: yes, precise11:43
dimiternso it's not working on quantal11:44
rogpeppe2dimitern: i'm trying to figure out the correct logic for processAgent11:46
rogpeppe2dimitern: i can't convince myself that it's currently right, and the tests aren't great11:46
rogpeppe2dimitern: i've been writing out a truth table11:46
dimiternrogpeppe2: I see11:46
rogpeppe2TheMue: i wonder if you could talk me through the logic in processAgent (it was processStatus)11:47
dimiternrogpeppe2: the idea is to have Life(), AgentAlive(), AgentTools() and Status() for entities that support it - units and machines11:48
dimiternrogpeppe2: and process them similarly11:48
rogpeppe2i can't quite get my head around this condition: status != params.StatusPending && !agentAlive && !entityDead11:48
rogpeppe2dimitern: i realise that11:49
dimiternrogpeppe2: I can help with that11:49
rogpeppe2dimitern: it's just that under *some* conditions the agentAlive status is lost11:49
dimiternrogpeppe2: this is used to determine if the agent is down11:49
dimiternrogpeppe2: it's down if the machine is alive, but the agent is not and the status is not pending (i.e. provisioned and started)11:50
TheMuerogpeppe2: never touched processAgent(), sorry11:51
rogpeppe2TheMue: it's the same logic you wrote in processStatus11:51
TheMuerogpeppe2: one moment, have to open then code11:53
rogpeppe2TheMue: it's ok, i think i'm there11:53
dimiternrogpeppe2: I wrote that actually11:53
rogpeppe2dimitern: ah, ok11:54
dimiternrogpeppe2: see above, does it make sense?11:54
rogpeppe2dimitern: it's all those double-negatives makes me see boggle-eyed11:54
dimiternrogpeppe2: simple logic :)11:55
rogpeppe2dimitern: yeah. i have a better intuitive grasp when it's if !(status == params.StatusPending || agentAlive || entityDead) {11:57
dimiternrogpeppe2: change it, if you think it'll be more readable11:57
dimiternrogpeppe2: as long as it's the same logic11:57
rogpeppe2dimitern: i'm not sure. it helped me, but probably only 'cos i'd been staring at it the other way11:58
rogpeppe2dimitern: standard boolean transformation11:58
dimiternrogpeppe2: yeah11:58
rogpeppe2dimitern: i can't remember the name of the rule though11:58
rogpeppe2dimitern: i think there's no point in calling AgentAlive if the status is pending12:00
dimiternrogpeppe2: yeah12:01
dimiternrogpeppe2: correct12:01
dimiternhttp://paste.ubuntu.com/5718578/ - still cannot bootstrap from the ppa (tried both us-east-1 and eu-west-1)12:04
fwereadedimitern, 2013/04/18 14:00:46 ERROR command failed: cannot find tools: use of closed network connection12:05
fwereadedimitern, I have been seeng that sometimes, but not always12:05
fwereadedimitern, I don't *think* it's us12:05
TheMuedimitern: i now have a problem with us-west-2 when creating the s3 control bucket (conflicting condition)12:05
TheMuedimitern: us-east-1 worked fine12:06
fwereadeTheMue, s3 bucket names are global12:06
rogpeppe2dimitern: i've reworked the code a little (the logic should still be the same though) http://paste.ubuntu.com/5718581/12:06
dimiternfwereade: yeah, the issue before (with uncommented public-bucket) was different (no compatible tools found)12:06
fwereadeTheMue, you need a new name12:06
dimiternrogpeppe2: looks good12:07
TheMuefwereade: even if the former is destroyed?12:07
rogpeppe2dimitern: at least my small brain can wrap itself around it now :-)12:07
dimiternrogpeppe2: :)12:07
fwereadeTheMue, I don;t recall that ever working, no12:08
fwereadeTheMue, dimitern: I have successfully bootstrapped and deployed in both us-east-1 and eu-west-1 with the ppa12:08
TheMuefwereade: hmm, so when testing between east and southeast i seem to have switched my buckets. can't remember, but it looks like.12:08
dimiternfwereade: you're running precise, maybe that's why12:09
TheMuefwereade: and i successfully to us-east-112:09
fwereadeTheMue, dimitern: I have "successfully bootstrapped" in ap-southeast-2, as in I have a bootstrap instance running, but it's been running for an hour and I'm still unable to get the instance12:09
TheMuefwereade: do we have an issue or doc to collect the test results12:09
fwereadeTheMue, creating an issue now12:10
TheMuefwereade: thx, +112:11
* rogpeppe2 hates the spot-the-difference competition when a DeepEqual fails: http://paste.ubuntu.com/5718597/12:12
TheMuerogpeppe2: hehe, i know that from my tests. i then replaced those map[i]i by \n and some manual sorting. so it gets easier12:14
rogpeppe2current mostly-manual solution: Edit ,|gofmt Edit ,x/"}/c/",\n}/ Edit ,x/,/a/\n/ Edit ,x/{./v/}/x/{/a/\n/12:14
rogpeppe2:-)12:14
fwereadeTheMue,  dimitern: https://bugs.launchpad.net/juju-core/+bug/117032612:14
fwereadedimitern, rogpeppe2: btw how is progress? should I be reviewing, supporting, etc?12:14
dimiternfwereade: I'm done - mine has landed12:15
fwereadedimitern, awesome12:15
rogpeppe2fwereade: just working out the right way to fix the tests. at least i'm convinced the logic is good now.12:15
fwereaderogpeppe2, ?/312:15
fwereaderogpeppe2, excellent12:16
rogpeppe2fwereade: i've submitted the log changes12:16
fwereaderogpeppe2, cool12:16
rogpeppe2fwereade: not the finished ones though12:16
rogpeppe2fwereade: i think status life is 100x more important12:16
fwereaderogpeppe2, well, I did originally ask dimitern to do it because (1) he'd already started and (2) you had 2 other branches to do12:17
fwereaderogpeppe2, and now he is kickinghis heels12:17
rogpeppe2fwereade: yeah, sorry, i thought life was trivial to do along with what i was doing anyway12:18
fwereaderogpeppe2, I didn't think you were actually doing anything with status12:18
rogpeppe2fwereade: for some reason i thought i was12:18
fwereaderogpeppe2, we agreed and minuted otherwise12:20
fwereaderogpeppe2, https://docs.google.com/a/canonical.com/document/d/1bSiicbYOV25fq73dZqXz738OU5l96QjkY3mqnXFdLlE/edit#12:20
fwereaderogpeppe2, but hey ho12:21
rogpeppe2fwereade: ah, i think it was because i was already half way through some changes when that was minuted12:21
dimiternso, with tip I'm able to bootstrap with default region and public-bucket (commented out) (no --upload-tools or --series), with default-series: precise12:22
dimiternbut not with 1.9.14 from the ppa12:22
fwereadedimitern, what's the error?12:22
dimiternfwereade: the same - use of closed network connection12:23
fwereadedimitern, and it's meaningless to compare tip and 1.9.14, I think12:23
fwereadehey!12:23
fwereadehas goamz updated recently?12:23
dimiternso 1.19.14-amd64-quantal1 is confirmed broken12:23
dimiternfwereade: it has i think12:23
fwereadeGAAAH12:24
fwereademagic dependency updates FTL12:24
rogpeppe2right, tests pass.12:24
rogpeppe2the difference between agent-state=pending and instance-id=pending is subtle12:25
fwereadehmm, no goamz changes since march it seems12:26
rogpeppe2fwereade: you might be stuck12:27
rogpeppe2fwereade: try removing the goamz directory and go getting again12:27
fwereaderogpeppe2, I went to look on launchpad ;)12:27
fwereaderogpeppe2, but hmm maybe I hadn't actually updated since before then?12:27
fwereaderogpeppe2, but no12:27
fwereaderogpeppe2, we've done releases that didn't exhibit these issues, right?12:28
rogpeppe2fwereade: you're right12:28
rogpeppe2fwereade: 35 is my latest revno12:28
rogpeppe2fwereade, dimitern: https://codereview.appspot.com/885204312:29
rogpeppe2one known issue - the summary in the test is wrong; fixing12:30
benjithe current state is that when the user clicks "Add" on a charm page the charm details will dissapear and the left sidebar will stay visible and the service configuration panel will be displayed at the right12:32
benjiin full-screen mode we will switch to sidebar mode and go to the same state12:32
benjiI don't know if Rick is communicating with Jovan or not12:33
rogpeppe2fwereade, dimitern: now proposed with that fixed12:34
dimiternrogpeppe2: reviewed12:35
dimiternfwereade: I updated bug 117032612:36
dimitern_mup_: wtf?12:36
dimiternhttps://bugs.launchpad.net/juju-core/+bug/117032612:36
fwereaderogpeppe2, reviewed12:37
fwereadeTheMue, how do you "install" in juju?12:37
TheMuefwereade: typo, deployed a service and waited until it is started12:38
TheMuefwereade: currently in us-west-112:38
rogpeppe2fwereade: what instance id would we set?12:39
rogpeppe2fwereade: when there's no instance12:39
fwereaderogpeppe2, the one in state?12:39
fwereaderogpeppe2, I don;t understand why you'd ever call instance.Id()12:39
TheMuehmm, sadly can't edit, will comment it after current test.12:39
rogpeppe2fwereade: excellent point12:40
TheMuebut looks good so far, mysql is pending12:40
fwereadeTheMue, are you watching the provisioner logs?12:40
rogpeppe2fwereade: FWIW this is an old problem - the logic there hasn't changed12:40
fwereaderogpeppe2, it was also stuff that I'd figured out with dimitern before you took it over unilaterally12:41
TheMuefwereade: as long as the commands and status tell me it's ok not. shall i look for something special?12:41
rogpeppe2fwereade: v sorry about that12:41
* dimitern lunch12:42
fwereaderogpeppe2, no worries, it happens, I'm just a bit confused that it did when I thought I'd been extra clear -- but it takes at least 2 to experience a communication problem ;)12:42
rogpeppe2fwereade: is there ever a case that we can have an alive agent when the entity status is pending?12:53
dimiternrogpeppe2: not really, with the nonced provisioning changes, even if this happens briefly, the agent will commit suicide soon after starting (even before setting AgentAlive I think)12:57
rogpeppe2dimitern: that's what i think. i was asking because of the review comment about that, so thought perhaps fwereade had some more useful input there.12:58
rogpeppe2fwereade, dimitern: PTAL https://codereview.appspot.com/885204312:58
dimiternrogpeppe2: changing status is *not* what an agent does first - it sets itself as alive first13:00
rogpeppe2dimitern: interesting13:00
rogpeppe2dimitern: perhaps it should be the other way around13:00
dimiternrogpeppe2: take a look at both uniter/modes and machiner13:01
dimiternrogpeppe2: how so?13:01
rogpeppe2dimitern: it's an early indication of liveness13:02
rogpeppe2dimitern: we save round trips in status13:02
rvbadimitern: thanks for the review (MAAS provider constraints branch)!  I see you guys are busy, just ping me when it's ok for me to land this.13:02
rogpeppe2dimitern: it's a once-and-for-all "i have started running!" - then the liveness status can change over time13:03
dimiternrvba: please don't - we're about to release and it can land after that13:03
dimiternrvba: it's a bit of a mess anyway, let's not complicate it13:03
rvbadimitern: sure, I will wait until you guys tell me it's good to go.13:04
dimiternrvba: cheers!13:04
dimiternrogpeppe2: i don't think it's an early indication of liveness13:04
dimiternrogpeppe2: setagentalive is that indication, not the status change13:04
dimiternrogpeppe2: it might seem so only from the command's perspective13:05
rogpeppe2dimitern: it's an indication that it got there anyway.13:05
rogpeppe2dimitern: if we set the status, then die immediately, then we at least see that it got that far13:05
dimiternrogpeppe2: how is that useful?13:06
dimiternrogpeppe2: if it dies, the status will be incorrect anyway13:06
dimiternrogpeppe2: the agent has to be alive to set the status13:06
rogpeppe2dimitern: if it dies, we'll print "down" for the status13:06
rogpeppe2dimitern: so there's no difference there13:07
dimiternrogpeppe2: "down" doesn't actually mean "oops i crashed while starting"13:07
rogpeppe2dimitern: it means "i crashed"13:07
dimiternrogpeppe2: it means "i was running ok, entity was started, then something went wrong and i died"13:07
rogpeppe2dimitern: for me, it means that the agent started running and then stopped working for some reason13:08
dimiternrogpeppe2: the significant distinction here is, the entity went into a started state before "down" being meaningful13:09
rogpeppe2dimitern: and a good (the only) indication we have that an agent started running is that it set its status13:09
dimiternrogpeppe2: exactly, in addition to being alive as well13:09
dimiternrogpeppe2: if the agents sets status to "started" and then sets itself alive, we'll see "down" a lot more often, and it will be a lie13:10
dimiternrogpeppe2: it's a very brief moment, i agree, but it still be a lie13:11
rogpeppe2dimitern: hmm, good point13:11
dimiternrogpeppe2: that's the point in setting the status after setting the agent to alive13:12
rogpeppe2dimitern: perhaps we should have a "pending but alive" status13:12
dimiternrogpeppe2: how?13:13
dimiternrogpeppe2: agent live and status are tightly linked ("agent-state" is what we use for status)13:13
fwereaderogpeppe2, dimitern: if we did, I think it'd probably be "running"13:13
rogpeppe2fwereade: that's a good idea13:13
fwereaderogpeppe2, dimitern: but I'm not sure13:14
dimiternfwereade: we'll be departing from py-juju compatibility a bit if we do this13:14
rogpeppe2fwereade: https://codereview.appspot.com/8658045/13:14
rogpeppe2dimitern: ^13:15
fwereadedimitern, not much tbh -- it's an extra step in unit status, and a bit closer to python for the machine -- although "running" will still not be a terminal state for a machine13:15
fwereaderogpeppe2, cheers13:15
rogpeppe2fwereade: still waiting on https://codereview.appspot.com/8852043/ too13:15
dimiternrogpeppe2: why remove Noticef("agent starting") ?13:16
dimiternfwereade: well, in that case we can have "agent-state": "running" when status is pending and the agent is alive13:17
rogpeppe2dimitern: which file?13:17
dimiternfwereade: but then we'll have "agent-state": "started" for most of the time13:18
dimiternrogpeppe2: see the comments inline13:18
fwereadedimitern, yeah, matching the unit agent13:18
rogpeppe2dimitern: ah, the *agent exiting* messages13:18
rogpeppe2dimitern: they're now redundant13:18
dimiternrogpeppe2: how so?13:18
fwereadedimitern, I *think* it is more important to impose consistency here by messing with the less-interesting-to-observe status13:18
rogpeppe2dimitern: as the messages are printed by supercommand13:18
rogpeppe2dimitern: which is the point of the CL13:19
rogpeppe2dimitern: no point in printing the info twice, i think13:19
dimiternrogpeppe2: I though the point was to report "command completed successfully" for cli commands, not agents13:19
rogpeppe2dimitern: i did that originally, but fwereade suggested the supercommand change13:19
rogpeppe2dimitern: and given that we print it *anyway* for agents, why not?13:20
dimiternfwereade: yeah, but it's slightly confusing to have "running" (for a short while) and "started" otherwise13:20
fwereadedimitern, yeah13:20
fwereadedimitern, maybe "starting" would be better13:21
fwereadedimitern, anyway I don;t think that's one for today13:21
dimiternrogpeppe2: I don't think these two are related - cli commands report success as a courtesy to the user; agents log stuff which is greppable by admins/etc.13:21
dimiternfwereade: +1 for "starting"13:21
fwereadewait, did we not do life for services/relations?13:21
rogpeppe2dimitern: they're both logging the same thing, no?13:22
dimiternfwereade: and I also agree it's better postponed for after today13:22
rogpeppe2dimitern: given that the exit status is now logged by supercommand, what's the reason for the Noticef in the juju commands?13:22
dimiternrogpeppe2: unless i'm on crack cli commands use stdout/err to report these things, not the logging infrastructure13:23
fwereadedimitern, you're on crack ;p13:23
fwereadedimitern, about one CLI command does13:23
dimitern:)13:23
rogpeppe2+1 :-)13:23
fwereadedimitern, they all *should* but that's not important enough for now13:23
rogpeppe2dimitern: this CL is entirely about log messages13:24
dimiternso, with this change - if I run "juju somecommand" will I see "command completed successfully" on the console after it finished?13:24
fwereaderogpeppe2, the status one's looking good given what you've done13:24
rogpeppe2dimitern: no13:24
rogpeppe2dimitern: only if you use --verbose13:24
fwereaderogpeppe2, but there's no life for services/relations that I can see13:24
dimiternrogpeppe2: so that's what I was thinking13:24
rogpeppe2fwereade: ah13:24
dimiternrogpeppe2: can we add the stdout/err message like this as well please?13:24
rogpeppe2dimitern: no13:25
rogpeppe2dimitern: :-)13:25
rogpeppe2dimitern: i don't think a command should print this stuff by default13:25
rogpeppe2dimitern: it's noise13:25
dimiternit's nice and reassuring13:25
dimiternbut, fine13:25
rogpeppe2dimitern: so is the next shell promt :-)13:25
fwereaderogpeppe2, LGTM with the extra life fields13:26
rogpeppe2dimitern: if there's an error, that *will* be printed13:26
dimiternrogpeppe2: fair enough13:26
dimiternrogpeppe2: LGTM then13:27
rogpeppe2dimitern: thanks13:27
rogpeppe2dimitern: i'll leave "completed successfully" for another day if that's ok13:27
dimiternrogpeppe2: series is still omitempty - should it be left like this?13:28
rogpeppe2dimitern: yeah, i decided it was fine13:28
dimiternrogpeppe2: you mean the wording or the stdout output?13:28
rogpeppe2dimitern: the wording13:28
rogpeppe2dimitern: i don't want to spend another 10 minute round-trip13:28
dimiternrogpeppe2: a command always finishes, even when it fails13:29
rogpeppe2dimitern: yeah, i'm +1 on the change, but i don't think it's that important right now13:29
dimiternrogpeppe2: ok, if we're absolutely rushing things, fine13:29
rogpeppe2dimitern: meeting at 313:29
dimiternrogpeppe2: so once you land these 2 we're done?13:30
fwereadegod, meeting13:30
fwereadeI'm going to lie down for 20 mins13:30
fwereadesee you then13:30
dimiternwho is gonna do the release following dave's process?13:30
rogpeppe2fwereade: how do you suggest we show relation life status?13:31
rogpeppe2fwereade: currently relations are13:31
rogpeppe2Relations     map[string][]string   `json:"relations,omitempty" yaml:"relations,omitempty"`13:31
rogpeppe2fwereade: no struct for a life field13:31
dimiternrogpeppe2: the same way?13:31
rogpeppe2dimitern: how do you mean?13:31
dimiternrogpeppe2: having relationStatus instead of string?13:31
rogpeppe2dimitern: that will break compatibility, no?13:32
dimiternrogpeppe2: i think so, yeah13:32
dimiternfwereade: when you're back comment on this one please13:33
dimiternrogpeppe2: well, we can always add it in parenthesis after it :)13:34
rogpeppe2dimitern: bad idea13:34
dimiternrogpeppe2: best compromise I think13:34
rogpeppe2dimitern: that breaks scripts horribly13:34
rogpeppe2dimitern: i'd add another field, RelationLife map[string]state.Life13:35
rogpeppe2dimitern: or something like that13:35
dimiternrogpeppe2: yeah, should work, at the expense of extra output size, but meh..13:35
rogpeppe2dimitern: we'd only include dead and dying ones13:35
dimiternrogpeppe2: good point!13:36
dimiternrogpeppe2: were we doing that in python?13:36
dimiternrogpeppe2: reporting not alive relations13:36
rogpeppe2dimitern: i'm not sure there was such a concept in the python13:37
dimiternrogpeppe2: really? oh, well13:37
rogpeppe2dimitern: quick: remind me of a good way to get a service that's in a dying state?13:41
dimiternrogpeppe2: st.Service(id) should work13:41
rogpeppe2dimitern: no, i mean, to create a service and then put it into a dying state13:41
dimiternrogpeppe2: ah, Destroy()13:42
rogpeppe2dimitern: doesn't that just remove the service if there's not something keeping it around13:42
rogpeppe2?13:42
dimiternrogpeppe2: yeah, you'll need to add a subordinate at least13:42
rogpeppe2dimitern: i think i probably need to add a unit13:42
rogpeppe2dimitern: but i'm not sure i can call Destroy if there's a unit13:43
* rogpeppe2 thinks sometimes that we should have a way of recreating a given desired State rather than going through all the steps necessary to arrive at it13:44
dimiternrogpeppe2: take a look at preventUnitDestroyRemove13:44
rogpeppe2dimitern: thank you13:44
rogpeppe2dimitern: perfect!13:44
rogpeppe2dimitern: oh, no13:45
rogpeppe2dimitern: that's the unit not the service13:45
dimiternrogpeppe2: hmm..13:46
rogpeppe2dimitern: actually, maybe i can destroy it even if it has units13:46
dimiternrogpeppe2: if it has 1 unit and no relations, destroy() will remove it, otherwise it'll set it to dying13:47
rogpeppe2dimitern: so i need a relation i guess13:48
dimiternrogpeppe2: looks that way - at least according to some of the tests that are faking a relation to test destroy()13:48
dimiternrogpeppe2: TestDestroyStillHasUnits13:49
rogpeppe2dimitern: interesting. i think that might be wrong actually.13:51
dimiternrogpeppe2: which one?13:51
rogpeppe2dimitern: if i destroy a service before it's been provisioned i think it should go away and all its units too13:51
dimiternrogpeppe2: provisioning applies to machines, not services, right?13:51
rogpeppe2dimitern: yeah13:52
dimiternrogpeppe2: I can't deploy a service with no units with the cli13:52
rogpeppe2dimitern: but it looks like i can't destroy a service until the units i created on it have started and stopped13:52
dimiternrogpeppe2: well, adding a unit assumes you want it started13:53
rogpeppe2dimitern: if i do {juju deploy wordpress; juju destroy-service wordpress} i shouldn't have to wait 10 minutes for the machine to come up13:53
rvbafwereade: I know you're busy right now and this is definitely not urgent but when you have time, could you please see what you have to say about Jeroen's comment here: https://code.launchpad.net/~maas-maintainers/juju-core/maas-provider-skeleton/+merge/157025/comments/34775213:53
rogpeppe2dimitern: yeah, but people change their minds13:53
=== wedgwood_away is now known as wedgwood
dimiternrogpeppe2: peaople should rtfm13:54
rogpeppe2dimitern: and there's little more annoying than a service that doesn't do what it could do13:54
dimitern:)13:54
rogpeppe2dimitern: i think we could do a better job here - the manual says "you can't stop what you've started, even though it's quite possible to do so|"13:54
dimiternrogpeppe2: they can change their mind, they just have to wait for the action they issued before destroying13:55
rogpeppe2dimitern: yeah. we could do better there13:55
dimiternrogpeppe2: possibly yeah, but that's not the only place, i assure you :)13:55
rogpeppe2dimitern: indeed :-)13:56
dimiternrogpeppe2: it could be worth adding a wishlish bug?13:56
dimiternwishlist13:56
rogpeppe2dimitern: just a bug would do13:57
dimiternkanban meeting guys?14:04
dimiternrogpeppe2, TheMue: kanban?14:06
TheMueouch, yes14:06
fwereadervba, responded14:25
rvbafwereade: ta14:41
rogpeppe2lunch15:05
* TheMue has to step out in a few moments for dinner in a restaurant, younger daughter has her 17th birthday today15:07
rogpeppe2TheMue: have fun15:29
rogpeppe2live tests passed against trunk for me (except the usual StopInstances failure)15:29
TheMuerogpeppe2: thx, we'll have. and daddy is allowed to pay. :)15:31
rogpeppe3anyone here know the easiest way to script install setuptools ?15:52
rogpeppe3the best i've got currently is:15:52
rogpeppe3wget -o setuptools.egg http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5=fe1f997bc722265116870bc7919059ea15:52
rogpeppe3sh *.egg15:52
rogpeppe3which seems a bit arbitrary15:52
rogpeppe3this is in a charm, BTW15:53
=== rogpeppe3 is now known as rogpeppe
rogpeppehere's my current juju environment which i'm using to try out some stuff: http://paste.ubuntu.com/5719234/16:53
rogpeppei've done upgrade-charm a few times16:53
rogpeppedimitern: ^16:53
rogpeppedimitern: seems to be working well!16:53
dimiternrogpeppe: good to hear! :)16:53
rogpeppedimitern: this was the script i used to set things up: http://paste.ubuntu.com/5719240/16:54
dimiternrogpeppe:  status looks nicer as well16:54
rogpeppedimitern: yeah, it's good to see it working16:54
rogpeppedimitern: hmm, actually i'm not entirely sure it is working16:56
dimiternrogpeppe: i can see the hook failed16:56
rogpeppedimitern: actually i don't think it did16:56
rogpeppedimitern: oh, it did16:56
rogpeppedimitern: i ssh'd in to the one machine that it didn't fail on!16:57
rogpeppedimitern: two "juju resolved" invocations later and it's all running17:00
dimiternrogpeppe: nice!17:00
rogpeppedimitern: yeah it feels really good to just play with it a bit17:01
rogpeppedimitern: just found a bug in juju get though17:01
rogpeppe% juju get logging17:02
rogpeppeerror: constraints do not apply to subordinate services17:02
dimiternrogpeppe: oh?17:02
rogpeppeinterrresting error17:02
dimiternrogpeppe: indeed17:02
rogpeppedimitern: it only happens when doing juju-get on the subord17:03
rogpeppedimitern: ha, found it i think17:04
rogpeppedimitern: yup17:05
rogpeppedimitern: fixed.17:07
rogpeppedimitern: am sorely tempted to push the fix :-)17:07
dimiternrogpeppe: what was it?17:07
rogpeppedimitern: in statecmd.ServiceGet, it calls svc.Constraints without checking if the service is principal or not17:08
rogpeppedimitern: personally i'd be tempted to make Service.Constraints return a zero constraints if the service is subord17:09
rogpeppedimitern: rather than an error17:09
rogpeppedimitern: hmm, another bug in juju get17:10
rogpeppedimitern: it doesn't appear to print default values17:10
dimiternrogpeppe: hmm.. well, bugs will appear anyway :) good that we have some time now to actually test it and find them17:11
rogpeppedimitern: definitely17:11
rogpeppedimitern: and none of these are show-stoppers17:11
dimiternrogpeppe: yeah17:12
rogpeppehmm, i thought juju get was supposed to work now17:25
rogpepperight, that's me done17:41
rogpeppeg'night all!17:41
=== deryck is now known as deryck[lunch]
=== deryck[lunch] is now known as deryck
thumperhmm... forgot to close irc last night21:01
thumperoh well,21:01
thumpermorning21:01
mgzmornin'21:05
dimiternmorning mgz, thumper21:22
dimitern;)21:22
=== ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: danilos | Bugs: 2 Critical, 61 High - https://bugs.launchpad.net/juju-core/
thumperdimitern: why start log messages with lower case?21:46
fwereadethumper, convention23:18
thumperoh hai fwereade23:18
fwereadethumper, heyhey23:18
thumperfwereade: good timing,23:18
thumperfwereade: https://codereview.appspot.com/8849043/ just being updated23:18
fwereadethumper, cool23:19
thumperfwereade: hmm...23:28
thumper$ juju bootstrap23:28
thumpererror: cannot find tools: use of closed network connection23:28
thumper$ juju version23:28
thumper1.9.14-raring-amd6423:28
thumperthat is the package version23:28
thumperfwereade: I don't suppose you could help me test the serialization?23:30
thumperfwereade: like how to make some fake charms and fake subordinates that just take time :)23:31
thumperfwereade: try and force contention23:31
=== wedgwood is now known as wedgwood_away
fwereadethumper, hum, that raring thing is not nice23:31
fwereadethumper, we have been unable to adequately characterise it23:31
thumperrunning with -v23:32
thumperalso, if I use the raring mongodb tests fail, with tarball, it works23:32
thumper2013/04/19 11:32:36 ERROR command failed: cannot find tools: Get https://s3.amazonaws.com/juju-c54985419ee80c98531550e15fdcc6a8/?prefix=tools%2Fjuju-1.&delimiter=&marker=: remote error: handshake failure23:33
fwereadethumper, AFAIWCT it is coming out of s3 somehow, very much more in some regions than others, and possibly varying by client series23:33
fwereadethumper, handshake failures appear to Just Happen23:33
fwereadethumper, I know this is shit23:34
fwereadethumper, but IME they have never progressed beyond a mild annoyance23:34
fwereadethumper, huh, was the mongo package used in 1.9.14?23:35
* thumper tries again then23:35
thumperI think the terminal running juju is using the packaged mongo too23:35
thumperno, using tar ball23:35
fwereadethumper, that shouldn't be hitting mongo except via mgo23:36
fwereadethumper, the tests will just use whatever's on your path23:36
* thumper nods23:36
thumpergot connection closed that time23:36
thumper2013/04/19 11:36:26 ERROR command failed: cannot find tools: use of closed network connection23:36
* thumper runs of local version23:37
fwereadethumper, assuming for now that you never see handshake failures, do you ever see anything other than closed connections to s3?23:37
thumpernot backage23:37
thumperI get it with trunk too23:37
fwereade!23:37
thumperI can't bootstrap at all23:38
fwereadeok we have a reproducible case with actual source23:38
fwereadehallelujah23:38
fwereademay I ask you to log the heel out of what is happening there?23:38
thumperwhy does our logging not give us file and line numbers?23:38
* thumper switches to trunk, and tries again23:38
thumper2013/04/19 11:39:31 INFO environs: reading tools with major version 123:40
thumper2013/04/19 11:39:37 INFO environs: falling back to public bucket23:40
thumper2013/04/19 11:39:37 ERROR command failed: use of closed network connection23:40
thumperfrom tip of trunk23:40
thumperwhere do I start logging?23:40
thumpermust be in the tools search right?23:40
fwereadethumper, sorry got distracted23:42
fwereadethumper, ok that is interesting23:42
fwereadethumper, I think we might want to delve into goamz/s3 and log in some detail23:42
thumperenvirons/tools.go line 26 fails23:42
fwereadethumper, what region are you in?23:42
fwereadethumper, that'd be a Storage.List(), right?23:43
thumperthe default23:43
thumperfwereade: right23:43
thumperhow do you format a bool for %s ?23:43
fwereadethumper, %v usually works23:43
fwereadethumper, there might be something I ought t ouse instead23:43
fwereadethumper, I'm sorry, but I think I have to sleep :(23:52
thumperfwereade: np, I'll keep digging23:52
fwereadethumper, the absolutely most valuable thing you can do is to mail that, indeed23:52
fwereadethumper, nail that23:52
thumperheh23:52
thumperfunny typo23:52
fwereadethumper, and possibly to just try deploying to ap-southeast-2 in case you're "lucky" enough to encounter difficulties finding instances from ids23:53
thumperok, I'll try that now23:53
fwereadethumper, I will finish your review first though23:53
fwereademramm, thumper seems to be able to repro one of the elusive issues against a source build23:54
thumperfwereade: changing region made no difference23:55
thumperfwereade: although unhelpfully, the region you are using isn't logged anywhere23:55
fwereademramm, in the light of mgz's comments re time, I think that we should not be releasing right now, but I would dearly appreciate clarity re the precise situation to which he alludes23:55
fwereadethumper, ha23:56
mrammfwereade: I have been in customer meetings all afternoon, so I'll try to check in on that now23:56
fwereadethumper, I will gladly LGTM just about anything that improves our logging23:56
thumperhmm, how am I supposed to log from within goamz??23:57
fwereadethumper, I know we produce a lot but we're cutting down the useless ones and so we have space for ones that might be more useful23:57
* thumper does printf23:57
fwereadethumper, what else does core/log import? I thought it should be doable directly without cycles23:58
fwereadethumper, (not saying it's nice, just expedient)23:58
thumperactually, should probably be able to import it23:58

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!