/srv/irclogs.ubuntu.com/2012/06/27/#juju-dev.txt

davecheneyniemeyer: thanks for your review00:11
niemeyerdavecheney: My pleasure00:11
davecheneythe reason the firewaller watches the environ is I imagined that would be how it woudl talk to the (unknown) firewalling service00:12
davecheneyas the Environ in this case represents ec200:12
davecheneyniemeyer: in the pythonic version, the provider managed openport/closeport00:12
niemeyerdavecheney: Duh.. it is obvious indeed, sorry00:13
davecheneyniemeyer: well, the facility doesn't exist on environ, yet00:14
niemeyerdavecheney: Which facility?00:14
davecheneyopen port/close port00:15
davecheneyit's there in goamz00:15
davecheneybut there is no generic interface in environs00:15
niemeyerdavecheney: Ah, I see, yep00:16
niemeyerdavecheney: It kind of sucks that we'll have to redo this soon to be machine-based00:17
niemeyerdavecheney: But it seems like the right thing to do, rather than get into a trip in unknown territory before we actually get things working at all00:17
davecheneyniemeyer: right o00:18
davecheneyniemeyer: i'm going to raise a bug on juju-core to define the open port / close port interface on environ00:18
davecheneyif that is ok00:18
niemeyerdavecheney: E.g. if we moved onto the machine agent, we actually have to watch relations too, at the machine level, so that we alter the firewall allowing them to go in and out00:18
niemeyerdavecheney: Sounds great00:19
niemeyerdavecheney: Thanks00:19
davecheneyniemeyer: the reason it's a new type, not an addition to the Provisioner, was a hope that the service could be moved between agents00:19
davecheneybut i don't know how realisitic that will be00:19
niemeyerdavecheney: I love the fact you splitted it out00:19
niemeyerdavecheney: It was a bad decision on the original implementation00:20
niemeyerdavecheney: Even if we can't move it as-is, we can move something, or we can even erase it by itself if nothing else00:20
davecheneyyup00:20
niemeyerdavecheney: Hmm00:25
niemeyerdavecheney: I don't think we should allow the environment name to change00:25
niemeyerdavecheney: We that value internally in the environment itself00:26
niemeyerWe use00:26
davecheneyniemeyer: ok, i'll do it another way00:26
niemeyerdavecheney: E.g. security groups00:26
davecheneyit wasn't being set at all00:26
niemeyerdavecheney: It feels slightly bad that we do this00:26
niemeyerdavecheney: It would be better to have that as an external property of the environment00:26
niemeyerdavecheney: But it's helpful to tell the user about what those machines he's running are about00:27
davecheneyniemeyer: the current thing that is blocking bootstrap is there are no tools in my s3 bucket00:27
davecheneyi can't see anything to generate those00:27
niemeyerdavecheney: Oh00:27
davecheney012/06/27 10:27:45 JUJU findTools searching for {{0 0 0} precise amd64} in []00:27
davecheneyerror: cannot start bootstrap instance: cannot find juju tools that would work on the specified instance: no compatible tools found00:27
niemeyerdavecheney: bootstrap -upload-tools?00:28
niemeyerdavecheney: bootstrap --upload-tools?00:28
niemeyerdavecheney: I think you're right, though00:28
niemeyerdavecheney: We do need to set the environment name, at least once00:29
davecheney2012/06/27 10:28:50 JUJU environs: putting tools tools/juju-0.0.0-precise-amd64.tgz00:29
davecheneyniemeyer: i'll find a nicer way to do it00:29
niemeyerdavecheney: I feel like your CL is a good step forward, actually00:29
niemeyerdavecheney: I'm probably mixing problems up00:29
niemeyerdavecheney: We should prevent the user from replacing the name, but that's not the way to do it00:29
davecheneyniemeyer: the name is an unfortunately property that comes out of the yaml file00:30
niemeyerdavecheney: I'll file a bug, and LGTM your change00:31
davecheneyniemeyer: ta, just a small cosmetic i found00:31
niemeyerdavecheney: This may well be what's preventing it from running00:31
davecheney2012/06/27 10:31:34 JUJU environs/ec2: started instance "i-acd0d3d5"00:32
davecheneyniemeyer: dude, it bootstrapped !00:35
niemeyerdavecheney: Woah!00:35
davecheneyniemeyer: i'm going to pull this branch apart into smaller pieces, but i think it's working00:35
davecheneyno idea if there is a PA at the other end yet00:35
davecheneyalso, juju status would be nice to have :)00:35
niemeyerdavecheney: +1 :)00:35
davecheneyniemeyer: time for a celebratory cup of tea00:36
niemeyerdavecheney: Feel free to start looking at it actually, if you ever get blocked/bothered00:36
niemeyers/bothered/bored/00:36
davecheneyniemeyer: ok00:36
davecheneyniemeyer: i'm not going to step on franks toes ?00:36
davecheneyscrew it, i'll just email juju dev and claim it00:36
davecheneyniemeyer: 2012/06/27 10:34:27 JUJU environs: putting tools tools/juju-0.0.0-precise-amd64.tgz00:38
davecheneyerror: cannot upload tools: cannot write file "tools/juju-0.0.0-precise-amd64.tgz" to control bucket: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.00:38
davecheney^ kinda unreliable, but that is australia for ya00:38
davecheneymaybe i'll try a different availability zone00:38
niemeyerdavecheney: Definitely not stepping on Frank's.. he's not looking at status ATM00:38
davecheneysorry, william00:38
* davecheney facepalm00:38
niemeyerdavecheney: That error was kind of weird00:39
niemeyerdavecheney: I don't think anyone is doing status yet00:39
niemeyerdavecheney: I've not seen that before.. worth investigating00:39
davecheneyniemeyer: anyway, time for a break00:40
niemeyerWoohay.. new battery arrived00:46
niemeyerIt's time for me to leave too00:46
niemeyerI'm late, actually00:46
niemeyerDinner with Go folks tonight.. back later00:46
davecheneyniemeyer: how was dinner ?05:04
* davecheney is a tad jealous05:04
niemeyerdavecheney: It was awesome indeed05:05
davecheneyniemeyer: i think i figured out why juju can't bootstrap in other avail zones05:05
niemeyerdavecheney: Not the full team, unfortunately05:05
davecheneyniemeyer: i bet rsc wasn't there05:05
davecheneyhe's elusive05:06
niemeyerdavecheney: Yeah, we had Rob and Andrew05:06
niemeyerdavecheney: He lives elsewhere.. I bet he'd join if he was around05:06
davecheneyhe lives in boston right ?05:06
davecheneyniemeyer: can I commit rogers branch ? or should I merge it into one I own to lbox propose ?05:11
niemeyerdavecheney: The right thing would be to branch from it and continue, or possibly even break it down in smaller chunks05:12
niemeyerdavecheney: Yeah, he lives in Boston, or next to it I think05:12
davecheneyniemeyer: i'll try to break it into two chunks, it's only 100 lines05:12
davecheneyactually less than 9005:12
niemeyerdavecheney: Yeah, but it's also incomplete, right?05:13
davecheneyit's very close05:13
davecheney[  102.246904] init: juju-provision-agent main process (4356) terminated with status 12705:13
davecheney[  102.246937] init: juju-provision-agent main process ended, respawning05:13
davecheney^ jujud isn't in the path, that is pretty much the only bit missing05:13
niemeyerdavecheney: I had the impression it was a spike mostly.. rog mentioned it had no testing05:14
niemeyerdavecheney: I confess to not have looked into it, though05:14
davecheneyhe's done some tests, > 50% of the change is in livetests05:15
niemeyerdavecheney: Ah, nice05:22
niemeyerdavecheney: I wonder what he was referring to then05:22
davecheneymaybe it was the environ name problem05:23
niemeyerdavecheney: Well, you're in a much better position than I am05:23
davecheneysince I fixed that it's been working pretty well for me05:23
niemeyerdavecheney: Whatever you feel should go to review, I'll be happy to look at05:23
davecheneyok05:23
davecheneyniemeyer: did my diagnosis of the s3 problem make sense ?05:24
davecheneyniemeyer: ubuntu@domU-12-31-39-0C-58-F1:~$ pgrep jujud -lf05:25
davecheney4539 /var/lib/juju/tools/juju-0.0.0-precise-amd64/jujud provisioning --zookeeper-servers localhost:2181 --log-file /var/log/juju/provision-agent.log05:25
niemeyerdavecheney: I think it does.. it's definitely an error we'll want to fix05:37
davecheneyi think it should be pretty easy to add support for that05:38
niemeyerdavecheney: My surprise was with the fact Amazon refuses to take a request on a region-specific endpoint because the payload doesn't mention the region-specific endpoint name *as well*05:38
niemeyerdavecheney: Woah? Provisioner running?05:39
davecheneyyes, almost05:39
davecheneyi think i got initzk working as well (path issues)05:39
davecheneybut it's blocked waiting on the right value for /environment05:39
davecheneyniemeyer: [zk: localhost:2181(CONNECTED) 2] ls /05:43
davecheney[services, charms, relations, zookeeper, initialized, machines, units]05:43
davecheneyinitzk worked, just missing /environment05:43
davecheneyniemeyer: i don't understand why location constraint has to be xml, why can't it just be a header05:44
niemeyerdavecheney: That's expected05:44
niemeyerdavecheney: /environment comes from juju deploy05:45
niemeyerdavecheney: Well.. it doesn't even need to be a header05:45
niemeyerdavecheney: The endpoint URL itself is already a great indicator05:45
niemeyerdavecheney: There's no way to be talking to that endpoint URL and not wanting to.. well.. talk to it05:45
davecheneyniemeyer: i thought there was05:45
niemeyerdavecheney: Kind of strange.. either a very awkward decision, or I'm missing something more interesting05:46
davecheneythere is a note in one page I found that said there is a generic s3 location you can talk to05:46
davecheneybut they recommend not using it, because it will internally redirect your request05:46
niemeyerdavecheney: Sure, but we're not using it, AFAIK05:46
davecheneyeither way, it's a bit of a wart05:46
davecheneyniemeyer: so, juju deploy will be responsible for keeping /environment up to date ?05:47
niemeyerdavecheney: Yeah06:08
niemeyerdavecheney: Well, hopefully not up-to-date06:08
niemeyerdavecheney: Only the first time06:08
niemeyerdavecheney: THe Python version updates it every time, but that's a mistake06:08
davecheneyniemeyer: ok, so i should not make the bootstrap process seed /environment06:09
niemeyerdavecheney: It's tricky because to do that we'd have to wait until the environment bootstrapped06:11
niemeyerdavecheney: Chicken and egg06:11
niemeyerdavecheney: We don't want to send credentials over via user-data06:11
davecheneyniemeyer: right, so to confirm, the cloud-init script is not secure ?06:15
niemeyerdavecheney: Right, it puts thing up in an address that can be openly accessed from within the machine06:29
fwereadeniemeyer, davecheney: heyhey06:40
davecheneyfwereade: howdy06:40
fwereadeniemeyer, perhaps I'm missing something re: https://codereview.appspot.com/6335057/diff/3001/state/presence/presence.go#newcode39706:40
fwereadeniemeyer, but it seems to me that we can only guarantee absence notifications from the watches if we don't stop the childLoops at all06:42
fwereadebah06:42
fwereadeok, I'm off to take laura to nursery06:43
davecheneyyeah, i'm off soon as well06:45
niemeyerdavecheney: Yo06:54
niemeyerdavecheney: Please feel free to tackle the location bug if you're up for it06:54
niemeyerdavecheney: I'll certainly have a low attention level in the next few days with I/O going by06:54
niemeyerTime to head to bed now06:56
niemeyerdavecheney: Have a good time there, and talk to you tomorrow06:56
niemeyerNight all06:56
TheMueMorning.07:31
fwereadeTheMue, heyhey07:33
fwereadeTheMue, any objection to me just straight-up deleting relationUnitWatcher? after the meeting yesterday we have a better plan for doing essentially the same stuff10:45
TheMuefwereade: No problem, so I'll just don't touch it when doing the Deleted/Removed change.10:46
fwereadeTheMue, I consider this trivial enough to go straight in with a cursory review; sound sane?10:47
fwereadeTheMue, I'll be proposing in a couple of mins...10:47
TheMuefwereade: Does the mimic also may relate the ServiceUnitsWatcher?10:47
fwereadeTheMue, sorry, can't parse10:47
TheMuefwereade: The way you will do it in future, will this be interesting for the service units watcher too?10:48
fwereadeTheMue, ah sorry; I don't *think* so10:50
fwereadeTheMue, I don't think that should be considering agent presence10:50
fwereadeTheMue, but I may not have thought it through properly10:50
fwereadeTheMue, should it?10:50
TheMuefwereade: OK, I'll take a look when you proposed it. ;)10:50
TheMuefwereade: No, it's only that both are interested in units, but in different contexts.10:51
fwereadeTheMue, yeah, this is specifically unit *relation* presence10:51
TheMuefwereade: ic10:51
fwereadeTheMue, (the better plan is to use the forthcoming presence.ChildrenWatcher)10:51
fwereadeTheMue, https://codereview.appspot.com/635104610:59
fwereadeTheMue, should be trivial :)11:00
TheMuefwereade: LGTM, those pure red ones are pretty simple. ;)11:01
fwereadeTheMue, cheers; agree ok to submit directly? :)11:01
TheMuefwereade: If the removed watcher isn't yet used then it's ok.11:02
fwereadeTheMue, don't worry, I ran the tests :)11:02
fwereadeTheMue, submitting now11:02
fwereadeTheMue, btw, if you have another couple of moments11:04
fwereadeTheMue, it seems to me that the watcher use in cmd/jujud is a bit inconsistent/idiosyncratic11:04
fwereadeTheMue, I would appreciate your opinion on whether it's justified, or if we should clean it up11:05
fwereadeTheMue, by which I mainly mean moving stopWatcher and mustErr into state/watcher, and using them consistently11:05
TheMuefwereade: i take a look11:06
fwereadeTheMue, cheers11:07
TheMuefwereade: hmm, do you have a pointer for me?11:07
fwereadeTheMue, cmd/jujud/machine.go:9311:09
fwereadeTheMue, I'm pretty sure that it's wrong to let a tomb.ErrStillRunning slip through11:09
fwereadeTheMue, and in line 80 we discard errors11:09
fwereadeTheMue, I haven't looked at provisioner properly but I'm pretty sure it suffers similar problems11:10
fwereadeTheMue, it may or may not be justifiable to rearrange NewMachiner such that it follows the usual model -- ie start the watcher before we start the loop11:12
fwereadeTheMue, IMO it's a cleaner way of doing things, and is anyway better in the context of juju because it's more like the model we use throughout state11:13
TheMuefwereade: yes, this uncommon way may lead to misunderstandings11:13
fwereadeTheMue, cool, I'll have a go at that; we can see what niemeyer thinks when he;s around11:13
TheMuefwereade: yes11:14
TheMuefwereade: lunchtime11:14
fwereadeTheMue, cool, enjoy11:14
Aramhi.11:41
fwereadeAram, heyhey11:47
fwereadeTheMue, it crosses my mind that *actually* the PA has it right wrt the watcher... there doesn't ever seem to be a good reason to create a subwatcher outside the loop11:49
fwereadeTheMue, or even to keep it in a field on a, er, superwatcher11:50
TheMuere11:51
TheMueAram: Hi11:51
TheMuefwereade: Maybe, it only puzzles me if something is different than in most/all other places.11:52
fwereadeTheMue, I can't see anywhere a watcher field is used outside a loop method11:52
TheMuefwereade: Maybe it's my old OO as well as Erlang thinking that I do most initialization in a constructor and use the loop method only as loop.12:21
fwereadeTheMue, IMO a field that's used in only one method should not really be a field ;)12:22
fwereadeTheMue, anyway, I'm experimenting with what might be a noticeable simplification of the stuff in state/watcher.go, I'll let you know if it works out :)12:22
TheMuefwereade: But then I would not call the method loop(). I often name them backend() in my projects. ;)12:23
fwereadeTheMue, I would personally gravitate towards run() :)12:23
TheMuefwereade: That's the job of the whole program: come on, run, Forrest ...12:24
fwereadeTheMue, haha12:24
fwereadeTheMue, making this change has made some of the state watchers start to look a bit inconsistent to me: an awful lot of them seem to me to be at risk of sending "changes" that don't actually represent changes12:46
fwereadeTheMue, I rather consider that to be poor behaviour; opinion?12:47
fwereadeTheMue, basically all of them seem to do what FlagWatcher did before I suggested changing it yesterday12:49
fwereadeTheMue, excluding the Machine ones, that is12:51
TheMuefwereade: Hmm, maybe they should be fixed  then one-by-one in small changes.12:51
fwereadeTheMue, yeah, that sounds sensible12:51
TheMuefwereade: We've done it this way for the error messages in state.12:52
fwereadeTheMue, sorry, expand please12:52
fwereadeTheMue, ah, how we make the changes12:52
TheMuefwereade: +112:53
TheMuefwereade: yep12:53
TheMuefwereade: I think the should be fixed to work exactly as expeted.12:53
fwereadeTheMue, cool12:53
fwereadeTheMue, I need to think about what I've done a little more but it feels like a big win ATM12:54
TheMuefwereade: great12:54
fwereadeTheMue, I'll see what niemeyer thinks before I start hanging more changes off it though ;)12:54
TheMuefwereade: I'm just doing the last changes for the service unit watcher. niemeyer had a very good hint how to test it more elegant.12:55
fwereadeTheMue, yeah, I saw that12:55
fwereadeTheMue, sounds good12:55
TheMuefwereade: it's so much clearer now.12:55
fwereadeTheMue, sweet -- that style does make my head hurt a bit sometimes :)12:55
TheMuefwereade: it still uses test tables, but more simple ones12:56
fwereadeTheMue, perfect12:57
TheMuefwereade: will now propose it and then start a new branch for the deleted to removed change12:57
* fwereade worries that that change will be a real hassle to merge with his current work12:57
fwereadeTheMue, can you wait a few mins? I'll propose what I have -wip so you can (1) tell me if it's sane and (2) decide whether the conflicts will be worth the hassle12:58
TheMuefwereade: Which change you mean, the first one or the deleted/removed one?13:02
fwereadeTheMue, added/removed13:02
fwereadeTheMue, (sorry, remind me what the first one is?)13:02
TheMuefwereade: The fixes after the review of the service units watcher.13:04
TheMuefwereade: They are proposed now.13:04
fwereadeTheMue, I'm happy to suck up the conflicts on that one myself13:04
TheMuefwereade: The work on the deleted-to-be-named-removed will start now.13:04
fwereadeTheMue, but Added/Removed might hurt13:05
TheMuefwereade: This is why I will do it in an extra branch. There are influences in more packages than just state.13:05
fwereadeTheMue, true13:06
TheMuefwereade: But I'll wait for your proposal.13:06
fwereadeTheMue, https://codereview.appspot.com/634704513:08
fwereadeTheMue, the reason I worry slightly is that it totally guts state/watcher.go13:09
fwereadeTheMue, actually, looking at it, I doubt Added/Removed will hurt that much13:10
fwereadeTheMue, but I'd still appreciate your preliminary opinion13:10
fwereadeTheMue, the important bit is the changes to state/watcher.go, most of the rest can probably be a separate CL13:10
TheMuefwereade: In the moment the field is renamed in watcher.go any user of this field has to be renamed too.13:11
TheMuefwereade: So it may be better type by type instead file by file.13:12
fwereadeTheMue, you mean the Added/Removed change?13:12
fwereadeTheMue, my CL does not alter observable behaviour AFAICT13:12
TheMuefwereade: Yes13:12
fwereadeTheMue, cool13:12
TheMuefwereade: The other one - don't raise events for unwanted content changes - is a different topic and should be discussed with niemeyer13:14
fwereadeTheMue, yeah, I haven't even started on that13:14
TheMuefwereade: Maybe you should file it first.13:14
fwereadeTheMue, hopefully I'll get a chance to see him and discuss it tonight13:15
TheMuefwereade: So which change of mine do you think could conflict yours?13:15
fwereadeTheMue, I decided it probably wouldn't hurt too much in the end13:15
fwereadeTheMue, there will be conflicts but they'll be trivial13:15
fwereadeTheMue, feared they might be ugly before I looked at the actual diff13:16
TheMuefwereade: ah, ok, understand13:17
fwereadeTheMue, anyway, does that change look worth it to you?13:17
TheMuefwereade: i'm not yet through, one moment13:18
fwereadeTheMue, MW and MUW are not finished there13:20
fwereadeTheMue, you should be able to ignore them and still have a clear idea of what's involved13:20
TheMuefwereade: Mostly LGTM, I only have troubles with the panic() in MustErr().13:27
fwereadeTheMue, that was already there :)13:27
TheMuefwereade: Maybe, but not by me. ;)13:28
fwereadeTheMue, the idea is that if a subwatcher's channel is closed while the watcher is reading from it, it indicates a fundamental logic error13:28
TheMuefwereade: The question is: Is this situation simply an error itself and recoverable or do we really need to tear down the program?13:28
fwereadeTheMue, it means that we, as coders, are demonstrably on crack13:29
fwereadeTheMue, ie not recoverable IMO13:29
TheMuefwereade: If it can't happen by any runtime situation then I'm fine with hit.13:29
TheMueit13:29
fwereadeTheMue, that's the idea13:29
TheMuefwereade: This is what I really like in Erlang. Here a supervisor would recognize this crash and start the needed tasks to recover the system (which sometimes is only the restart and registration of a coroutine (process) and sometimes may be the stopping and restarting of large process trees).13:33
fwereadeTheMue, but if the error is "the programmers are demonstrably failing basic logic" then I think it's better to fail hard AAP13:34
TheMuefwereade: supervisors can also supervise other ones, so it cascades if one fails during restart13:34
fwereadeASAP13:34
TheMuefwereade: yes, indeed, if it is definitely not recoverable and a logical error13:35
fwereadeTheMue, if a watcher has closed its change channel while its owner is still making use of it, that is IMO evidence of fundamental crackfulness13:36
twobottuxaujuju: Security groups creation in EC2 <http://askubuntu.com/questions/156715/security-groups-creation-in-ec2>13:37
mthaddonhey folks, does anyone know who's responsible for maintaining http://jujucharms.com/ ?14:37
james_wmthaddon, that's hazmat's14:42
mthaddonjames_w: ah, thx14:43

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!