| mramm | How goes? | 02:04 |
|---|---|---|
| jam | morning wallyworld_, I hope your weekend went well. | 05:32 |
| wallyworld_ | jam: hello, very busy but good | 05:35 |
| mramm | Anybody about that can give me a quick update on openstack progress, subordinates, or TLS? I'm putting together a status update for internal folks, but also thinking about sending out e-mail with some status info to the list. | 06:29 |
| mramm | I seem to have found a time where all the core guys are not online... Bully for me! | 06:30 |
| mramm | ok, signing out for a bit, back later this evening when EU folks are up ;) | 06:46 |
| TheMue | Morning. | 07:13 |
| rogpeppe | morning all | 08:29 |
| rogpeppe | fwereade: have a nice holiday? | 08:29 |
| fwereade | rogpeppe, thank you, yes :) | 08:30 |
| TheMue | rogpeppe, fwereade: Hiya. | 08:30 |
| rogpeppe | fwereade: i got TLS working live BTW | 08:30 |
| fwereade | rogpeppe, ate in cafes, went to beach, the usual late-november things :) | 08:30 |
| fwereade | rogpeppe, awesome! | 08:30 |
| fwereade | TheMue, morning | 08:30 |
| rogpeppe | fwereade: the late november thing here currently is sheltering from the rain and squelching through mud | 08:31 |
| TheMue | fwereade: Walking to the beach is definitely a benefit of your home town. Here it is too windy and cold. | 08:31 |
| TheMue | rogpeppe: +1 | 08:31 |
| fwereade | rogpeppe, TheMue: :) | 08:31 |
| * TheMue had some nice malts on Friday, warming from inside. | 08:32 | |
| fwereade | whoops, need to pop out a mo | 08:39 |
| fwereade | (I *think* I just wrote something nice...) | 08:39 |
| rogpeppe | yay! all live tests pass with TLS enabled... | 09:08 |
| TheMue | rogpeppe: Cheers. | 09:09 |
| fwereade | rogpeppe, cool! | 09:20 |
| rogpeppe | fwereade: of course, it'll probably take weeks to get the CLs through :-| | 09:21 |
| rogpeppe | fwereade: there's one little thing i encountered that i'm not sure of the best way to fix | 09:21 |
| fwereade | rogpeppe, quite, but this is a significant milestone nonetheless | 09:21 |
| fwereade | rogpeppe, oh yes? | 09:21 |
| rogpeppe | fwereade: when you first make the connection to the state, you might succeed in connecting before the initialisation process has set up the initial password | 09:22 |
| rogpeppe | fwereade: so you can get an "unauthorized" error | 09:22 |
| fwereade | rogpeppe, ha, but usually unauthorized means that something's really wrong | 09:23 |
| rogpeppe | fwereade: there's only a very short time window for it to happen | 09:23 |
| rogpeppe | fwereade: yeah | 09:23 |
| fwereade | rogpeppe, grar | 09:23 |
| rogpeppe | fwereade: so i'm wondering if we should only open the mgo port after bootstrap-state has completed | 09:23 |
| fwereade | rogpeppe, +1 | 09:23 |
| rogpeppe | fwereade: that involves a few icky special-case hacks though | 09:24 |
| rogpeppe | fwereade: for instance, i think we'd need a new entry point in Environ | 09:24 |
| rogpeppe | fwereade: and... oh bugger, we can't do it | 09:25 |
| fwereade | rogpeppe, what if we were to bunch all the mgo setup stuff up as a Stater task? | 09:25 |
| rogpeppe | fwereade: we can't change ports | 09:25 |
| rogpeppe | fwereade: before the first client connection | 09:25 |
| fwereade | rogpeppe, oh ok | 09:25 |
| rogpeppe | fwereade: because we haven't got the enviroment credentials yet | 09:25 |
| rogpeppe | fwereade: there's another possibility actually | 09:26 |
| rogpeppe | fwereade: and maybe it's not a bad one, as we're going to be doing something similar later with the API | 09:26 |
| rogpeppe | fwereade: we could start a proxy process | 09:26 |
| rogpeppe | fwereade: that just port-forwards | 09:26 |
| rogpeppe | fwereade: and start *that* after bootstrap-state has completed. | 09:27 |
| fwereade | rogpeppe, sounds very sane to me | 09:27 |
| rogpeppe | fwereade: i think i'll leave it as a possibility for the moment - the retry logic is sufficient for the time being. | 09:28 |
| fwereade | rogpeppe, cool | 09:28 |
| mramm | Greetings from the opposite side of the world from my normal life | 09:28 |
| rogpeppe | mramm: yo! | 09:29 |
| mramm | I'm exactly 12 hours off normal schedule | 09:29 |
| rogpeppe | mramm: where are you? | 09:29 |
| mramm | so, instead of 1, 2, and 4 pm meetings | 09:29 |
| mramm | I have midnight through four in the morning meetings :) | 09:29 |
| mramm | Thailand | 09:29 |
| rogpeppe | mramm: for work or play? | 09:29 |
| mramm | visiting friends of my mom, and hanging out on the beach | 09:29 |
| rogpeppe | mramm: very nice! | 09:29 |
| rogpeppe | mramm: temperature? | 09:30 |
| mramm | I also bought a suit or two for work | 09:30 |
| mramm | hot | 09:30 |
| mramm | ;) | 09:30 |
| * rogpeppe is jealous | 09:30 | |
| * rogpeppe as he hears the rain drumming outside | 09:30 | |
| rogpeppe | mramm: you might be interested to know i got all the TLS stuff working. | 09:30 |
| mramm | about 32 degrees | 09:31 |
| mramm | and that's because it has cooled off a bit since yesterday | 09:31 |
| rogpeppe | mramm: nice. not *too* ridiculous | 09:32 |
| mramm | right | 09:32 |
| mramm | it has rained every day | 09:32 |
| mramm | but not for too long | 09:32 |
| mramm | and warm rain | 09:32 |
| rogpeppe | mramm: i hope you've been taking full advantage of the local cuising | 09:32 |
| rogpeppe | cuisine | 09:32 |
| mramm | so, today I'm working so that I can be ready for all those late night meetings | 09:32 |
| mramm | yea, I love me some thai food | 09:32 |
| mramm | got some basil thai fried rice from a street vendor for lunch | 09:33 |
| mramm | 40 baht ($1.20) | 09:33 |
| mramm | and it was delicious | 09:33 |
| mramm | anyway I'll be updating a few of the existing briefs, and adding a new one for API stuff | 09:34 |
| mramm | and I also created a new status report doc, so that I can pull from it to give big Mark updates on a regular basis | 09:35 |
| rogpeppe | davecheney: yo! | 09:40 |
| davecheney | rogpeppe: hey | 09:40 |
| rogpeppe | davecheney: since i'm telling everyone... i got TLS working. | 09:40 |
| davecheney | rogpeppe: congratulations!! | 09:43 |
| rogpeppe | davecheney: interesting test failure you saw last night | 09:47 |
| rogpeppe | davecheney: i think we should probably never rely on error messages from go | 09:48 |
| TheMue | *: https://codereview.appspot.com/6853075/ likes another review. | 10:19 |
| TheMue | fwereade: I somehow have a problem changing the original LXC configuration for testing. A possible solution may be a kind of chroot() and have an own /etc directory. But I'm not sure. Any other good idea is welcome. | 10:21 |
| fwereade | TheMue, hum: is it possible to just make the path configurable and settable for the tests? | 10:22 |
| fwereade | TheMue, much like the charm stor url in juju-core/charm IIRC | 10:22 |
| TheMue | fwereade: Yeah, should be possible. Good idea. | 10:23 |
| mramm | dimitern, etc: how are things going on the provider front? | 10:31 |
| dimitern | mramm: we're into refactoring the POC client code and writing test services (doubles of openstack api subset we're implementing) | 10:32 |
| mramm | cool | 10:33 |
| dimitern | mramm: we have identity auth service + client code mostly done, and the nova and swift packages taking shape | 10:33 |
| mramm | nice | 10:33 |
| dimitern | mramm: we also have a complete stub openstack provider ready in juju-core | 10:33 |
| mramm | awesome | 10:33 |
| dimitern | only needs its actual "meat" over the bones | 10:34 |
| mgz | dimitern, also known as the code that actually does stuff :) | 10:35 |
| jam | morning dimitern | 10:35 |
| dimitern | jam, mgz: morning | 10:37 |
| mramm | jam, mgz: Morning | 10:54 |
| rogpeppe | jam, mgz: what's the story with goose? is it going to start with merges from scratch again, or are we moving on from its current state? | 10:54 |
| rogpeppe | jam, mgz, dimitern: morning, BTW! | 10:55 |
| dimitern | rogpeppe: morning! well done for the TLS! :) | 10:56 |
| rogpeppe | dimitern: thanks. we'll see how long it takes to land :-) | 10:56 |
| rogpeppe | dimitern: at least i can see the light at the end of the tunnel... | 10:56 |
| jam | rogpeppe: moving on from the current state | 10:58 |
| rogpeppe | jam: ok, there are a few comments i'd like to make at some point, if that's ok | 10:58 |
| jam | rogpeppe: silence! the code is perfect as is, and I will hear no dissent. | 10:58 |
| * rogpeppe mumbles | 10:59 | |
| jam | rogpeppe: I would be happy to get feedback. | 10:59 |
| * rogpeppe feels the lack of a codereview CL to click on :-) | 10:59 | |
| jam | rogpeppe: can you propose the code against a blank tree? | 11:00 |
| jam | I think reitveld just works from a patch anyway. | 11:00 |
| jam | (diff) | 11:00 |
| jam | bzr diff -r 0..-1 would give it to you | 11:00 |
| rogpeppe | jam: yeah, i'll give that a go | 11:00 |
| mgz | the puns! the puns! | 11:02 |
| TheMue | fwereade: Done, thanks for the good hint. | 11:04 |
| mramm | alrght everybody I'll be back in a few hours. Going to get some food and rest up for tonight's marathon of meetings starting at midnight. | 11:04 |
| niemeyer | Good morning jujuers | 11:06 |
| rogpeppe | niemeyer: yo! | 11:07 |
| niemeyer | rogpeppe: Hey man, how was the weekend? | 11:07 |
| fss | niemeyer: morning :-) | 11:08 |
| rogpeppe | niemeyer: great. i did a fun gig in St Andrews, and spent the train journey up there getting TLS working | 11:08 |
| rogpeppe | niemeyer: which it now does, all live tests pass | 11:08 |
| niemeyer | rogpeppe: Ohhh, sweet | 11:08 |
| niemeyer | fss: Heya | 11:09 |
| fss | niemeyer: yesterday I tried go version | 11:09 |
| mramm | niemeyer: good morning! | 11:10 |
| niemeyer | mramm: Heya, how were holidays? | 11:10 |
| niemeyer | fss: Nice, how was it? | 11:10 |
| fss | niemeyer: looks neat :-) I'm glad most of proxy stuff worked because go's http client supports http_proxy and https_proxy | 11:11 |
| mramm | niemeyer: good. I have another week here on the beach. | 11:11 |
| TheMue | niemeyer: Hiya | 11:11 |
| mramm | niemeyer: but am working a bit, just to stay on top of meetings and whatnot | 11:12 |
| niemeyer | mramm: Oh, so you're still mostly off this week? | 11:12 |
| mramm | yea | 11:12 |
| niemeyer | TheMue: Heya | 11:12 |
| niemeyer | mramm: Cool | 11:12 |
| mramm | today I'm working all day | 11:12 |
| niemeyer | fss: So did it all work? | 11:12 |
| fwereade | early lunch, bbiab | 11:13 |
| niemeyer | mramm: How was the 6AM meeting? :-) | 11:13 |
| fwereade | heya niemeyer | 11:13 |
| niemeyer | fwereade: Heya | 11:13 |
| fss | niemeyer: almost | 11:13 |
| fss | niemeyer: it doesn't work in the agent, because it would need to be exported in the upstart config file (using env) | 11:13 |
| fss | niemeyer: I don't know if there is another way | 11:14 |
| fss | niemeyer: (regarding http_proxy and https_proxy env vars) | 11:14 |
| niemeyer | fss: Ah, I see | 11:14 |
| rogpeppe | jam: https://codereview.appspot.com/6844087 | 11:14 |
| mramm | niemeyer: which one? I'm on UTC/GMT +7 hours | 11:15 |
| niemeyer | mramm: The one 3h ago | 11:15 |
| jam | rogpeppe: so that looks like it worked, but did it announce itself to anyone to actually follow it? | 11:15 |
| rogpeppe | jam: nope | 11:15 |
| mramm | niemeyer: interesting, that wasn't on my calendar | 11:15 |
| rogpeppe | jam: or... i dunno | 11:15 |
| mramm | I do have one for midnight tonight, and then another at 1am ;) | 11:15 |
| rogpeppe | jam: probably not | 11:15 |
| rogpeppe | jam: but we can advertise it if we like | 11:16 |
| jam | rogpeppe: well, the big thing is that when you actually make comments, I would like to see them. | 11:16 |
| niemeyer | mramm: Hmm.. I think I'm on crack.. I see it's now scheduled to 17 UTC | 11:16 |
| fss | niemeyer: we'd like to help getting it to work on sa-east-1, terminate-machine, destroy-service, vpc and proxy (our scenario) | 11:17 |
| mramm | niemeyer: no worries we all hit the crack a little bit too hard sometimes ;) | 11:17 |
| niemeyer | mramm: 1AM? Ugh | 11:18 |
| rogpeppe | jam: if you comment on the CL, you'll see any further comments | 11:18 |
| niemeyer | fss: The proposal of a sprint is still up | 11:18 |
| jam | rogpeppe: which is... a bit non-ideal, but what I just did. | 11:18 |
| fss | niemeyer: nice, next friday is our deadline for getting tsuru working in vpc, so I will keep working in the python fork today and tomorrow, and them I will start helping out go version | 11:21 |
| mramm | niemeyer: That's what I get for going to vacation 12 hours removed from my normal tz! | 11:21 |
| niemeyer | mramm: Yeah, a bit intense :) | 11:23 |
| niemeyer | mramm: Well, to be honest though, that's what you get for *working* during vacation :-) | 11:23 |
| niemeyer | fss: The proposal won't be up for much longer, though.. if you're actually interested, we have to sort that out today or tomorrow | 11:24 |
| niemeyer | fss: Otherwise we'll be doing it remotely, which is no big deal, but will take a bit longer | 11:24 |
| fss | niemeyer: hmm, I don't think we will be able to sort it out today or tomorrow, we will probably work remotely. I'll let you know | 11:26 |
| mramm | niemeyer: yea, but if I *didn't* work during vacation, what would I do ;) | 11:28 |
| niemeyer | Btw, I've upgraded my network connection to amazing 10Mbps on Friday, which means my video should suck a bit less now.. it took some troublesome cabling effort, but it worked out in the end. | 11:39 |
| niemeyer | "amazing" because I didn't quite trust it to work for real here, but apparently it's quite reasonable | 11:40 |
| niemeyer | fwereade: Please ping me when you're back | 12:10 |
| niemeyer | fwereade: I'd like to sort out the machine id situation somehow | 12:10 |
| fwereade | niemeyer, heyhey | 12:10 |
| fwereade | niemeyer, ok, I have a pair of branches that I am just about to push on the strength of which you may find it helpful to judge | 12:11 |
| niemeyer | fwereade: Okay | 12:11 |
| niemeyer | fwereade: I have an idea I'd like to talk as well.. | 12:11 |
| niemeyer | fwereade: Can we have a quick call? | 12:11 |
| fwereade | niemeyer, ofc | 12:11 |
| fwereade | niemeyer, will you invite or shall I? | 12:12 |
| niemeyer | fwereade: Sending | 12:12 |
| TheMue | kunchtime | 12:31 |
| mgz | is that a special german custom? :) | 12:33 |
| === niemeyer_ is now known as niemeyer | ||
| fss | niemeyer: just confirmed, it's not possible to arrange that sprint | 12:43 |
| niemeyer | fss: Cool | 12:43 |
| niemeyer | fss: Thanks | 12:44 |
| fss | =/ | 12:46 |
| rogpeppe | jam: i've made some comments. https://codereview.appspot.com/6844087/ | 12:51 |
| rogpeppe | jam: it seems a pity that you can't push changes to that CL though. | 12:52 |
| rogpeppe | jam: as another possibility (too late!) i was wondering if it might be better to make a minor change to each file (e.g. a comment "// pending review") and propose that as a CL to the existing trunk. then people could comment on any file, and changes would be targetted to trunk. | 12:53 |
| jam | rogpeppe, must I remind you: (2:58:50 PM) jam: rogpeppe: silence! the code is perfect as is, and I will hear no dissent. :) | 12:56 |
| mgz | too many camels | 12:56 |
| jam | thanks for the comments, the inline comments is an interesting idea | 12:56 |
| rogpeppe | jam: oops | 12:56 |
| jam | I'll look over your comments and see if it is something that can be addressed quickly, or not. | 12:56 |
| rogpeppe | jam: there are many small issues | 12:56 |
| mgz | the top few I generally agree with and a few have been mentioned/are being fixed | 12:57 |
| rogpeppe | jam: it would have been nicer if we could have commented as the branch was being built up | 12:57 |
| mgz | so, less redundant naming of things in packages, constants where just using the string would do, and so on | 12:57 |
| mgz | rogpeppe: you were sitting right next to us :D | 12:57 |
| rogpeppe | mgz: i saw no proposals :-) | 12:57 |
| rogpeppe | mgz: yeah, just lots of go-idiomatic things, mostly | 12:58 |
| niemeyer | fss: What was the issue, out of curiosity? | 12:59 |
| mgz | rogpeppe: okay, on the json error to go error thing, this is what I want to get sorted | 13:00 |
| rogpeppe | mgz: that was probably the most substantive comment, yeah | 13:01 |
| mgz | but, discussing it this morning, I still don't really get the go conventions for rich error instances | 13:01 |
| rogpeppe | mgz: basically you can document that a function may return a particular error type | 13:01 |
| rogpeppe | mgz: then the caller can dynamically check (if it wants to) whether such an error type was actually returned | 13:02 |
| fwereade | TheMue, hmm, my understanding of something is fatally flawed: why is firewaller using a MachinePrincipalUnitsWatcher? | 13:02 |
| rogpeppe | mgz: there are actually three conventions | 13:02 |
| mgz | but you can't have a type, and then a subtype of that, as I understand is, so no `except EnvironmentError:` kind of thing | 13:02 |
| fwereade | TheMue, I thought the whole point of the MachineUnitsWatcher was to do exactly what the fw needed? | 13:02 |
| rogpeppe | mgz: you can return several different possible types | 13:03 |
| mgz | so, having a general OpenstackInteractionError then specific types that derive from that is no go | 13:03 |
| fss | niemeyer: I don't know. We all should pray for transparency :-( There was an excuse, but I'm not sure that was the reason | 13:03 |
| rogpeppe | mgz: e.g. EnvironmentError, HttpError, JsonError, etc | 13:03 |
| rogpeppe | mgz: then the caller can do: if e, ok := err.(*EnvironmentError); ok { ... we got an environment error } | 13:04 |
| mgz | the point of EnvironmentError in python is it catches OSError (and subclasses) and IOError (and subclasses) | 13:04 |
| rogpeppe | mgz: well, we don't do subclasses. you'd need to decide which errors come within which category. | 13:05 |
| rogpeppe | mgz: (errors can of course contain other errors) | 13:05 |
| rogpeppe | mgz: sorry, i'm not very familiar with python or its class hierarchy | 13:05 |
| mgz | the other question... | 13:06 |
| mgz | in python I'd make an exception class with several attributes | 13:06 |
| mgz | and a __str__ method that took those attributes and presented something pretty | 13:06 |
| mgz | so, you could do `err.code == 404` but when propogates you still get the nice stringification | 13:07 |
| rogpeppe | mgz: that sounds very similar to a struct type with an Error method | 13:07 |
| mgz | where's a good go example for that? | 13:07 |
| rogpeppe | mgz: egrep for ' Error\(' | 13:07 |
| rogpeppe | mgz: well... | 13:07 |
| rogpeppe | mgz: i'll fine you an example | 13:08 |
| rogpeppe | find | 13:08 |
| rogpeppe | mgz: basically, if you define an "Error() string" method on a type, it can be used as an error. | 13:08 |
| rogpeppe | mgz: so if you've got a struct type describing your error, you can write an Error method that produces a pretty string version of the error | 13:09 |
| mgz | okay, thanks. | 13:09 |
| mgz | I'll do such after lunch. | 13:09 |
| jam | rogpeppe: the OpenStackHTTPClient stuff is already in a branch to land, for some of the naming tihngs. | 13:10 |
| rogpeppe | mgz: example at random: look in go/scanner/errors.go in the go source tree | 13:10 |
| rogpeppe | jam: ok. | 13:10 |
| jam | rogpeppe: so what is the difference between fmt.Errorf and errors.New() ? It would seem New() won't let you ever put in custom formatting, while Errorf() is pretty obvious for that. | 13:11 |
| rogpeppe | jam: that's the only difference. errors.New is a teeny bit more efficient as it doesn't need to scan the string. | 13:12 |
| jam | It seems a bit odd to use both in the codebase (having to import errors just to get New, when most of the time you use fmt.Errorf() because you want extra context) | 13:12 |
| rogpeppe | jam: i suggested using errors.New because the errors package was already imported | 13:12 |
| rogpeppe | jam: it's not a significant issue though | 13:13 |
| niemeyer | fss: What was the excuse? | 13:14 |
| niemeyer | fss: If that's publc | 13:15 |
| niemeyer | public | 13:15 |
| TheMue | fwereade: I can't help you here. I used machine.WatchUnits() in my latest version before I added the global mode. At this time the change to the principal unit watcher has been introduced. | 13:15 |
| TheMue | fwereade: I think it may be a temporary change due to the work on the unit watcher. | 13:16 |
| fwereade | TheMue, ah, ok, I shall examine it further | 13:18 |
| fwereade | TheMue, cheers | 13:18 |
| niemeyer | jam: Do you have a moment before you go? | 13:19 |
| jam | rogpeppe: I'll read through your suggestions a bit more tomorrow. My son is indicating in no uncertain terms that its my EOD. | 13:19 |
| jam | niemeyer: I might have 30s or so | 13:19 |
| rogpeppe | jam: okeydokey | 13:20 |
| fss | niemeyer: msg :-) | 13:20 |
| niemeyer | jam: Cool, that won't be enough.. let's catch up tomorrow then | 13:20 |
| jam | niemeyer: works for me. | 13:21 |
| niemeyer | jam: Have a pleasant evening there | 13:21 |
| hazmat | niemeyer, re depart hooks, the pinger presence expiration won't trigger them? | 14:04 |
| niemeyer | hazmat: Nope | 14:04 |
| rogpeppe | anyone fancy having a look at some of my outstanding reviews? i'm sure a couple of them could be submitted if i had two LGTMs. | 14:07 |
| mgz | rogpeppe: the ones I reviewed I don't think I said lgtm but did mean it | 14:09 |
| rogpeppe | mgz: thanks. yeah, you're the first LGTM! | 14:09 |
| rogpeppe | mgz: just wondering: why do you define an interface type for each of the various clients (e.g. swift.Swift, swift.Nova, etc) ? | 14:29 |
| rogpeppe | s/swift.Nova/nova.Nova/ of course | 14:29 |
| mgz | which part of that question is surprising for you? I think the bit that's novel for me might not be what you mean... | 14:30 |
| mgz | needing an interface seems to be a neat way in go of providing a real implementation and a testing backend that support the same stuff. | 14:30 |
| mgz | but I suspect you mean why not just one interface for everything? | 14:30 |
| mgz | openstack exposes various seperate services, with different endpoints, so, for instance, you could have a deployment that had nova for compute, but no swift (as canonistack did for a long time) | 14:32 |
| mgz | so, we might for instance want a ceph object-store client as well, which we could then perhaps factor a common interface out of | 14:34 |
| TheMue | rogpeppe: Which CL do you want to be reviewed? | 14:36 |
| rogpeppe | mgz: sorry, only just saw your reply. (i generally don't notice things on irc unless directly addressed) | 14:37 |
| rogpeppe | TheMue: any of the ones in https://code.launchpad.net/juju-core/+activereviews with no prereqs would be a good start | 14:37 |
| rogpeppe | mgz: i mean both actually | 14:37 |
| rogpeppe | mgz: i don't see that defining an interface that provides exactly the same things as the type you're also defining is useful. | 14:38 |
| mgz | rogpeppe: right, it's only really useful when you have another type as well | 14:38 |
| rogpeppe | mgz: it just means that you have to do more work when changing type sigs, because you need to keep the interface type in sync too | 14:38 |
| rogpeppe | mgz: in general, in Go we define interfaces when we need them. | 14:39 |
| mgz | which I think is what the plan was over providing test versions, but there's still some debate over what we're trying to do there exactly | 14:39 |
| rogpeppe | mgz: it's not that usual to use interfaces in Go just to enable mocking. | 14:40 |
| mgz | fair enough, and we may do that all at the http level anyway | 14:40 |
| rogpeppe | mgz: that's the approach we took for ec2, and it seems to have worked ok | 14:40 |
| mgz | you seem to rely a lot on testing edge cases just against ec2 itself | 14:41 |
| mgz | that's less tractable with the myriad possible openstack deployments | 14:41 |
| rogpeppe | mgz: for example? | 14:41 |
| * niemeyer => lunch | 14:41 | |
| mgz | but there are also other ways to address that with a faked out server | 14:41 |
| rogpeppe | mgz: we run the "live" tests against the fake server too | 14:41 |
| mgz | dave cheney just fixed an issue where you get an odd error back from ec2 when in a different region and trying to access the public bucket | 14:43 |
| mgz | really, for that kind of thing with openstack, we want to test the specific error back from swift gets propogated to the client as something understandable so the user can fix their region | 14:44 |
| mgz | if you just have a well behaved pretend swift server, testing particular error cases means either adding params to it and spinning up a new one each test, or poking a running one to do something special next response | 14:45 |
| mgz | or I guess here, hardcoding some maic region value that both the test and the server expect to behave in a particular way | 14:46 |
| mgz | that starts to add a lot of complexity after the 50th quirk you want to test | 14:47 |
| mgz | rogpeppe: ^example :) | 14:47 |
| rogpeppe | mgz: we should definitely test that kind of thing locally. we happened to find the error when testing live, but there's no reason we can't use the local test server to check those kinds of issue. | 14:49 |
| rogpeppe | mgz: i'd provide a way to configure the test server so that it emulates one (or a set) of the possibilities | 14:49 |
| mgz | rogpeppe: I'm not seeing that being done in several merge proposals that have gone past, generally just the client code has been fixed | 14:49 |
| rogpeppe | mgz: i agree, that's a problem. | 14:50 |
| rogpeppe | mgz: but that doesn't mean the general approach is wrong | 14:51 |
| rogpeppe | mgz: what do you think of the idea of defining a common-subset interface across the various provider types? | 14:54 |
| mgz | rogpeppe: don't think it's practical | 14:55 |
| rogpeppe | mgz: there's no common functionality at all? | 14:55 |
| rogpeppe | mgz: from my brief glance, it looks as if there's at least some | 14:56 |
| mgz | might work for some bits like storage, which tends to be a thin wrapper around the underlying apis anyway, | 14:56 |
| mgz | which a few added frustrations in the design from s3isms | 14:57 |
| rogpeppe | mgz: it's entirely possible to have a common-subset API which still provides access to extended features | 14:57 |
| fwereade | rogpeppe, it's a question of convenience in a particular setting, though | 14:57 |
| mgz | openstack and ec2 should be able to share a fair bit, but maas doesn't look much like them outside of having somewhere to poke files | 14:57 |
| rogpeppe | mgz: oh sorry | 14:57 |
| rogpeppe | mgz: i wasn't talking about between openstack and ec2 | 14:58 |
| rogpeppe | mgz: i was talking about the various clients within openstack | 14:58 |
| rogpeppe | mgz: e.g. nova, swift | 14:58 |
| rogpeppe | ahem | 14:59 |
| mgz | rogpeppe: so, there's not much direct overlap at present, potential for s3-compat/swift/ceph aside | 14:59 |
| rogpeppe | perhaps i should google a bit before talking about something i don't know anything about :-) | 14:59 |
| mgz | but certainly services that provide roughly similar apis like that should be exposed through a common interface for usage that doesn't care about which one you're using for object-store | 15:00 |
| TheMue | rogpeppe: One LGTM with a small hint. | 15:00 |
| rogpeppe | mgz: it does look like there are potential types in common though. Entity, for example, is the same in each, no? | 15:00 |
| rogpeppe | TheMue: append(certPEM, keyPEM...) *can* touch certPEM | 15:01 |
| mgz | rogpeppe: the other thing along those lines is some compat stuff where apis used to be in nova but have been split out in newer versions, clients ideally shouldn't care where the volume management bits are, just that a volume management interface is available | 15:01 |
| mgz | rogpeppe: some apis take uuids that refer to the same objects, I'm not sure what other things apart from auth are shared | 15:02 |
| rogpeppe | mgz: it seems to me like that's something that can be managed at a later stage. | 15:02 |
| rogpeppe | mgz: Link ? | 15:02 |
| mgz | for example, booting a server requires knowing what image to use | 15:02 |
| TheMue | rogpeppe: OK. Is it platform dependend? Can't reproduce it. | 15:03 |
| rogpeppe | mgz: oh yeah, another thing i noticed (can't remember quite where now) - if you want to unmarshal a possibly null string using JSON, you can use *string rather than interface. | 15:03 |
| mgz | this is just given as a uuid, and at present will need to manually configured as we haven't got a neat cloudwotsit that tells you what the latest precise image on hp is, for instance | 15:03 |
| rogpeppe | TheMue: try: x := []int{1,2,3}; y = append(x[0:1], 4); fmt.Println(x) | 15:04 |
| mgz | but you don't need a rich image object, which is what you'd expect back when looking up images using glance (seperate project farmed out from nova a few releases back) | 15:04 |
| rogpeppe | mgz: i'm talking about shared concepts, not just shared things. can't we use a common Link type across each package? | 15:05 |
| mgz | probably, what specifically does that get us? | 15:06 |
| TheMue | rogpeppe: OK, that's the trick, IC. Thx. | 15:06 |
| TheMue | rogpeppe: Tested it without indexing. | 15:06 |
| rogpeppe | mgz: it reduces the number of overall entities. | 15:06 |
| mgz | richer basic types would be good. | 15:06 |
| mgz | sure, but what would Link be exactly? | 15:07 |
| rogpeppe | mgz: what is it now? (i don't see any docs :-]) | 15:07 |
| mgz | ...I think this is all stuff for unmarshalling/marshalling specific json | 15:08 |
| mgz | which is likely not very sharable, at least if we want to be well behaved | 15:08 |
| rogpeppe | mgz: IMHO it's nice to factor out common elements when appropriate. then it becomes easy to use common code across those elements. | 15:08 |
| rogpeppe | mgz: but ListFlavors, for example, returns a []Entity | 15:09 |
| mgz | rogpeppe: one issue I have is optional params | 15:09 |
| rogpeppe | mgz: so it's *not* just about marshalling/unmarshalling | 15:09 |
| mgz | as I understand it, unmarshalling lacking some json keys is fine, the value for that key is then just nil | 15:09 |
| rogpeppe | mgz: yeah | 15:09 |
| mgz | but the reverse is a little suspect | 15:10 |
| mgz | if you're meant to provide key A or key B, you don't really want {... A: "", B: "something"} | 15:10 |
| rogpeppe | mgz: you can have omitempty for that | 15:10 |
| mgz | ah, that sounds good. | 15:10 |
| rogpeppe | mgz: it doesn't work for structs though, i think | 15:11 |
| rogpeppe | mgz: just atoms | 15:11 |
| rogpeppe | mgz: but i may be wrong there | 15:11 |
| rogpeppe | mgz: anyway, it's not necessary to use the same types for unmarshalling as you're returning from the functions. | 15:11 |
| mgz | sure, but it is pretty handy, and we'd need even more structs to make that distinction | 15:13 |
| mgz | the Link/Entity thing is pretty mysterious... | 15:14 |
| mgz | does seem they're meant to be a common interface many objects use | 15:15 |
| rogpeppe | mgz: do they have a common json representation too? | 15:17 |
| mgz | nope, but share some keys | 15:17 |
| mgz | many things have an id:UUID, name:string, links:{} as well as the rest of their details | 15:18 |
| mgz | so, having that up one level might make sense (not for swift though) | 15:19 |
| mgz | the nova derived parts are more homogenous | 15:20 |
| rogpeppe | mgz: you *might* be able to use an embedded Entity struct to make that work | 15:20 |
| rogpeppe | mgz: is there an API reference for these things BTW? i just found java docs. | 15:21 |
| mgz | api.openstack.org | 15:21 |
| rogpeppe | ah, i was looking at docs.openstack.org | 15:22 |
| * rogpeppe loves the way a single example is considered sufficient documentation | 15:23 | |
| mgz | hey, at least there's an example these days | 15:23 |
| rogpeppe | :-) | 15:25 |
| mgz | I did quite a bit of coding to the implementation... | 15:25 |
| mgz | which when bouncing through and external api then the internal rpc was often quite fun to work out exactly what some param did | 15:26 |
| rogpeppe | mgz: i'm sure | 15:26 |
| mgz | reminds me, I still want to see if I can do away with the need for private storage via cunning compute api usage | 15:28 |
| mgz | the main problem is the habit of stuffing charms in there... which I think we can just do with an auto-generated local-public bucket | 15:28 |
| fwereade | mgz, +1 | 15:29 |
| mgz | the current hack of just making the whole bucket public-readable is... not something I'm happy with | 15:30 |
| rogpeppe | mgz: we also use the private storage for pushing development versions of tools to | 15:33 |
| rogpeppe | mgz: but in general it's an idea we're working towards | 15:34 |
| rogpeppe | fwereade: will we be able to use your new UnitsWatcher for watching service units too? | 15:47 |
| fss | niemeyer: regarding proxy support, there's another issue, we would need to specify it to cloud init too, so it is used when apt-getting | 15:48 |
| fss | niemeyer: our current approach is to set the proxy in /etc/apt/apt.conf.d in a customized AMI | 15:49 |
| niemeyer | fss: We probably need an env setting for that | 15:49 |
| fss | niemeyer: makes sense | 15:51 |
| fss | niemeyer: thanks | 15:55 |
| niemeyer | fss: Do you need https proxy as well, or just http? | 15:55 |
| fwereade | rogpeppe, nope, there will be too many service units for that approach | 15:57 |
| rogpeppe | fwereade: any chance of a review of this trivial branch? https://codereview.appspot.com/6847091/ | 15:58 |
| * fwereade looks | 16:00 | |
| fss | niemeyer: yep. We use the HTTP proxy for tunneling HTTPS connections. Unfortunately, twisted does not support that. We're using HTTP AWS endpoints, but we don't plan to keep using this | 16:02 |
| rogpeppe | TheMue: ultra-trivial review for you (and also fixes trunk when compiling against go tip): https://codereview.appspot.com/6849101 | 16:03 |
| TheMue | rogpeppe: Indeed ultra-trivial. ;) | 16:05 |
| niemeyer | rogpeppe: https://codereview.appspot.com/6847091/diff/4001/environs/bootstrap.go#newcode77 | 16:05 |
| niemeyer | rogpeppe: Please please please don't commit that | 16:05 |
| niemeyer | rogpeppe: It's getting pretty frustrating to see that on *every* branch | 16:05 |
| rogpeppe | niemeyer: i've fixed it now, AFAIK | 16:06 |
| rogpeppe | niemeyer: i'll re-gofmt that branch | 16:06 |
| niemeyer | rogpeppe: It's still there | 16:07 |
| rogpeppe | niemeyer: i haven't re-proposed it yet | 16:07 |
| niemeyer | rogpeppe: Understood.. the only thing I can comment on is what I can see :) | 16:07 |
| rogpeppe | niemeyer: FWIW, i wouldn't be able to submit it like that anyway | 16:08 |
| rogpeppe | niemeyer: re-submitted | 16:08 |
| rogpeppe | niemeyer: proposed, rather | 16:08 |
| fss | niemeyer: ops, the correct answer should be no. We use http proxy for tunneling https requests. Sorry | 16:08 |
| niemeyer | fss: Okay, they're both the same then, I see | 16:09 |
| niemeyer | rogpeppe: Thanks! | 16:09 |
| rogpeppe | niemeyer: have you got any other comments on that CL, BTW? it seems pretty straightforward to me. | 16:18 |
| niemeyer | rogpeppe: I'm surprised ot not see any comments from me there | 16:18 |
| niemeyer | rogpeppe: I'm pretty sure I had reviewed it already | 16:19 |
| rogpeppe | niemeyer: i haven't seen anything | 16:19 |
| niemeyer | rogpeppe: LGTM either way | 16:19 |
| rogpeppe | niemeyer: thanks | 16:20 |
| fwereade | TheMue, I am lacking firewaller context again | 16:23 |
| fwereade | TheMue, there was talk with Aram of some bug in the firewaller | 16:23 |
| fwereade | TheMue, can you remind me what it was and whether it is addressed somewhere? | 16:23 |
| TheMue | fwereade: His watcher changes lead to a different handling of the global mode. And there sometimes occurs a race. At least it seems so. | 16:24 |
| fwereade | TheMue, hmm; do you know if it was addressed? | 16:25 |
| TheMue | fwereade: No, because the watcher and firewaller changes are still in review and he tried to find out what exactly is happening. | 16:27 |
| fwereade | niemeyer, ^^ | 16:27 |
| fwereade | niemeyer, his branch still has the contentious c.Skip("disabled until firewaller initialization is fixed") | 16:27 |
| niemeyer | TheMue: What is the different handling more precisely? | 16:28 |
| fwereade | niemeyer, *that* was why I judged it a potential rabbit hole | 16:28 |
| niemeyer | fwereade: Are we within the hole already or not? | 16:28 |
| fwereade | niemeyer, AFAIK the current trunk uses the old-style MPUW | 16:28 |
| niemeyer | fwereade: IOW, is this exposing we already have, or one we're getting into? | 16:28 |
| fwereade | niemeyer, if we were to merge Aram's MUW, it would expose this nebulously-defined bug | 16:29 |
| fwereade | niemeyer, sorry | 16:29 |
| fwereade | niemeyer, if we were to merge the branch which *uses* Aram's MUW, it would expose... | 16:29 |
| niemeyer | fwereade: Yeah, everything I heard about it so far was indeed nebulously defined :) | 16:29 |
| TheMue | niemeyer: I would have to take a look again. He moved parts of the firewallers main loop into an own method. And there the logic IMHO changed because the watcher now returns the ids (if I'm right) and the lifecycle state has to be checked. | 16:29 |
| niemeyer | TheMue, fwereade: Yes, and that's exactly what we should do to finish porting watchers | 16:30 |
| niemeyer | TheMue, fwereade: We need someone to stop nebulously defining what's going on, and actually get stuff working :) | 16:30 |
| TheMue | fwereade: If it's ok for you we both could dig deeper into it tomorrow morning together. From two sides, the watcher and the firewaller. | 16:32 |
| fwereade | niemeyer, oddly enough, that is what I am trying to do :) | 16:32 |
| niemeyer | fwereade: and I'm sorry that this is getting on your track.. I really wished your non-trivially filled plate wouldn't get fuller, but if your goal goes over that track, the best course of action is to go head on and understand what's going on | 16:33 |
| fwereade | niemeyer, I have perhaps not made clear how closely my branch tracks aram's second one | 16:34 |
| fwereade | niemeyer, but it doesn't include machiner changes | 16:34 |
| niemeyer | fwereade: I don't understand how you can track the branch changes without touching the call sites that use it | 16:36 |
| fwereade | niemeyer, I can deal with the additional bits on my plate, but I would like to tackle them in such a way that I can get this little, disproportionately-useful, chunk merged | 16:36 |
| fwereade | niemeyer, I want to *just add a type* | 16:36 |
| fwereade | niemeyer, because this is a useful change | 16:36 |
| fwereade | niemeyer, and stands well on its own | 16:36 |
| fwereade | niemeyer, and doesn't dilute the focus of the branch | 16:36 |
| niemeyer | fwereade: You want to add a type, that does the same thing already being done elsewhere, right? | 16:36 |
| fwereade | niemeyer, it does something different, that is very nearly the same as what aram's second branch does | 16:37 |
| niemeyer | fwereade: Duplicating logic that was supposed to be the same, right? | 16:37 |
| fwereade | niemeyer, I don;t consider the machiner bits of that branch to be anywhere near ready | 16:37 |
| niemeyer | fwereade: So let's do them | 16:37 |
| niemeyer | fwereade: Seriously.. this has been coming forever, for no good reason | 16:38 |
| fwereade | niemeyer, all in one branch? | 16:38 |
| niemeyer | fwereade: We already have two branches | 16:39 |
| niemeyer | fwereade: and those two branches already had their lives in other branches | 16:39 |
| fwereade | niemeyer, right; one of them is a potential rats' nest, and it is blocking the other one | 16:39 |
| niemeyer | fwereade: I'm sorry you're getting involved on that, but I'm clearly concerned about that stuff by now | 16:39 |
| fwereade | niemeyer, I have extracted the best parts of that other one into a fresh CL | 16:39 |
| fwereade | niemeyer, which is IMO a useful stepping stone towards having everything fixed | 16:40 |
| niemeyer | fwereade: This problem has grown up disproportionally to the task | 16:40 |
| niemeyer | fwereade: which is why we can't just keep patching stuff around the actual task | 16:40 |
| niemeyer | fwereade: It's time to sort it out | 16:40 |
| niemeyer | fwereade: I'm happy to do that, but only if the rest of these branches are coming along | 16:41 |
| fwereade | niemeyer, I am willing to take on the responsibility of landing that functionality | 16:41 |
| niemeyer | fwereade: If the idea is cherrypicking what you need and leaving the rest, the no, let's not do that please | 16:41 |
| niemeyer | fwereade: Okay, so tell me about that | 16:41 |
| niemeyer | fwereade: What's the rest of the plan? | 16:42 |
| fwereade | niemeyer, figure out what the deal is with the nebulous bug, in parallel with the other work that will be unblocked by implementing the new-style UnitsWatcher | 16:42 |
| fwereade | niemeyer, namely the Deployer and associated Container changes we discussed | 16:43 |
| fwereade | niemeyer, and then replace Machiner entirely, once Deployer has been approved and merged | 16:43 |
| fwereade | niemeyer, are you concerned that I will forget to fix the firewaller? ;) | 16:44 |
| fwereade | niemeyer, I am mainly concerned that I will spend days up to my elbows in it, and then aram will show up and push a branch that fixes it | 16:44 |
| niemeyer | fwereade: I'm concerned that we'll lose months of work on that silly change | 16:45 |
| niemeyer | fwereade: Because we're again re-branching and re-starting | 16:45 |
| niemeyer | fwereade: and even worse this time: we're doing stuff in parallel | 16:45 |
| niemeyer | fwereade: Which opens a big opportunity to keep the old version around so we don't spent time on it | 16:45 |
| niemeyer | fwereade: That said, | 16:46 |
| niemeyer | fwereade: You have a good track record on that stuff | 16:46 |
| niemeyer | fwereade: So if you're indeed going to finish that migration over the next couple of weeks, you'll have my sympathy and my time on it too | 16:46 |
| fwereade | niemeyer, awesome, tyvm :D | 16:47 |
| fwereade | niemeyer, let me just -wip that branch and repropose with unscrewed API in a bit | 16:47 |
| niemeyer | fwereade: But let's do it.. we can't afford to leave the situation unhandled | 16:47 |
| fwereade | niemeyer, upon my honour, I will land that branch or a close relative thereof :) | 16:48 |
| niemeyer | fwereade: SGTM :-) | 16:48 |
| rogpeppe | niemeyer: i've just proposed this branch. it changes environs.Bootstrap to work like we discussed, i hope. https://codereview.appspot.com/6782119 | 17:03 |
| niemeyer | rogpeppe: Thanks a lot | 17:05 |
| rogpeppe | niemeyer: if you were to review some of my branches today, these ones (particularly the first) are the ones i'd most like some feedback on. (i can't propose more because subsequent branches have more than one dependency) | 17:21 |
| rogpeppe | 149-state-info-rootcert https://codereview.appspot.com/6855054/ | 17:21 |
| rogpeppe | 164-bootstrap-generate (https://codereview.appspot.com/6782119/ | 17:21 |
| rogpeppe | 166-juju-cert-flag https://codereview.appspot.com/6842088/ | 17:21 |
| niemeyer | rogpeppe: Sounds good.. I'm on a call right now, but once I'm back I'll continue through the queue first | 17:22 |
| rogpeppe | niemeyer: thanks | 17:22 |
| niemeyer | rogpeppe: Will do as much as I can on the two hours remaining after that | 17:22 |
| rogpeppe | niemeyer: you've already reviewed almost all the first one - it just lacks a LGTM, i think | 17:23 |
| niemeyer | rogpeppe: Sounds good, will start with that then | 17:32 |
| rogpeppe | niemeyer: FWIW here's the branch dependency tree that takes us up to working TLS: http://paste.ubuntu.com/1389506/ | 17:36 |
| fss | niemeyer: could you take a look at that iam CL later? | 18:04 |
| niemeyer | fss: Will do | 18:05 |
| fss | niemeyer: thanks | 18:05 |
| fss | niemeyer: this is the error when bootstrapin on sa-east-1: cannot query old bootstrap state: Get : 301 response missing Location header | 18:28 |
| rogpeppe | i'm off for the evening. | 18:29 |
| rogpeppe | 'night all | 18:29 |
| niemeyer | fss: Still on a call, but will be back in a bit | 18:36 |
| niemeyer | fss: That issue is known, and already being fixed I believe | 18:37 |
| niemeyer | rogpeppe: Have a good one | 18:37 |
| fss | niemeyer: oh, nice :-) thanks | 18:43 |
| niemeyer | fss: https://code.launchpad.net/~dave-cheney/juju-core/052-environs-ec2-always-request-public-tools-from-us-east-1/+merge/136077 | 18:44 |
| fss | niemeyer: nice | 18:46 |
| fss | niemeyer: also, could you improve the error message when public-bucket is not defined? | 18:47 |
| fss | $ juju bootstrap | 18:47 |
| fss | error: cannot find tools: no compatible tools found | 18:47 |
| niemeyer | fss: Yeah, that'd be useful, and a nice simple task to get started with the workflow on the Go side, actually (hint! hint!) :-) | 18:48 |
| fss | niemeyer: sure, will do this. Looks like I can't develop juju-core on mac os :-( I'm setting up an instance for development | 18:49 |
| fss | niemeyer: I can't find what's the default message for required settings. do you have one? :-) | 19:02 |
| niemeyer | fss: Hmm | 19:18 |
| niemeyer | fss: What's the situation? | 19:18 |
| niemeyer | fss: You can find that kind of case in environs/config/config.go | 19:18 |
| niemeyer | fss: But I'm not entirely sure if that's what you're looking for | 19:18 |
| fss | niemeyer: I'm validating the presence of public-bucket | 19:19 |
| fss | niemeyer: I get some gofmt and govet errors when running lbox | 19:19 |
| niemeyer | fss: That's not a required setting | 19:20 |
| fss | hum | 19:20 |
| niemeyer | fss: I guess I misunderstood what you meant earlier | 19:20 |
| niemeyer | fss: Sorry about that | 19:20 |
| niemeyer | fss: You can use juju during development purely with a private bucket | 19:21 |
| niemeyer | fss: and there will be a default value for public-bucket | 19:21 |
| fss | niemeyer: oh, I see. So there's no need to improve this error message because it won't happen when a default value for public-bucket is defined | 19:22 |
| * niemeyer breaks out.. will come back for a few more reviews later if I feel energized enough | 20:12 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!