[02:15] <axw> babbageclunk: https://github.com/juju/1.25-upgrade/issues/32 <- huh? what do you mean we don't use the local juju client? the plugin is called by it...
[02:15] <axw> babbageclunk: or do you just mean, other than the calling into the plugin
[02:16] <babbageclunk> axw: I mean, the plugin doesn't run juju - so it doesn't matter how they're installed.
[02:16] <axw> babbageclunk: okey dokey
[02:16] <babbageclunk> axw: sorry, those are mostly notes for myself for things that have occurred to me while testing with xavpaice
[02:17] <axw> babbageclunk: np
[02:17] <babbageclunk> axw: although, one I wanted to ask you about was the lxc-lxd conversion - it seemed like it didn't leave the containers running?
[02:18] <axw> babbageclunk: yeah just saw that, that's odd - it's always worked for me...
[02:18] <babbageclunk> axw: I haven't had a chance to dig in there - he's had to go do some real work
[02:18] <axw> you can see in the code that we start the LXD containers and wait for them to be accessible
[02:18] <axw> babbageclunk: in fact, the migrate-lxc command actually runs an SSH command to check that they're acccessible
[02:19] <babbageclunk> Hmm
[02:19] <babbageclunk> He ran that without showing me output. Next time I do it I'll get him to pastebin it so we can see.
[02:20] <axw> babbageclunk: okey dokey
[02:20] <babbageclunk> axw: how was assembly? Heather and I had a fun chat-meeting.
[02:23] <axw> babbageclunk: not bad, just went because Charlotte was getting a certificate. the class item was a bit meh :)
[02:23] <babbageclunk> stink
[02:23] <axw> babbageclunk: did I miss anything? how're things going?
[02:25] <babbageclunk> Going through an upgrade with xav has been super helpful - no blockers yet, I've been adding issues for things that could be smoothed out as they come up.
[02:26] <axw> veebers: what's up with the "Can one of the admins verify this patch?" message from the bot? what does that even mean?
[02:27] <babbageclunk> The lxc migration and import's done, but upgrade-agents hasn't been finished yet - we were just manually restarting the lxds and about to retry when he got interrupted by other stuff.
[02:27] <babbageclunk> I'm getting set up with canonistack to test the fixes I'll make to openstack tagging.
[02:40] <veebers> axw: where are you seeing that?
[02:40] <veebers> axw: I'm trying out some jenkins things, I had assumed I had correctly limited it to just my one test branch
[02:42] <veebers> axw: to answer your question, the plugin I'm testing apparently has a whitelist of 'admin' that can actually trigger a build (and thus asks for one of them to comment on the PR)
[02:44] <veebers> Oh, I see where that might have leaked though. Sorry, that should be plugged now
[02:53] <axw> veebers: no worries, was just curious - thank you
[02:54] <veebers> axw: I think I've removed all instances of the message. Sorry for the noise :-P
[02:54] <axw> all good :)
[03:29] <babbageclunk> axw: w00t - blahdeblah is happy with his upgraded canonistack model (modulo the security group problem)
[03:29]  * blahdeblah does a little dance
[03:29] <axw> babbageclunk: sweet :) nice work
[03:29] <blahdeblah> \o\ /o/ \o\ /o/
[03:30] <axw> blahdeblah: was there any lxc in your environment?
[03:30] <babbageclunk> no
[03:30] <axw> okey dokey
[03:30] <blahdeblah> axw: zarroo lxcs present!
[03:30] <babbageclunk> that's the thing i'm working on now with xavpaice
[03:30]  * axw nods
[03:31] <babbageclunk> axw: actually, any ideas about this? https://pastebin.canonical.com/198968/
[03:31] <babbageclunk> specifically the ssh errors at the end
[03:32] <axw> babbageclunk: the ssh_exchange_identification?
[03:32] <babbageclunk> yup
[03:32] <axw> babbageclunk: might be that the container's still initialising. try running migrate-lxc again, should resume from the right place
[03:32] <axw> oh but... already migrated
[03:32] <axw> so I guess that's what you/he just did
[03:33] <babbageclunk> I can ask him to try it again again though
[03:34] <axw> babbageclunk: doesn't hurt I suppose... but if that's what was already done, I wouldn't expect a change
[03:35] <axw> babbageclunk: strange that it only happens on a subset of containers tho
[03:35] <axw> maybe they are still coming up, and those ones are lagging behind the others
[03:38] <axw> babbageclunk: the loop in waitLXDContainerReady was supposed to prevent this ... :/
[03:58] <babbageclunk> axw: Looking at this, https://unix.stackexchange.com/questions/156908/intermittent-ssh-exchange-identification-connection-closed-by-remote-host-err
[03:58] <babbageclunk> What if we're creating too many connections to the host and overloading it?
[04:00] <axw> babbageclunk: you mean to the host of the lxc container? possible
[04:00] <babbageclunk> yeah
[04:00] <axw> babbageclunk: get him to try with --match=x/lxc/y
[04:00] <axw> to match just one container that's failing
[04:01] <babbageclunk> axw: he got this before (not with --match) https://pastebin.canonical.com/198977/
[04:01] <axw> better
[04:01] <babbageclunk> but still one failure even though it didn't need to migrate any
[04:02] <axw> babbageclunk: yeah it still needs to stop the juju agents though
[04:02] <babbageclunk> ok
[04:02] <babbageclunk> Oh, for idempotence?
[04:02] <axw> babbageclunk: yup
[04:03] <babbageclunk> yeah, makes sense although wasn't obvious to me at first.
[04:11] <babbageclunk> axw: If I want to throttle the max connections per host, what's a good way to structure that in Go?
[04:11] <babbageclunk> axw: might be easier to just do a max number of concurrent exec tasks, rather than per-host.
[04:17] <axw> babbageclunk: not sure there's a one size fits all, I think I'd just batch the errgroup.Go's and call Wait when the limit is hit
[04:18] <babbageclunk> axw: ok, thanks
[04:20] <babbageclunk> axw: I was looking at this: http://jmoiron.net/blog/limiting-concurrency-in-go/ but it seems a bit lower level - mixing it with errgroup seems clumsy.
[04:22] <axw> babbageclunk: actually that's probably not a bad approach. fill up a channel to whatever capacity, take one on entry to Go, immediately after defer adding one back
[04:23] <axw> babbageclunk: you could have a map of channels keyed on host addr too
[04:24] <babbageclunk> yeah, makes sense - I might try that.
[04:24] <babbageclunk> s/ might/'ll/
[05:30] <babbageclunk> axw: ha ha, fast
[05:30] <axw> :)
[05:30] <axw> reviewing the other now
[05:31] <babbageclunk> axw: Thanks
[05:32] <babbageclunk> axw: mostly wondering about the panicking/timing out stuff. Actually I think I'll change the panics to errors.
[05:32] <axw> babbageclunk: sounds fine, I don't mind either way - it's a case where panic is reasonable, but we don't really do panics
[05:34] <babbageclunk> just feels a bit icky to have panics and lots of goroutines
[05:35] <axw> babbageclunk: your call, code looks sound anyway
[05:35] <babbageclunk> cool, thanks
[05:41] <axw> babbageclunk: did you find out about your flight yet?
[05:48] <babbageclunk> axw: no, not yet
[05:50] <axw> babbageclunk: hope you're not too tired for the tour. I guess we'll all be jet lagged anyway though.
[05:51] <babbageclunk> I think the worst-case scenario is that thumper and I have to get the flight that gets into NYC at about 5 - in that case I'll probably bail on the tour.
[05:52] <axw> ouch, yeah
[05:54] <babbageclunk> the advisory at the moment lists our flight NZ8 as one of the ones not needing a refuelling stop - but it doesn't say whether that's for all days or just for now
[20:49] <balloons> thumper, what happened to github.com/juju/loggo/loggocolor
[20:49] <thumper> um... what do you mean?
[20:50] <balloons> github.com/juju/loggo/loggocolor is a 404 now
[20:51] <thumper> it is there for me
[20:52] <balloons> thumper, can you check the repo settings perhaps? unless I'm crazy
[20:52] <thumper> https://github.com/juju/loggo has it
[20:52] <thumper> where are you seeing it missing?
[20:52] <balloons> I can't import it
[20:53] <thumper> last commit into loggo was 5 june
[20:53] <balloons> so my build fails
[20:53] <balloons> I guess it's me.. has been all along
[20:57] <balloons> thumper, specifically cmd/juju/commands/debuglog.go import fails, which actually is the only place it's imported
[20:58] <thumper> which branch?
[20:58] <thumper> and where?
[20:58] <balloons> thumper, this is the archive package. I'm going to assume it's just more annoyingness
[20:59] <thumper> balloons: check the dependencies are updated properly
[20:59] <balloons> thumper, ack, I don't doubt it because we're building debs, snaps, and I can build manually myself.
[20:59] <balloons> but no dice for archive builders
[20:59] <thumper> does the commit hash of loggo match the dependencies.tsv
[21:01] <balloons> thumper, I'm willing to bet it's the debian rules messing with things
[21:02] <balloons> thumper, anyways, I'll head back to it
[21:02]  * thumper nods
[21:02] <thumper> ok
[21:02] <balloons> and yes, it matches
[21:02] <thumper> wat?...
[21:02] <thumper> FFS
[21:02] <balloons> ohh yep, it's debian/rules
[21:33]  * thumper feels his head is close to exploding
[21:36] <babbageclunk> Oh dear
[21:37] <babbageclunk> I mean, I feel like that too, but just because I'm getting so smart these days!
[21:53] <thumper> babbageclunk: care to be a teddybear?
[21:53] <babbageclunk> thumper: of course!
[21:53] <thumper> https://hangouts.google.com/hangouts/_/canonical.com/stuff
[21:54] <thumper> babbageclunk: ^^
[23:37] <babbageclunk> did you just ragequit
[23:37] <babbageclunk> thumper: ?
[23:39] <thumper> back
[23:40] <thumper> no it went to sleep (or thought it did)
[23:40] <thumper> I was thinking and talking, and didn't realise you weren't there for a while
[23:40] <thumper> babbageclunk: ^^
[23:40] <babbageclunk> thumper: I dropped anyway - team meeting soon
[23:40] <thumper> yeah
[23:40] <thumper> ok
[23:46] <thumper> babbageclunk: meeting?