| thumper | hatch: which fix? | 00:26 |
|---|---|---|
| thumper | hatch: although probably not | 00:26 |
| thumper | hatch: you can probably expect a 1.18 soonish | 00:27 |
| hatch | thumper there was one related to mongodb that was fixed but not sure if it was going in 16 or 18 | 00:27 |
| thumper | yeah, not sure sorry | 00:27 |
| thumper | axw may know when he gets online | 00:27 |
| hatch | no problem thanks though | 00:33 |
| wallyworld | thumper: meeting? | 01:03 |
| thumper | oh yeah... | 01:03 |
| thumper | wallyworld: my popup reminder conveniently ended up behind emacs | 01:04 |
| rick_h__ | come on, surely emacs will do reminders :P | 01:27 |
| thumper | rick_h__: probably, but it isn't hooked up to my google calendar! | 01:48 |
| thumper | axw: sorted out the screen issue? | 02:13 |
| axw | thumper: all sorted | 02:13 |
| axw | sorry I'm late, will make it up later | 02:13 |
| axw | thumper: I got my new laptop - thought I had it sorted last night :/ | 02:13 |
| thumper | what's the new laptop? | 02:14 |
| axw | xps 15 | 02:14 |
| thumper | nice? | 02:14 |
| thumper | with the high res screen? | 02:14 |
| axw | thumper: yup, the one I just turned down to 1080p because I can hardly read anything :( | 02:14 |
| thumper | I have some dude taking out my power meter and replacing it with a smart meter | 02:15 |
| thumper | no power in the house | 02:15 |
| axw | at 3200x1800 that is | 02:15 |
| thumper | so on battery and using mobile 3G | 02:15 |
| thumper | wow, that is good | 02:15 |
| thumper | how's the battery? | 02:15 |
| thumper | and how heavy? | 02:15 |
| axw | thumper: dunno yet, hardly used it - just been frigging around getting my preferred OS on it ;) | 02:16 |
| axw | ~2kg | 02:16 |
| thumper | heh | 02:16 |
| thumper | I'm curious to know when you have hammered it a bit | 02:16 |
| axw | I'll let you know when I do :) | 02:16 |
| axw | the keyboard is a bit puny compared to what I'm used to | 02:17 |
| axw | getting used to it though | 02:17 |
| === ray is now known as rawang | ||
| _thumper_ | bugger! | 03:40 |
| === _thumper_ is now known as thumper | ||
| axw | thumper, wallyworld: is there a reason why we don't generate new SSH keys for environments? | 03:45 |
| wallyworld | axw: we use the ssh key of the user who created the environment | 03:46 |
| wallyworld | i guess that was deemed sufficient | 03:46 |
| axw | wallyworld: yeah I know that's what we do now, just wondering why we shouldn't just create a new key | 03:46 |
| axw | mmk | 03:46 |
| wallyworld | not sure, that design decision was way before my time | 03:46 |
| axw | nps | 03:46 |
| axw | wallyworld: https://bugs.launchpad.net/juju-core/+bug/1257371/comments/9 | 03:48 |
| _mup_ | Bug #1257371: bootstrap fails because Permission denied (publickey) <bootstrap> <regression> <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1257371> | 03:48 |
| axw | some people might not have a default ssh key, which buggers things up - that's why I ask | 03:48 |
| wallyworld | ah, just read the bug | 03:50 |
| wallyworld | i can see why we don't generate a new key | 03:50 |
| wallyworld | s/dont't/ might not want to | 03:50 |
| wallyworld | it would be another key to manage | 03:50 |
| wallyworld | when we could just add existing user keys (or import using ssh-import-id) | 03:51 |
| wallyworld | but if a user doesn't have a key already.... | 03:51 |
| wallyworld | axw: are you adding code to allow a key to be specified if not in ~/.ssh/id_rsa.pub? | 03:52 |
| axw | wallyworld: just considering options atm | 03:52 |
| wallyworld | ok | 03:53 |
| axw | wallyworld: that's one option, or there's the option of generating one automatically | 03:53 |
| axw | or both | 03:53 |
| wallyworld | generating one would be more user/bot friendly | 03:53 |
| axw | yeah that's what I'm thinking | 03:53 |
| wallyworld | i'd +1 that approach :-) | 03:54 |
| wallyworld | maybe run it by jam or william as well | 03:54 |
| axw | yeah, I'll just send an email to juju-dev now | 03:55 |
| thumper | wallyworld: http://pastebin.ubuntu.com/6523153/ | 03:59 |
| thumper | wallyworld: and I logged in and checked, and it did indeed have 4G of disk, 1G of ram and 2 cores\ | 03:59 |
| thumper | \o/ | 03:59 |
| wallyworld | \o/ | 03:59 |
| wallyworld | awesome | 03:59 |
| * thumper proposes | 03:59 | |
| thumper | wallyworld: https://codereview.appspot.com/37610043 | 04:11 |
| * thumper goes to write some emails | 04:11 | |
| * wallyworld looks | 04:11 | |
| * wallyworld ->doctor, bbiab | 04:23 | |
| davecheney | doh | 06:33 |
| davecheney | so close | 06:33 |
| davecheney | lucky(~/src/launchpad.net/juju-core) % juju ssh 0 | 06:33 |
| davecheney | WARNING discarding API open error: <nil> | 06:33 |
| davecheney | ERROR environment has no access-key or secret-key | 06:33 |
| davecheney | ^ what does this mean | 06:33 |
| davecheney | any ideas ? | 06:33 |
| davecheney | axw: thanks for the tip | 06:45 |
| davecheney | that fixed it | 06:45 |
| axw | davecheney: cool | 06:45 |
| axw | seems to me it's another win for using gc-built jujud | 06:45 |
| axw | the size, that is | 06:46 |
| davecheney | axw: oh, i'm not passing -Os | 06:47 |
| davecheney | maybe that would help | 06:47 |
| axw | davecheney: maybe at the expense of performance | 06:47 |
| davecheney | man, even with sync bootstrap juju deploy the first time is still very slow | 06:48 |
| davecheney | ie, i type juju depoy mysql m1 | 06:48 |
| davecheney | and it still took 2 minutes to return and tell me i spelt the command wrong | 06:48 |
| axw | huh. I didn't think it did much before parsing the command line... | 06:50 |
| davecheney | i think we still connect to the state too early in the subcommand | 06:50 |
| davecheney | axw: brace yourself | 06:52 |
| davecheney | lucky(~/src/launchpad.net/juju-core) % ls -alh ~/bin/juju{,d} | 06:52 |
| davecheney | -rwxr-xr-x 1 dfc dfc 36M Dec 5 17:21 /home/dfc/bin/juju | 06:52 |
| davecheney | -rwxr-xr-x 1 dfc dfc 40M Dec 5 17:21 /home/dfc/bin/jujud | 06:52 |
| axw | :o | 06:52 |
| davecheney | ding ding ding! | 06:59 |
| davecheney | http://ec2-54-253-189-54.ap-southeast-2.compute.amazonaws.com/wp-admin/install.php | 06:59 |
| axw | very nice :) | 06:59 |
| davecheney | if only it wasn't wordpress | 07:00 |
| davecheney | we're not allowed to mention wordpress | 07:00 |
| === dspiteri is now known as darrenS | ||
| === darrenS is now known as DarrenS | ||
| === rogpeppe1 is now known as rogpeppe | ||
| rogpeppe1 | mornin' all | 08:39 |
| dimitern | morning! | 08:40 |
| dimitern | we should decide what to do about putcharm today | 08:41 |
| dimitern | otherwise the discussion can drag on too long | 08:41 |
| dimitern | fwereade_, i'm not sure i understand the icon question for local charms | 09:28 |
| dimitern | fwereade_, why do we need GET support? | 09:28 |
| fwereade_ | dimitern, ah sorry -- did the mail I forwarded lack the original context? | 09:29 |
| dimitern | fwereade_, the context is there, but I still have no clue how to do that | 09:29 |
| dimitern | fwereade_, I don't know how it works with the charmworld now | 09:29 |
| fwereade_ | dimitern, I think the charmworld bit is a red herring tbh | 09:30 |
| dimitern | fwereade_, can the icon be part of the charm package? | 09:30 |
| fwereade_ | dimitern, the question is maybe best phrased as "can we somehow serve individual charm files to authenticated clients" | 09:30 |
| fwereade_ | dimitern, yeah, icon.svg | 09:31 |
| dimitern | fwereade_, hmm.. | 09:31 |
| dimitern | fwereade_, i suppose, if we duplicate part of charmworld in the api server yes | 09:31 |
| fwereade_ | dimitern, if we do go through an unpack/repack step for every local charm we do at least have all that data available somehwere already, and it'd just a matter of putting it in the right place | 09:32 |
| dimitern | fwereade_, we can have GET /charms/xyz/icon.svg | 09:32 |
| fwereade_ | dimitern, and I don't think we need to worry about store charms so much because they can already get the data from elsewhere | 09:32 |
| fwereade_ | dimitern, although it would probably be best to make this functionality available for allcharms | 09:33 |
| dimitern | fwereade_, i was thinking of removing the data after repackaging - why bother - it's already in the provider storage | 09:33 |
| dimitern | fwereade_, so it gets hairer by the minute :) | 09:33 |
| dimitern | fwereade_, proxy any charm files from the provider storage or charm store through the api | 09:34 |
| thumper | dimitern: hey, when do you go on leave? | 09:34 |
| dimitern | fwereade_, this seems like a potential DDOS target - pinging the api server with random urls of valid charm, just so it has to fetch and unpack them | 09:35 |
| dimitern | thumper, hey, on 23rd | 09:35 |
| fwereade_ | dimitern, well we *do* want charm access to be authenticated | 09:35 |
| thumper | dimitern: ah, for some reason canonical admin is all fubared | 09:35 |
| thumper | dimitern: how's the upload charm stuff going? | 09:35 |
| fwereade_ | dimitern, we dropped the ball on that because signed urls are an openstack extension iirc | 09:35 |
| dimitern | thumper, oh? it was looking fine last time i looked | 09:35 |
| thumper | dimitern: I'm not surprised it is screwed, it was mramm that told me | 09:36 |
| dimitern | fwereade_, nevertheless it's a ddos target of a sort - an authenticated client written badly can bring down the api server like that perhaps | 09:36 |
| fwereade_ | dimitern, but we ought to be clawing it back if possible -- in general we shouldn't just be making the contents of environment storage available to anyone who asks | 09:36 |
| fwereade_ | dimitern, more so than *any* endpoint? | 09:37 |
| fwereade_ | dimitern, if we're ok with one-time tokens for PUTs, we could surely do the same with GETs | 09:37 |
| dimitern | fwereade_, well, a bit more so, because it requires some busy work of fetching and unpacking a charm just to serve a file | 09:37 |
| dimitern | fwereade_, i'm not sure what approach did we decide to pick | 09:38 |
| fwereade_ | dimitern, quite, we should definitely not do that | 09:38 |
| fwereade_ | dimitern, and your work maybe involves extracting all the charm files anyway | 09:38 |
| fwereade_ | dimitern, hence the potential connection in my mind | 09:38 |
| dimitern | fwereade_, if we need the get stuff, rogpeppe1's proposal seems to fit better than mine | 09:38 |
| dimitern | fwereade_, have POST/PUT/GET support for charm urls prefix, like /charms/ and handle everything there | 09:39 |
| dimitern | fwereade_, potentially caching the unpacked charms locally forever | 09:39 |
| fwereade_ | dimitern, well we can't do that exactly | 09:39 |
| fwereade_ | dimitern, HA | 09:39 |
| rogpeppe1 | fwereade_: why is HA a problem there? | 09:40 |
| fwereade_ | dimitern, we presumably want them, and everything else that doesn't have to be provider-level, in gridfs storage | 09:40 |
| dimitern | fwereade_, do what? cache? | 09:40 |
| jam | fwereade_: you could write it all into mongo | 09:40 |
| jam | rogpeppe1: dimiter was saying 'localyl' | 09:40 |
| fwereade_ | dimitern, which machine unpacked then? | 09:40 |
| jam | as in /tmp, I believ | 09:40 |
| rogpeppe1 | jam: i think that was with respect to caching | 09:40 |
| rogpeppe1 | jam: which would be fine, i think | 09:40 |
| rogpeppe1 | jam: although you'd want to keep a handle on the size of the cache | 09:41 |
| dimitern | fwereade_, doesn't matter which machine | 09:41 |
| fwereade_ | dimitern, well, we need the files on all the machines, right? | 09:41 |
| dimitern | fwereade_, whichever api server serves a charm, it checks its local cache and populates it before serving | 09:41 |
| rogpeppe1 | jam: and actually, it might not be worth caching - you don't need to unpack for a GET | 09:41 |
| dimitern | fwereade_, no | 09:41 |
| dimitern | rogpeppe1, you need specific files inside the package, like the icon.svg | 09:42 |
| jam | rogpeppe1: so the request from fwereade_ was that you could serve just the icon.svg, for example, and that is not trivial to proxy without unpacking | 09:42 |
| rogpeppe1 | ah, i hadn't realised that | 09:42 |
| * rogpeppe1 goes to look at the gridfs docs | 09:42 | |
| rogpeppe1 | jam: actually, i don't think it would be too hard | 09:44 |
| dimitern | fwereade_, rogpeppe1, although I still think going with a pure http-based interface will screw us up badly when we need to think of role-based auth | 09:44 |
| rogpeppe1 | dimitern: how's that? | 09:44 |
| jam | dimitern: basic auth is username + password | 09:44 |
| dimitern | i now that | 09:44 |
| jam | I think oauth is an HTTP header | 09:44 |
| jam | and you can always have X-Juju-Authorization-Token: XYZ | 09:45 |
| rogpeppe1 | jam: +1 | 09:45 |
| dimitern | but what don't know is would it be enough/like the api login | 09:45 |
| rogpeppe1 | dimitern: it doesn't necessarily need to be | 09:45 |
| rogpeppe1 | dimitern: did you see my reply to Gary's comment? | 09:45 |
| fwereade_ | rogpeppe1, jam, dimitern: well, I want us to remain open to the possibility of alternative auth mechanisms | 09:45 |
| rogpeppe1 | fwereade_: definitely | 09:45 |
| dimitern | rogpeppe1, so using the same user/pass for login and for basic http auth | 09:45 |
| jam | fwereade_: I think HTTP headers are just as turing complete as anything else | 09:46 |
| rogpeppe1 | jam: actually, they're not quite | 09:46 |
| fwereade_ | rogpeppe1, jam, dimitern: so issuing tokens from the API feels like it gets round that potentialcomplexity completely | 09:46 |
| fwereade_ | ? | 09:46 |
| rogpeppe1 | jam: you can't do a multi-stage login protocol | 09:46 |
| dimitern | fwereade_, exactly | 09:46 |
| dimitern | fwereade_, but then we don't solve the get issue | 09:46 |
| rogpeppe1 | fwereade_: we could issue auth tokens from the API | 09:47 |
| dimitern | fwereade_, and having to implement a get handler + a bunch of apis, rather than a bunch of http handlers only seems simpler | 09:47 |
| jam | rogpeppe1: you could with round trips, and I'm pretty sure basic PUT can return an "please finish Auth" request. | 09:47 |
| rogpeppe1 | fwereade_: which don't have any specific relationship to charms | 09:47 |
| rogpeppe1 | jam: ah, ok | 09:47 |
| rogpeppe1 | jam: sounds complex though. | 09:47 |
| dimitern | jam, PUT can return 401 as POST or GET | 09:48 |
| jam | rogpeppe1: that said, client sides may not implement the 100 Continue stuff, which would mean they try to upload the whole content and then get a "hey, you need to auth, please upload again" | 09:49 |
| rogpeppe1 | fwereade_: in the future, i'm thinking of a method, say Client.AuthToken that returns an authentication token that can be used to authenticate future URL operations. | 09:49 |
| rogpeppe1 | fwereade_: but for the time being, i don't think it's necessary. | 09:49 |
| dimitern | rogpeppe1, actually it is i think | 09:52 |
| dimitern | rogpeppe1, we can always generate a token and return it as a result of login | 09:52 |
| dimitern | rogpeppe1, then we can use this token as a session key for url requests | 09:52 |
| dimitern | rogpeppe1, and have a call to renew a token perhaps | 09:52 |
| rogpeppe1 | dimitern: that's not a bad idea - i don't think it's *necessary* but it would work well | 09:52 |
| rogpeppe1 | dimitern: for the time being we could just return the username and password | 09:53 |
| dimitern | rogpeppe1, you mean ask for? | 09:54 |
| dimitern | rogpeppe1, not return | 09:54 |
| rogpeppe1 | dimitern: i mean that the auth token returned by Login would just embed the user name and password that was passed to Login | 09:54 |
| rogpeppe1 | dimitern: the client would treat it as an opaque identifier | 09:55 |
| dimitern | rogpeppe1, not in plain text though | 09:55 |
| rogpeppe1 | dimitern: so if we changed to using a more sophisticated scheme in the future, the client would not need to change | 09:55 |
| rogpeppe1 | dimitern: why not? | 09:55 |
| dimitern | rogpeppe1, because it's a security leak | 09:55 |
| rogpeppe1 | dimitern: how so? | 09:55 |
| dimitern | rogpeppe1, returning the "usernamepassword" in plain text from the server? | 09:56 |
| rogpeppe1 | dimitern: yeah | 09:56 |
| dimitern | rogpeppe1, and then using that as a token? | 09:56 |
| rogpeppe1 | dimitern: yup | 09:56 |
| dimitern | rogpeppe1, why? | 09:56 |
| rogpeppe1 | dimitern: how is it a security leak? | 09:57 |
| dimitern | rogpeppe1, basic auth does that already - user/pass auth | 09:57 |
| rogpeppe1 | dimitern: sure, and that's essentially what we're doing now | 09:57 |
| rogpeppe1 | dimitern: how is it a security leak? | 09:57 |
| dimitern | rogpeppe1, my point is that a token is probably either a part of the url (unsafe) or a header (with ssl probably safe) | 09:57 |
| rogpeppe1 | dimitern: it would be part of the header (and the url is actually safe too with https, i believe) | 09:58 |
| dimitern | rogpeppe1, unsafe, meaning if the token is not opaque, like you're suggesting | 09:58 |
| rogpeppe1 | dimitern: we're talking about an authentication token, right? any time that leaks, it's a security leak regardless, whether it's plain text or not), | 09:59 |
| rogpeppe1 | dimitern: i'm just suggesting we go with an ultra-simple approach to start with, which is sufficient for our current needs, and also capable for the future. | 10:00 |
| dimitern | rogpeppe1, no it's not | 10:00 |
| rogpeppe1 | dimitern: why not? | 10:00 |
| jam | dimitern: rogpeppe1, fwereade_, TheMue, davecheney: weekly standup | 10:00 |
| dimitern | rogpeppe1, an opaque token, which is time-sensitive is better, even if leaked the damage is minimized | 10:00 |
| jam | mramm: standup if you're around | 10:00 |
| jam | mgz: ^^ | 10:01 |
| jam | https://plus.google.com/hangouts/_/calendar/bWFyay5yYW1tLWNocmlzdGVuc2VuQGNhbm9uaWNhbC5jb20.8sj9smn017584lljvp63djdnn8 | 10:01 |
| mgz | I'm there | 10:03 |
| === gary_poster|away is now known as gary_poster | ||
| davecheney | jam: bad news, -O2/s has no appreciable effect on binary size | 11:20 |
| davecheney | well, -O2 made it 5% bigger | 11:20 |
| davecheney | -rwxr-xr-x 1 dfc dfc 19M Dec 5 22:20 /home/dfc/bin/juju | 11:23 |
| davecheney | -rwxr-xr-x 1 dfc dfc 22M Dec 5 22:20 /home/dfc/bin/jujud | 11:23 |
| davecheney | ^ jam stripped | 11:23 |
| davecheney | but I don't know if it is safe to do that with gccgo binaries | 11:23 |
| davecheney | i'd guess it is probably safer than gc binaries | 11:23 |
| davecheney | but i don't have enough experiecen | 11:23 |
| * dimitern lunch | 11:24 | |
| * davecheney bed | 11:24 | |
| jam | davecheney: thanks for the reference point, sleep well | 11:41 |
| davecheney | jam: antonio put me up to this, i swear | 11:44 |
| jamespage | jam: urgh - I just got the --upload-tools thing when running from trusty with gccgo and trying the manage precise | 12:45 |
| jamespage | esp if libgo is not statically linked.... | 12:46 |
| jamespage | gah | 12:46 |
| jam | jamespage: so as I understand from davecheney, it *is* intended that final binaries built with gccgo would be staticly linked to avoid 'what version is on what platform' problems | 13:34 |
| jamespage | jam: not sure about the statically linked thing | 13:35 |
| jamespage | that's something the security team where keen to avoud | 13:35 |
| jamespage | libgo contains a core set of SSL libraries | 13:35 |
| jamespage | so not having to rebuild for every security vulnerability is a + | 13:35 |
| TheMue | rogpeppe1: *carefulPing* I changed the tailer now that it starts N lines before the end. only downside is that those N lines aren't already filtered, so that the initially returned lines can be less. | 14:18 |
| rogpeppe1 | TheMue: presumably you could filter as you go back through the file? | 14:23 |
| TheMue | rogpeppe1: should be possible too, only a bit more logic to create the right strings to check | 14:25 |
| TheMue | rogpeppe1: and those may span two or more read buffers | 14:25 |
| TheMue | rogpeppe1: but will try it after hangout (in 2 mins) | 14:28 |
| rogpeppe1 | TheMue: yes, you may need to use something more like the code i sent you | 14:28 |
| TheMue | rogpeppe1: a bit more of it, yes | 14:30 |
| TheMue | rogpeppe1: btw, how about the storm in your hometown? here it's ok so far, but it still shall grow | 14:30 |
| rogpeppe1 | TheMue: pretty windy today | 14:31 |
| TheMue | rogpeppe1: yeah, we have some douglas firs in front of the house and they are swinging fine :) | 14:32 |
| TheMue | rogpeppe1: so, off for some minutes, hangout | 14:33 |
| dimitern | rogpeppe1, why is there a mutex in apiserver Login/ | 15:13 |
| rogpeppe1 | dimitern: because it might be called concurrently | 15:13 |
| dimitern | rogpeppe1, it seems I can reuse the same Login call for http basic auth | 15:14 |
| dimitern | rogpeppe1, except for the loggedIn flag | 15:14 |
| rogpeppe1 | dimitern: i'd abstract out a separate function called by both | 15:14 |
| rogpeppe1 | dimitern: i did that in the example http handler code i posted a while ago | 15:14 |
| dimitern | rogpeppe1, yeah, my thoughts exactly | 15:15 |
| rogpeppe1 | dimitern: authUser in https://codereview.appspot.com/22100045/diff/20001/state/apiserver/admin.go | 15:19 |
| sinzui | natefinch, I put you in an n+1 config yesterday. I built the windows installer juju 1.16.4 and verified I could bootstrap and deploy from windows with it. | 15:19 |
| natefinch | sinzui: awesome | 15:20 |
| sinzui | natefinch, I learned I can use wine+inno to make the installer. I don't have windows so I started an instance in aws. I learned about rdp and making ssh work. I think I know enough now to test this reguarly | 15:21 |
| natefinch | sinzui: awesome. I hoped wine would work, but hadn't tried it myself. | 15:21 |
| abentley | sinzui: I think I will upgrade CI's system juju to 1.16.4. Any reason not to? | 15:25 |
| sinzui | abentley, +1 to upgrade | 15:29 |
| rogpeppe1 | TheMue: you have a review | 16:20 |
| TheMue | rogpeppe1: thx | 16:21 |
| mgz | dstroppa: so, do you have lbox installed and working? | 17:19 |
| dstroppa | mgz: I'm getting an 'undefined: syscall.TCGETS' when I execute 'go get launchpad.net/lbox' | 17:23 |
| mgz | >_< | 17:24 |
| mgz | maaacs... | 17:24 |
| mgz | from goetveld/rietveld/terminal_darwin.go? | 17:25 |
| mgz | that should be worked around in latest release... | 17:26 |
| dstroppa | goetveld/rietveld/terminal.go | 17:27 |
| mgz | I have terminal_darwin.go and terminal_linux.go | 17:31 |
| dstroppa | here http://bazaar.launchpad.net/~goetveld/goetveld/trunk/files I can only see terminal.go | 17:36 |
| mgz | uuu | 17:37 |
| mgz | dstroppa: can you branch lp:~gophers/goetveld/trunk instead? | 17:38 |
| mgz | I'm not sure why we have two, at different revisions | 17:38 |
| mgz | dstroppa: or just try againt now, I've changed it on launchpad | 17:39 |
| mgz | so go get should now work | 17:40 |
| dstroppa | mgz: go get works | 17:40 |
| dstroppa | and now I got lbox | 17:40 |
| mgz | phew :) | 17:41 |
| mgz | okay, so now go to your gojoyent branch, and run: (adjust the path as needed) | 17:41 |
| mgz | `~/go/bin/lbox propose --for lp:~juju/gojoyent/for_review -cr -v -wip` | 17:41 |
| mgz | which does some stuff, then should bring up an editor for you to write a review message in | 17:42 |
| mgz | also creds for g+ and launchpad as part of it... painful but hopefully not too bad | 17:42 |
| dstroppa | Branch is not clean (I got some file that are not pushed yet as I'm still working on it) | 17:44 |
| dstroppa | shall I move them or can work around it? | 17:44 |
| mgz | shelve those bits then run it again | 17:44 |
| mgz | `bzr shelve --all -m "wip whatever"` | 17:45 |
| mgz | and `bzr unshelve` when you want it back | 17:45 |
| dstroppa | sh: sensible-editor: command not found | 17:48 |
| mgz | heh | 17:48 |
| mgz | set EDITOR to something? | 17:49 |
| mgz | this is the flakiest tool... | 17:49 |
| dstroppa | error: Change summary is empty. | 17:53 |
| dstroppa | even though I added both summary and desc | 17:53 |
| mgz | dstroppa: the code opens a tempfile, execs your editor with that name, waits for it to exit, seeks to 0, then reads | 17:55 |
| mgz | is there anything there that would get upset for you? | 17:55 |
| mgz | or maybe just a blank line at the top or something daft? | 17:57 |
| dstroppa | I believe no | 17:57 |
| dstroppa | it opens up vi | 17:57 |
| dstroppa | I edit the file (replace the <enter…> with my text) | 17:57 |
| dstroppa | save and close | 17:57 |
| mgz | feel free to poke at lbox text.go to work out why it's being fussy | 17:58 |
| mgz | vim's what I use so I'd expect it to be fine for you as well | 17:58 |
| mgz | dstroppa: any luck? | 18:05 |
| dstroppa | tried vi and vim, same error | 18:06 |
| dstroppa | looking at text.go, but can see anything in particular | 18:07 |
| mgz | could just add a log statement to dump the file name/contents before that error, see if that makes anything clearer | 18:09 |
| dstroppa | mgz: looks like it's still reading the template | 18:26 |
| dstroppa | even though the temp file is saved | 18:26 |
| hazinhell | so is safe mode the default? | 19:02 |
| hatch | on juju 1.16.4-precise has anyone else been experiencing 'unauthorized mongo access' errors? | 19:05 |
| jam | hatch: from where, to where? from the 'juju' CLI or from agents, or ? (It hasn't been reported before, but it is worth investigating) | 19:15 |
| hatch | jam so I just did a apt-get upgrade and now when I type `sudo juju bootstrap` I get that error | 19:16 |
| jam | hatch: so this is for local provider ? | 19:17 |
| hatch | correct | 19:17 |
| jam | (otherwise you wouldn't be using sudo, presumably) | 19:17 |
| jam | hatch: can you try "sudo juju bootstrap --debug" and then paste bin the result (it might have secrets if you want to be careful) | 19:17 |
| hatch | sure one sec | 19:18 |
| jam | hatch: also, what version of 'mongodb' is on your machine? (dpkg -l mongodb-server) | 19:20 |
| hatch | jam I think here are the relevant bits https://gist.github.com/hatched/3e8a13af98250c236c9d | 19:20 |
| hatch | 1:2.2.4-0ubuntu1~ub | 19:20 |
| jam | hatch: what series are you on? (precise/raring/etc) | 19:21 |
| hatch | precise | 19:21 |
| hatch | these issues appeared after the update to ,4 | 19:21 |
| jam | so I'm not saying to do this *yet* but we do have mongodb-2.4.6 in the cloud-archive: sudo add-apt-repository cloud-archive:tools | 19:21 |
| jam | hatch: the actual changes of 1.16.3 vs 1.16.4 don't seem like they would be causing db auth problems, but I'll dig a bit | 19:22 |
| hatch | I saw some emails going by that talked about a juju specific mongodb | 19:22 |
| hatch | did those land in .4? | 19:22 |
| jam | hatch: I just "juju destroy-environment; and then juju bootstrap" locally and it worked on precise, so it *might* be the newer mongodb | 19:22 |
| jam | hatch: no | 19:22 |
| jam | that is going to be a while in the future | 19:23 |
| hatch | ahh ok ok | 19:24 |
| jam | hatch: so I know we didn't intend to change anything about how we connect to the db in 1.16.4 vs 1.16.3, and I confirmed the diff doesn't appear to contain a change there. | 19:25 |
| hatch | very odd... | 19:26 |
| jam | hatch: are you bootstrapping with a DB already configured or something like that ? (I wouldn't think you would get this far) | 19:26 |
| hatch | jam I'm going to say no.... | 19:26 |
| hatch | I've tried deleting the local/ and environments/ but that doesn't appear to help | 19:28 |
| jam | hatch: well if you delete those, then you can't "juju destroy-environment" properly, which means you possibly *do* have a stale DB that we aren't aware of (maybe) | 19:30 |
| hatch | oh hmm | 19:30 |
| hatch | any idea on how I would wipe it clean? | 19:30 |
| jam | hatch: just to try it "juju destroy-environment -e local; sudo juju bootstrap -e local" | 19:30 |
| jam | or whatever you named it | 19:30 |
| hatch | same | 19:30 |
| hatch | issue | 19:31 |
| jam | hatch: I'll try to install 2.2.4 here, but so far I have no luck reproducing your issue (next step is to install the 1.16.4 binary that was built, because it might be flawed somehow vs building from trunk) | 19:35 |
| hatch | jam is there a way I could clear out my mongodb besides the destroy-environment call? | 19:36 |
| jam | hatch: btw, do you know about paste.canonical.com? it requires Auth tokens so is generally 'safe enough' when pasting private stuff. | 19:36 |
| jam | hatch: well, "create mongo journal dir: /home/jameinel/.juju/local/db/journal" so I'm 95% sure that deleting ~/.juju/local would have done that | 19:37 |
| hatch | ahh ok that was my thought process as well | 19:37 |
| hatch | jam yeah I usually dont' use that paste because it doesn't allow edits | 19:39 |
| jam | hatch: can you paste your local config ? | 19:39 |
| hatch | sure | 19:39 |
| hatch | https://gist.github.com/hatched/72ce29821d4c56c76280 | 19:40 |
| hatch | nothing changed in there since the update | 19:40 |
| jam | hatch: sure, you could *try* just commenting out the admin-secret | 19:40 |
| jam | as we should generate one for you if it isn't present | 19:40 |
| jam | but I haven't been able to reproduce the problem | 19:40 |
| hatch | ok trying | 19:40 |
| hatch | nope :/ | 19:41 |
| hatch | darn | 19:41 |
| jam | hatch: I didn't expect much there, as there is no reason to expect the value there is invalid. I also can't reproduce on Precise with the jujud binary installed from the ppa and mongo from that ppa as well. | 19:41 |
| jam | hatch: maybe you can pastebin more of the log? In case there was a config earlier that is wierd | 19:42 |
| jam | weird | 19:42 |
| jam | natefinch: I'm about to head to bed, any chance you can give this a look ? I'm skeptical that this is caused by 1.16.4 vs 1.16.3, but I'd really like to know what a root cause is | 19:43 |
| hatch | thanks for the help jam - maybe what I'll do is remove juju and mongo and start frash | 19:43 |
| hatch | fresh* | 19:44 |
| hatch | maybe something got messed up in the update | 19:44 |
| jam | or maybe wallyworld or thumper depending on who logs in first (we're almost around to start-of-day NZ I believe) | 19:44 |
| jam | hatch: I would like to understand what is broken, but I can also understand you just want to get it working | 19:44 |
| hatch | well the good news is that I don't need to test any new GUI work until tomorrow :) | 19:45 |
| hatch | haha so I have until then | 19:45 |
| jam | its nearly 8am in NZ, so thumper should be around soon. and rogpeppe2 is often known as a very helpful fella :) | 19:45 |
| jam | though its past his EOD, I believe | 19:46 |
| rogpeppe2 | jam: it is :-) | 19:46 |
| natefinch | jam: I'm here | 19:46 |
| natefinch | jam: took a jaunt to the Lexington, MA office, since they wanted to see Ubuntu on my laptop (high DPI screen) | 19:46 |
| jam | natefinch: k, brief summary, hatch upgraded to 1.16.4 from the ppa, and now when he tries to bootstrap it gives him unauthorized failures | 19:47 |
| jam | I'm unable to reproduce, even using Precise + mongo-2.2.4 + juju-1.16.4 from the PPA | 19:47 |
| jam | natefinch: sounds fun | 19:47 |
| jam | how far away is it for you? | 19:47 |
| hatch | natefinch this is the error https://gist.github.com/hatched/3e8a13af98250c236c9d | 19:48 |
| natefinch | 1/2 hour, not bad at all. Had lunch with David Pitkin, old colleague from our younger startup days | 19:48 |
| natefinch | hatch: hmm.. interesting | 19:49 |
| hatch | I've tried deleting the environments/ and local/ as well as doing a destroy-environment and no luck | 19:49 |
| jam | natefinch: a thought occurs, could it be something with apparmor ? | 19:50 |
| jam | it doesn't seem like it should be, but that unauthorized message looks odd to me | 19:50 |
| natefinch | yeah, I was looking at the unauthorized message | 19:51 |
| jam | looks like it is stock mongo failures | 19:51 |
| jam | http://stackoverflow.com/questions/13850191/mongodb-set-user-password-to-access-to-db | 19:51 |
| jam | hatch: other things to check "what is ps -ef | grep mongod" look like | 19:53 |
| jam | what happens when you "juju destroy-environment -e local" and then check ps again | 19:53 |
| jam | (i'm wondering if you have a rogue mongodb process that for some reason isn't being stopped like it should) | 19:54 |
| jam | anyway, sleepytime | 19:54 |
| hatch | jam no change, 1 mongo process running | 19:54 |
| hatch | jam thanks for your help! have a good night | 19:54 |
| natefinch | hatch: I'm doing some investigating on this end. trying to see if we changed some of the mongo login code or otherwise changed our form of access to mongo | 20:00 |
| natefinch | hatch - saw you and jam talking about mongo versions... just curious, you don't have to do it yet, but would it be a problem to upgrade 2.4.6? I'm not convinced it'll fix anything, just want to get the lay of the land. | 20:03 |
| hatch | natefinch nope no problem at all, it's just my testing box | 20:04 |
| hatch | normal repos don't show a 2.4.6 version | 20:05 |
| natefinch | hatch: hmmm I must have a ppa for mongo | 20:05 |
| natefinch | 2.4.6 is their most recent stable version (from just a few months ago) | 20:06 |
| hatch | looking for another ppa | 20:08 |
| hatch | natefinch here is my dpkg output https://gist.github.com/hatched/b2e72169891505be29d4 for mongo* | 20:11 |
| natefinch | ahh yeah, that's what it was, mongodb-10gen | 20:13 |
| hatch | so I'm all up to date then it looks like | 20:13 |
| natefinch | yep, seems like it. good | 20:14 |
| hatch | basically I'm at the point where it looks like a R&R on Juju and Mongo is in order | 20:15 |
| hatch | since noone else is having these issues heh | 20:15 |
| natefinch | yeah, it's weird. It would be good for us to understand what happened so we can make sure it won't happen again, though. | 20:20 |
| hatch | yeah well I'm available to help but I will need to get it back working for tomorrow :) | 20:21 |
| natefinch | I know the feeling | 20:22 |
| natefinch | certainly, if there's a point where you need to bail and reset everything, just let me know and go for it. | 20:22 |
| hatch | I can probably use ec2 tomorrow/Monday to give us some extra time | 20:24 |
| natefinch | hatch: if we can't get it figured out, I'll talk to jam, and we'll decide if we will gain anything by making life more difficult for you :) | 20:30 |
| hatch | haha | 20:31 |
| hatch | sounds like a plan | 20:31 |
| === negronjl_ is now known as negronjl | ||
| natefinch | hatch: very strange.... the code really looks totally right.... we create the db with your admin secret as the password... unless the admin secret changed in the middle of bootstrap somehow, which seems impossible | 21:20 |
| hatch | heh yeah | 21:21 |
| natefinch | hatch: obviously *something* is going on | 21:21 |
| thumper | morning | 21:22 |
| natefinch | thumper: morning | 21:22 |
| hatch | natefinch when I ps -ef | grep mongod there is a mongod instance running | 21:22 |
| hatch | assuming that's supposed to be there | 21:22 |
| hatch | /usr/bin/mongod --auth --dbpath=/home/hatch/.juju/local/db | 21:22 |
| natefinch | hatch: yeah, it should be running. That looks correct. | 21:23 |
| natefinch | thumper: hatch is having a problem bootstrapping the local provider after upgrading from the ppa to 1.16.4. He's getting an unauthorized error from mongo during bootstrap | 21:24 |
| hatch | https://gist.github.com/hatched/3e8a13af98250c236c9d this is the error | 21:24 |
| thumper | weird... | 21:24 |
| natefinch | hatch: had you had mongo or juju local running before the upgrade? | 21:25 |
| hatch | natefinch nope | 21:25 |
| hatch | I'm rebooting the machine now | 21:25 |
| natefinch | k | 21:25 |
| hatch | maybe that'll fix it heh | 21:25 |
| thumper | hatch: are you bootstrapping using sudo? | 21:25 |
| natefinch | thumper: it won't let you bootstrap without sudo | 21:26 |
| hatch | thumper yes `sudo juju bootstrap` and 'local' is my default | 21:26 |
| thumper | hatch: if you go 'which juju' what does it say? | 21:26 |
| hatch | /usr/bin/juju | 21:27 |
| hatch | ok the machine has been rebooted | 21:27 |
| hatch | and.... | 21:27 |
| hatch | *drumroll* | 21:27 |
| hatch | .....longer drumrolll | 21:27 |
| natefinch | heh | 21:27 |
| natefinch | if that works, I'll eat my hat | 21:27 |
| hatch | bootstrapped | 21:27 |
| thumper | haha | 21:28 |
| hatch | ....w t h | 21:28 |
| natefinch | haha | 21:28 |
| natefinch | good thing I don't actually have a hat | 21:28 |
| hatch | lol | 21:28 |
| thumper | natefinch: I'll buy one for you | 21:28 |
| natefinch | thumper: that's very kind of you | 21:28 |
| thumper | natefinch: np | 21:28 |
| hatch | so...wow...why? heh | 21:28 |
| hatch | I even killed and restarted mongo | 21:28 |
| thumper | ours is not to reason why, ours is just to reboot | 21:29 |
| hatch | haha | 21:29 |
| hatch | damn....that took so long to debug | 21:29 |
| hatch | I guess we should have just listened to the IT Crowd | 21:29 |
| hatch | "did you turn it off and back on again" | 21:29 |
| natefinch | step 1: reboot | 21:29 |
| hatch | well thanks everyone for their help haha | 21:30 |
| natefinch | I'm glad it's fixed. I can't even... I don't... | 21:31 |
| hatch | well honestly....I even killed and restarted mongo and it didn't help | 21:31 |
| hatch | so I have no idea what else was stuck in there causing it to break | 21:31 |
| natefinch | YEah,. that's the thing, I'd think killing mongo would have to be the same as rebooting | 21:32 |
| thumper | hmm... | 21:32 |
| * thumper shrugs | 21:32 | |
| natefinch | Whatever, I'm just glad I could be here to fix it for you ;) | 21:33 |
| thumper | wallyworld: I'm heading to the gym, if you are happy with my changes, can I get you to approve the MP so it lands? | 22:40 |
| thumper | cheers | 22:40 |
| wallyworld | thumper: done | 23:11 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!