[08:05] Good morning Juju world! [08:06] dgonzo hi. Sorry the MWC swamped all my time this week, but today is the day I actually spin that g2. Upcoming news soon === frankban|afk is now known as frankban [09:52] aaand good morning [09:52] lets try bootstrapping my controller, day 3 :P [10:15] Hello, lads. What's the best way to get bundletester? If I pip-install it, it breaks stuff on my system - for instance, pull-lp-source from ubuntu-dev-tools doesn't work anymore [10:45] Mmike: virtualenvwrapper ftw? [10:46] cnf, can we pretend I never asked this question? :) [10:46] done! [10:46] :P [10:46] :D [10:47] i'm afraid it will shortly end up here: https://irclogs.ubuntu.com/2017/03/03/%23juju.html ;) === frankban is now known as frankban|afk === marcoceppi_ is now known as marcoceppi [13:14] magicaltrout: these are not the logs you are seeking! [13:14] s/seeking/looking for/ :> [13:15] I missed the VO version :p === disposable3 is now known as disposable2 [15:29] hmm [15:29] juju really doesn't want to connect to the bootstrapped controller on ssh [15:29] and no debug logging on it, it seems :( [15:30] Attempting to connect to 172.20.20.16:22 [15:30] aaaand stuck [15:34] hmz :/ [15:35] how do i get more info than --debug --show-log [15:42] hmm, i guess it's friday afternoon === frankban|afk is now known as frankban [15:56] still stuck at Attempting to connect to 172.20.20.16:22 [16:05] cnf: and this is still attempting over the sshuttle tunnel? [16:05] lazyPower: yes [16:05] it's as close as I can get to a direct connection [16:05] manual ssh works over it [16:06] I see you've tuned it for verbosity as well [16:06] yeah, but the ssh bit isn't being very verbose [16:06] so i don't know what is going on [16:09] lazyPower: tbh, it seem --show-log doesn't do much beyond what --debug does [16:11] yeah i think thats max verbosity, what you have listed there [16:11] yeah, sadly it doesn't give any debug messages for the underlying ssh [16:13] My very first juju controller was managed over a sshuttle connection and it worked just fine [16:14] but that was about a year ago [16:14] yeah, 2.0 seems to have changed some things? [16:14] idno, i'm new to juju [16:14] Budgie^Smore: did you have to do anything specific? [16:15] I was using 2.0... not that I remember... whats your sshuttle command look like? [16:15] sshuttle -r tele-maas --ssh-cmd='assh wrapper ssh' 172.20.20.0/24 195.130.158.250 [16:16] hmmm, I wonder if it is your ssh-cmd that is causing it, I didn't use that flag [16:17] well, without it, it won't work well [16:17] it needs a few hops to get anywhere [16:17] Budgie^Smore: but i can manually do "ssh 172.20.20.16" and it works [16:17] but juju won't connect to it, it seems [16:17] if you can ssh to it then you shouldn't need that flag [16:18] no, i can't ssh to the target [16:18] 172.20.20.16 needs to go over the sshuttle [16:19] it's me ---- machine ---- MAAS controller --- machine booting to be the juju controller [16:19] oh yeah, sorry barely woke up yet [16:19] however just cause you can manually ssh doesn't mean that wrapper doesn't change things so that juju can't [16:19] the wrapper is for sshuttle, though [16:20] petevg: Can you make any sense of this error I'm getting from Model.deploy()? http://pastebin.ubuntu.com/24102747/ [16:20] cnf yes but any wrapper can change something in it's stream so that it is no longer sending expected packets [16:20] cory_fu: usually I get that with bad yaml [16:21] well, the wrapper is just calling the ssh binary [16:21] cory_fu: or an invalid value, a string for a tag or something [16:21] i don't see how it would affect anything [16:21] or doing something stupid like merging stdout and stderr [16:21] rick_h: But there's no yaml in that request? [16:21] cory_fu: e.g. is it a user tag vs a string username/etc [16:21] calling it with what flags? [16:21] cory_fu: Yes. You're giving it an int or something when it expects a string. [16:21] Budgie^Smore: all it does is generate ssh parameters like ProxyCommand etc [16:21] petevg: ... thanks [16:21] so i can chain hops [16:21] cory_fu: "None" [16:22] cory_fu: that in there isn't a string or valid value for json [16:22] rick_h: That's a Python None, which would be converted to a null [16:22] cnf I would reckon there is a conflicting flag between what juju is providing and the wrapper [16:22] petevg: Does the "to" field not accept None? [16:22] Budgie^Smore: but the wrapper is for shhuttle only [16:23] cory_fu: I don't remember. The code around the "to" field is really annoying and hard to keep in one's head. [16:23] petevg: Docstring says, "If None, a new machine is provisioned." so that seems pretty explicit [16:23] Budgie^Smore: juju just needs to do /usr/bin/ssh 172.20.20.16 [16:24] cory_fu: interesting. The Python code in python-libjuju/juju/placement.py represents my best understanding of what Go wants for that field. [16:24] cory_fu: note that it always seems to want things to be packed into an array/list [16:24] petevg: Looks like None gets converted to an empty list [16:24] petevg: I wonder if it's the Constraints param? [16:25] cory_fu: could be. The version of placement that I'm looking at will just return "None" when it gets passed None, no list involved. [16:25] ... so maybe that is the error -- maybe it should return [] rather than None in that first check. [16:26] petevg: Well, it won't ever actually get passed None, at least not from Model.deploy, because there's an outer None check that converts it to [] [16:26] cory_fu: Got it. [16:27] petevg: I think it's the constraints. It's getting passed {} which it thinks is a valid parsed value, but it should probably be None instead [16:28] cory_fu: interesting. The constraints.py thing that I wrote will pass back None. I'm assuming that it's getting converted into the empty dict in the facade object ... [16:28] Well, that got me past the ghost failure, but mysql failed [16:28] petevg: http://pastebin.ubuntu.com/24102792/ [16:28] cory_fu: ah. Unless you give it a dict, then it will pass a dict back, without checking to see if it has stuff in it. [16:28] petevg: Yeah. I was giving it a dict, like a... chump [16:29] cory_fu: eh. I was passing back an empty dict like a chump :-) [16:29] petevg: Probably worth replacing the 'if None' and 'if == ""' with a single "if not" [16:29] agreed. [16:29] I'll do that in my conjure-up branch [16:30] That error still says "number" to me, though. I wonder if you need to stringify everythign -- "num_units" is an int rather than a string. [16:30] Not according to the docstring nor default arg [16:31] petevg: mysql is the only one that has config values. Could the 20000 value be the issue? [16:32] petevg: Should we switch regular deploy to use configYAML as well? [16:32] hmz, this is frustrating [16:33] cory_fu: interesting. That could be it, too. [16:33] cnf, it probably doesn't just do a clear ssh command, probably adds some control options but the devs would have to say for sure [16:33] petevg: + # stringify all config values for API, and convert to YAML [16:33] it is doing something weird, that's for sure [16:33] cory_fu: yeah. That is supposed to fix things. :-/ [16:34] and if it isn't that, then it is might be key related [16:34] cory_fu: and I've successfully deployed stuff with configs that look like mysql. [16:34] petevg: Actually, I don't like that we have duplication between Model.deploy and BundleHandler.deploy. Do you think we could combine those? [16:35] cory_fu: probably. I think that I only switched to using config_yaml in BundleHandler. [16:35] ... so if your code is passing through Model.deploy, it might not be stringifying things. [16:35] Right [16:37] silly question probably but did you add your ssh pubkey to maas so it can add it to juju? [16:41] OK I am going to shut up now, that question doesn't make sense :-/ going to go find coffee, bbiab [16:41] Budgie^Smore: yeah, and i can ssh manually [16:41] Budgie^Smore: enjoy :P [16:42] something in me still things there might be a key issue, i.e. juju not talking to ssh-agent, etc. [16:43] but I am more inclined to thing that it is an overlapping ssh flag issue [16:43] i wish i'd get ssh debug info :/ [16:50] hmm [16:50] maybe got something closer [16:50] maybe [16:51] cnf: while i wouldn't normally advocate this... you can try something [16:51] no, nm [16:51] cnf: you can temporarily rename the ssh bin to something like ssh.orig, and put a wrapper in place that invokes that .orig ssh bin with -vvv [16:51] hmm [16:51] but i feel like this means we shoudl file a bug to request a flag to dump ssh debug info [16:51] indeed [16:51] i'm not seeing anything anywhere that indicates we have a flag already to provide that detail [16:52] --debug should put ssh in -vvv mode [16:52] so, hacky work-around solutions seem to be the path forward for now [16:52] cnf: since you filed the other bug i'll file this one ;) [16:52] haha, thanks :P [16:52] it is, once again, almost time to go home for me... [16:52] almost WE [16:55] cnf: https://bugs.launchpad.net/juju/+bug/1669848 [16:55] Bug #1669848: When bootstrapping with the --debug flag, ssh should also pass -vvv for debugging ssh issues [16:56] lazyPower: thanks! [16:56] if you dont mind hitting the top left "This bug affects you" to add some bug heat. You may also want to subscribe for updates so you can follow up with any discussion there. [16:57] SimonKLB: You're next after i get caio some feedback on a PR submit this morning. [16:57] i look forward to this :) [16:58] ah, i was looking for that! [17:02] lazyPower: nice! [17:09] lazyPower: your trick doesn't work! it isn;t showing the output :P [17:09] cnf: well it was worth a shot [17:10] yep [17:10] well, i'm out of ideas [17:10] not the way i wanted to start the WE ^^; [17:13] k, i'm going home [17:13] thanks for the help lazyPower, Budgie^Smore [17:14] np cnf === rogpeppe1 is now known as rogpeppe === ejat_ is now known as ejat === stormmore is now known as Budgie^Smore === Dmitrii-Sh_ is now known as Dmitrii-Sh === wolsen_ is now known as wolsen === diddledan_ is now known as diddledan === FourDollars_ is now known as FourDollars === yo61_ is now known as yo61 === rmcadams_ is now known as rmcadams [18:23] stub: https://bugs.launchpad.net/postgresql-charm/+bug/1669872 [18:23] Bug #1669872: charm install fails installing snapd on trusty [18:27] from what I can find, snap should install via apt on trusty [18:27] not sure why I'm not getting it [18:28] http://paste.ubuntu.com/24103535/ [18:35] stub: in what revision was layer snap introduced to the postgresql charm? [18:42] stub: ahh, rev 117 [18:54] ok I am back and in the office === frankban is now known as frankban|afk [22:58] cory_fu: so i didn't get to it this morning but i just deployed the autoscaler. Did you use this outside of bundletesting? [22:58] lazyPower: Nope [22:58] cory_fu: do you want to see whats up with it? :D [22:58] i'm in a hangout running the paces rn [22:59] Uh, sure [23:01] lol its cool man :) you're busy and its after 5 on a friday [23:01] just making the offer [23:01] but i'm in the batcave