=== verterok` is now known as verterok === karlthane_ is now known as karlthane === zz_CyberJacob is now known as CyberJacob === lukasa_away is now known as lukasa === CyberJacob is now known as zz_CyberJacob === dooferlad_ is now known as dooferlad === Odd_Blok1 is now known as Odd_Bloke === zz_CyberJacob is now known as CyberJacob === Guest43458 is now known as med_ === med_ is now known as med === med is now known as med_ [18:23] core, dev: Hey guys, trying to get started with juju-quickstart and I'm getting a certificate verify failed error: http://paste.ubuntu.com/12691122/ Any thoughts on this? [18:30] core, dev: Whats going on?? [18:32] I'm trying to give some devs overhere at darkhorse a juju demo, and get them bootstrapped.... I'm having issues and seem to be blocked .... can't seem to get anyone bootstrapped to a local env, or openstack provider.... [18:33] core, dev: "juju bootstrap -e local" and "juju-quickstart mediawiki-single" are failing on fresh trusty boxes [18:36] core, dev: no luck with "juju bootstrap -e openstack" either......any insight would be greatly appreciated..... I seem to be stuck between a rock and a hard place here.... I will submit bugs for all of this, just thought I would run it by you guys on here first.... ~ thanks [18:37] core, dev: I'm the poor soul bdx is trying to help get started with juju-quickstart [18:59] o/ derekcat bdx [19:00] derekcat bdx is there some kind of proxy or firewall with ingress/egress filtering? [19:00] marcoceppi: negative [19:00] the url, https://api.jujucharms.com/charmstore/v4/bundle/mediawiki-single/archive/bundle.yaml doesn't produce any SSL issues or warnings for me atm [19:01] bdx: can you load that URL in a browser to see if you get the same errors? [19:01] bdx: can you also lmk the version of juju (`juju version`) and quickstart (dpkg -l | grep quickstart) you're using? [19:01] marcoceppi: It works find in a browser for me? [19:01] the browser resolves it just fine....no errors there...only with juju-quickstart... [19:01] as you can probably guess, that's not supposed to happen ;) [19:02] marcoceppi: Don't have dpkg - hang on [19:02] marcoceppi: Wait... ubuntu command? Sorry, on OS X here [19:02] marcoceppi: https://bugs.launchpad.net/juju-quickstart/+bug/1503003 [19:02] Bug #1503003: Cannot communicate with charm store [19:02] derekcat: ahhh, clarity [19:03] derekcat: bdx said trusty machine so I assumed ubuntu [19:03] derekcat: did you brew install quickstart? [19:03] marcoceppi: correct [19:03] * marcoceppi checks that version [19:03] marcoceppi: "brew install juju-quickstart" was the command [19:04] derekcat: that may be lagged by a few releases, I'm not 100% certain [19:05] I'm traveling atm, don't have access to my OSX box unfortunately to check there [19:05] 2.2.1 is the latest version and that's the version you should have gotten with homebrew [19:06] Looks like juju-quickstart is current in brew - 2.2.1 [19:06] marcoceppi: Yeah 2.2.1 is what brew installecd [19:06] 2.2.1 is the latest version and that's the version you should have gotten with homebrew [19:06] derekcat: what's the quickstart command you're running? [19:07] marcoceppi, aisrael: juju-quickstart mediawiki-single [19:07] derekcat: and what environment/provider are you using (juju switch) [19:07] bdx: are you having other issues with Juju atm? [19:07] marcoceppi: yes ...many unfortunately [19:08] bdx: well I'll see if I can help there while aisrael helps derekcat [19:08] marcoceppi: I'm writing up a bug right now for broken local provider .... [19:08] awesome [19:08] bdx: so, you can't bootstrap - what version of juju? [19:09] marcoceppi: I cannot get juju to bootstrap local provider or openstack provider [19:09] 1.24.6-trusty-amd64 [19:09] bdx: alright, and if you bootstrap with `juju bootstrap --debug` what's the output? [19:10] when I try to bootstrap on a fresh trusty box I get : http://paste.ubuntu.com/12691346/ [19:10] aisrael: openstack [19:10] local provider bootstrap^^ [19:10] derekcat: ok, cool. I'm deploying that bundle to aws now. Let me see if I can recreate [19:10] bdx: what happens if you run `sudo initctl list | grep juju` - could you post the output? [19:11] aisrael: Okey [19:11] marcoceppi: no output [19:12] bdx: the juju-db, which is monogdb, doesn't seem to be starting properly, are there any log files in /var/log/upstart/juju-* ? === wolsen_ is now known as wolsen [19:14] marcoceppi: http://paste.ubuntu.com/12691411/ [19:15] bdx: is that from /var/log/upstart/juju-db.log? [19:15] juju-db-bdx-local.log ...yea [19:16] bdx: well, that's the problem. juju-db (mongodb) package is stacktracing like crazy [19:16] I've never seen that [19:16] bdx: can you dpkg -l | grep juju-db to get a version? [19:17] marcoceppi: well theres the problem .... no output [19:17] bdx: how about for `juju-local` instead of juju-db? [19:17] I can't really remember what the package name is [19:18] marcoceppi: I probably don't have juju-db installed for some reason ....I didn't know it was a seperate dep....1.24.6-0ubuntu1~14.04.1~juju1 [19:18] bdx: there may not actually be a seperate deb [19:18] ok [19:18] bdx: it's juju-mongodb* [19:18] 2.4.9-0ubuntu3 [19:19] bdx: that's what I have, odd. [19:19] i'm using the 1.25-beta but I'll spin up a VM real quick to test [19:20] marcoceppi: thanks! [19:20] bdx: do you have mongodb proper installed as well? [19:20] yes [19:21] bdx: I wonder if that's why, maybe it's calling the wrong mongodb binary [19:21] which would explain the stacktrace [19:21] I'll try both scenarios [19:21] totally [19:21] ok [19:21] my mongodb version - db version v2.4.9 [19:21] mongod* [19:22] so to be explicit, I provision a new trusty/vagrant box with "vagrant init ubuntu/trusty64; vagrant up --provider virtualbox" [19:23] bdx: oh, we can probably make this easier for you, we have a vagrant box with juju already installed and configured for local [19:23] aisrael: where are the latest instructions forthat? the vagrant stuff? [19:23] ssh into the box, and run "sudo apt update && sudo apt upgrade && sudo apt install juju juju-core juju-local" [19:23] derekcat: Unable to reproduce it on my end. I am on 10.11, though. Could you run it again with with --debug and pastebin that? [19:24] is there anything else I should be installing to get local provider working? [19:24] ok [19:24] bdx: that's the sum of it [19:24] marcoceppi: Current is still https://jujucharms.com/docs/1.24/howto-vagrant-workflow. I'm polishing the rewrite I started in DC this week. [19:24] aisrael: http://paste.ubuntu.com/12691475/ [19:25] aisrael: There you go [19:25] bdx: ^ that vagrant image as the local provider already bootstrapped [19:25] marcoceppi: I would like to know how to provision local provider none the less, it should work anyway right? [19:25] derekcat: ok, thanks. Give me a few. I think I might know what's up [19:25] bdx: totally should work [19:26] bdx: just trying to get you up in running while I suffer on airplane wifi to get this vm spun up [19:26] lol [19:26] awesome....thanks [19:28] I am pumped on that....but more importantly, this has been plagueing me for a while, and I would like to see it fixed so as I can tell devs to bootstrap a local provider and not have to provide work arounds to all [19:29] bdx: I agree, and we're right here with you on that [19:29] bdx: this is x86 architecture, right? [19:29] yes [19:30] Do you have curl installed via brew or using the built-in version? [19:30] derekcat:^ [19:30] derekcat: also, openssl [19:32] derekcat: I suspect `brew install openssl` will fix the problem. If that's the case, we might want to add that as a dependency to juju-quickstart [19:32] aisrael: I think it's the built in version [19:33] aisrael: let me try it, just a sec [19:33] so ....here is a big missing step.....lets say a user wants to bootstrap a local provider......there seems to be alot of missing info on what 3rd party packages are needed [19:34] aisrael: Same error as the last paste-bin'ed >_< [19:35] bdx: so, for most people, it's just been apt-get install juju juju-local and you're good to go [19:37] e.g. mongodb-server, lxc, juju-local.....I think a lot of people are really confused because they don't know to have the extra deps.....on top of that there are multiple versions and locations to get mongodb from ....if these are post juju install ops they should be doc'd up much better.....e.g. on a fresh trusty box after "sudo apt update && sudo apt upgrade && sudo apt install juju-local" I don't have lxc or [19:37] mongodb installed [19:37] bdx: i just did that in a fresh trusty container without issue (though it's a LXD container on a linux laptop [19:37] derekcat: ok. Let me try to recreate from another machine. [19:38] bdx: it's not mongodb, it's juju-mongodb, and you should have lxc installed! bdx what does `dpkg -l | grep juju` show? [19:38] http://paste.ubuntu.com/12691581/ [19:38] aisrael: Okey [19:39] bdx: this is what happened when I ran your install line on a fresh trusty machine: http://paste.ubuntu.com/12691585/ [19:39] no lxc, thats for sure....you have lxc installed, and mongodb-server after "sudo apt install juju-local"??? [19:39] what in the, how did you not get all the other dependencies [19:39] bdx: see my paste, it seems like your trusty vm isn't processing apt dependencies [19:40] bdx: did you install juju-local? [19:41] totally....but then nothing is for that matter....I have tried this on openstack instances deployed with up to date cloud imgs....vagrant trusty box, intel nuc - fresh trusty server....same result on all [19:41] bdx: that is so bizzare. [19:42] marcoceppi: following "vagrant init ubuntu/trusty64; vagrant up --provider virtualbox && vagrant ssh" I run "sudo apt update && sudo apt upgrade && sudo apt install juju-local" [19:44] bdx: huh, I've never used the apt command before, but it doesn't seem to have any flags that would interfere with package install [19:45] bdx: I'm going to try to ssh into my OSX machine at home. I'm not even sure if SSH is enabled or whatever but my next step is to replicate your setup [19:46] marcoceppi: I just re-init'd my vagrant box, ran apt update, upgrade, and install juju-local and now have what look to be the correct pkgs e.g. lxc, mongodb [19:46] bdx: most of the core developers are at a sprint right now, where I'm currently traveling to, so if I can't get an answer before I land I'll be surrounded by the people responsilbe for the tool ;) [19:46] bdx: bizzare! [19:47] I know...I'm tripping out here.... [19:47] o [19:47] ok [19:47] I'm now attempting a bootstrap to local [19:47] totally rad, I can get to my OSX machine [19:48] bdx derekcat: what version of OSX are you two on? [19:48] 10.10 [19:49] cool, I've got a Yosemite vm laying around with a fresh install [19:49] marcoceppi: awesome! [19:51] marcoceppi: I've a successfull local provider setup....I'll continue to monitor my installs and bootstrappings more closely to see if I can't reproduce more what I've been experiencing [19:52] bdx: awesome! glad you were able to sort that out, though it's totally weird what you were seeing [19:52] derekcat: I just installed juju-quickstart from homebrew on a clean yosemite machine, going to try to repro against aws [19:53] bdx: glad you got it working, becuase I have no idea how to install a dmg from the command line [19:53] derekcat: are you using stock python or installed from brew? [19:53] aisrael: should be stock python [20:02] derekcat: What's ruby -ropenssl -e "p OpenSSL::X509::DEFAULT_CERT_FILE" show? [20:03] marcoceppi: on to the second issue: "juju bootstrap -e openstack" [20:03] I've read through this: https://jujucharms.com/docs/devel/howto-privatecloud [20:03] bdx: yes, next on the docket! [20:04] aisrael: "/System/Library/OpenSSL/cert.pem" [20:04] derekcat: great, thanks! Closer to recreating this [20:04] aisrael: ^_^ [20:05] marcoceppi: when I "juju bootstrap -e openstack --debug" : http://paste.ubuntu.com/12691759/ [20:05] bdx: okay, yeah, it's a simplestreams problem [20:05] it seems the default for tools url returns a 404 [20:05] this is always a pain to setup for the first time, but after it's setup it's fine [20:06] totally....I'm trying man [20:06] this seems like an issue, the default tools url: https://streams.canonical.com/tools [20:07] bdx: minus your credentials, what does the openstack stanza in your environments.yaml fiile look like? [20:07] bdx: OH, you're on 1.22.6 [20:07] I just realized that [20:08] I'm not sure if this is a noob question but does it make sense to use small single core (say 512mb RAM) VMs for state servers? [20:09] not bad, but the main archive lags behind the juju releases by a few minor releases because of the process to get updated releases into the main archive. We've ironed that out for releases going forward but there's still a but of a lag. While not directly related, you may want to add the stable juju ppa `sudo add-apt-repository ppa:juju/stable` which ill give you the latest juju 1.24.6 [20:09] hermanbergwerf: that should be fine for the stateserver, the biggest limiting factor is typically diskspace before anything else [20:09] bdx: ^^ [20:10] arrrrg....ok [20:10] omp [20:10] bdx: 1.22.6 isn't bad, it's just not latest, but it also won't directly solve your openstack issue [20:11] bdx: the environments.yaml stanza sans credentials would be a good start for working though that [20:12] derekcat: I was able to bootstrap and deploy the mediawiki-single against aws on a fresh 2.2.1 juju-quickstart on 10.10. The only difference is I had `bew install wget` run on that machine, which installs an updated version of openssl [20:14] marcoceppi: ok...upgraded to 1.24.6.... here's my environments.yaml openstack stanza : http://paste.ubuntu.com/12691827/ [20:15] marcoceppi: Hmm... What openssl version are you running? The only odd thing is that I'm on 10.10.4 instead of 10.10.5.. Just tried brew install wget but no difference with the juju-quickstart error [20:16] derekcat: very interesting [20:17] marcoceppi: now with update to 1.24.6 : http://paste.ubuntu.com/12691857/ [20:18] coreycb, mind doing the honors? gnuoy had +1'd, pending conflict resolution. https://code.launchpad.net/~1chb1n/charms/trusty/swift-proxy/amulet-update-1508/+merge/273445 [20:19] beisner, sure np [20:20] beisner, hmm I don't see the +1 [20:20] marcoceppi: derekcat: supposedly this is an issue with urllib2/httplib, related to python not having ssl support built-in. Trying to find verification of that now. [20:20] coreycb, oh it's in the comment [20:20] if happy to merge == +1 that is [20:20] gotcha [20:23] coreycb, one driveby on that. all along, swift-proxy's make lint was --excluding hooks o.O [20:23] beisner, ah convenient! [20:23] lol [20:24] `flake8 --exclude hooks unit_tests tests lib` not-ftw [20:24] aisrael: so odd, I'm not hitting it on my 10.10 [20:26] aisrael derekcat: this is what I have installed via brew: http://paste.ubuntu.com/12691913/ [20:29] marcoceppi aisrael: http://paste.ubuntu.com/12691936/ [20:29] marcoceppi aisrael: There's mien [20:29] marcoceppi aisrael: There's mine* [20:30] derekcat: the only main difference I see is no charm-tools, while this shouldn't have a change in juju-quickstart, I wonder if installing it (you'll probably want them anways) adds an updated version of a dep which addresses this ssl issue [20:31] marcoceppi: No dice after installing charm-tools [20:31] guh [20:31] >_< [20:32] Ok, I'm going to reload Yosemite on here and then update to Capitan.. I was overdue for this anyway >_< I'll jump back in a while and let you guys know what it's looking like. [20:33] aisrael marcoceppi: Thank you! Hopefully it's just some weird cruft.. BBL [20:34] bdx: fwiw, you can let derekcat know I have a vanilla 10.10.4 machine [20:35] bdx: did you create the imagestream metadata for your openstack? [20:35] I don't see it listed in the en vironments.yaml file [20:36] coreycb, thx man [20:44] maroceppi: derekcat is in the process of reimaging to 10.10.4 he will be back ASAP (5 mins) [20:45] marcoceppi: yes I did...omp [20:47] bdx: I don't see it listed in the environments.yaml file for image-metadata-url [20:58] marcoceppi: its a default....: http://paste.ubuntu.com/12692149/ [21:02] bdx: on openstack installs that won't work because we don't know how your cloud is setup. it works fine on public clouds, since we have a partnership with them and publish images to those clouds [21:02] bdx: since you're the one creating images in glance, you know the image ids, etc [21:02] so you need to create metadata to tell juju where those images are [21:02] I see [21:03] ie ubuntu-trusty = 7ad62ed4-6ba4-11e5-b31c-7b985e61fd10 [21:03] etc [21:04] bdx: the top half of that page goes though how to generate the metadata, I'll be honest I haven't generated metadata for an openstack cloud in a while, lmk if you run into an issue. I distincly remember last time thinking "a lot of this could be automated" [21:05] marcoceppi: yeah ....totally will... [21:05] bdx: I have about 30 mins of battery left, I'll be back online around 6pm PST [21:16] marcoceppi, aisrael: Hey guys I'm back [21:17] derekcat: hey, welcome back. aisrael stepped out to get some food I think and my battery is about 20 mins before failure [21:17] derekcat: I've got a chromebook, so I'll try to jump back online when this dies, but just an fyi [21:17] marcoceppi, aisrael: I'm about to pastebin another example, but it looks like it was cruft left on my system [21:18] marcoceppi, aisrael: http://paste.ubuntu.com/12692275/ [21:18] marcoceppi: so....after generating my metadata with "juju metadata generate-image -i e41dee50-872e-4f37-8aee-b090e020ddad -d /Volumes/WorkHD/Users/jbeedy/juju/juju_meta/" [21:19] marcoceppi: I get this message: http://paste.ubuntu.com/12692278/ [21:19] marcoceppi: following which, I run "juju bootstrap --metadata-source /Volumes/WorkHD/Users/jbeedy/juju/juju_meta/" [21:19] derekcat: okay, cool, so SSL is resolved, you're just going to have to wait for bdx to get a working openstack configuration done up. derekcat you'll basically have to edit the ~/.juju/environments.yaml file to put things like creds, and horizon ip in [21:19] marcoceppi, aisrael: fresh imaged computer, did the ruby install of brew, xcode CLI tools, brew install juju-quickstart, juju-quickstart mediawiki-single [21:19] marcoceppi: okey cool! [21:20] derekcat: in the mean time you can sign up for http://developer.juju.solutions/ a program where we hand out AWS cloud time/credentials to those working on juju and charming [21:20] marcoceppi: then get this output: http://paste.ubuntu.com/12692284/ [21:20] marcoceppi, aisrael: Thank you so much for helping us troubleshoot this! The fault is mine for having a crufty environment... [21:20] marcoceppi: Will do! [21:20] derekcat: no worries, glad we could get you going [21:20] marcoceppi: I feel this is due to the default tools url returning a 404 [21:21] bdx: interesting. [21:21] bdx: was that using --debug ? [21:22] no, omp [21:23] marcoceppi: http://paste.ubuntu.com/12692306/ [21:23] shoot....missed a key...oh well ...my bad [21:23] its a dev env anyway [21:24] bdx: can you...huh, bdx this will fix it temporarily, battery is marching to zero, but this is NOT the right way to solve this [21:24] oooooh nnnnnoooooo [21:24] bdx: add --upload-tools to your boostrap command [21:25] marcoceppi: still no luck even with --upload-tools : http://paste.ubuntu.com/12692320/ [21:26] bdx: can you paste the command you supplied? [21:26] juju bootstrap --upload-tools --metadata-source /Volumes/WorkHD/Users/jbeedy/juju/juju_meta/ --debug [21:26] --upload-tools should override whatever juju is doing and upload a copy of the local compiled tools [21:26] beisner are you around? [21:26] o/ hi marcoceppi [21:27] beisner: you probably have the most experience that's online right now, bdx is trying to setup access with juju to his openstack cloud [21:27] beisner: but he's getting an error about tools not found [21:28] bdx: what does `tree /Volumes/WorkHD/Users/jbeedy/juju/juju_meta/` show? [21:28] marcoceppi, bdx - hi. i've got ~15min before i've got to head out and about. but can be back online later in the eve too. [21:29] * beisner scrolls back, paste-hopping.. [21:29] beisner: ok, no worries, we can pick this up later if you like.... [21:30] marcoceppi: thanks for your help today [21:31] bdx: no worries, got a sec to pastebin the output of `tree /Volumes/WorkHD/Users/jbeedy/juju/juju_meta/`? I have an idea [21:31] bdx, can you paste sanitized environments.yaml, and `keystone catalog` and `keystone endpoint-list` and `neutron net-list` ? that'll give me a good idea of the whats and wheres. [21:32] marcoceppi: http://paste.ubuntu.com/12692371/ [21:32] beisner: omp [21:33] bdx: I think I see the problem [21:33] marcoceppi: ? [21:33] bdx: create a tools directory in your juju_meta dir, and add releases and streams into that [21:33] bdx: you should only have images and tools in the juju_meta dir, then inside tools a streams and releases dir [21:34] oooooh [21:34] marcoceppi: niceeee [21:34] bdx: give that a wirl, my bouncer will remain online, but I've got maybe 5 mins of batter left [21:35] if you get a "launching instance" you should be pretty good to go from that point forward [21:37] *suspense* [21:37] marcoceppi: http://paste.ubuntu.com/12692397/ [21:37] after making your modifications [21:37] beisner: environments.yaml -> http://paste.ubuntu.com/12692390/ [21:38] bdx: bugger. [21:38] bdx: you could try adding --agent-version=1.24.6 to your boostrap command, that *might* work [21:39] error: flag provided but not defined: --agent-version [21:39] bdx: perfect, just was I expected *eye roll* [21:39] bdx: I've got to hibernate, I'll try to jump on with my chrome book [21:40] beisner: keystone catelog -> http://paste.ubuntu.com/12692405/ [21:40] marcoceppi: ok [21:40] beisner: keystone endpoint-list -> http://paste.ubuntu.com/12692430/ [21:41] beisner: neutron net-list -> http://paste.ubuntu.com/12692442/ [21:44] bdx, the file:/// thing i'm seeing strikes me as odd. while that may work, it's different than my typical use. ie: [21:44] simplestreams.go:429 read metadata index at "http://10.245.160.50:80/swift/v1/simplestreams/data/streams/v1/index.json" [21:45] bdx, also, the environments.yaml says: use network ***5f89. but i don't see that net id in the paste. [21:46] which isn't the current roadblock of course, but once the streams are happy, will prob be an issue [21:47] beisner: gotcha....sorry about that .....the network id is currently up to date in my environments.yaml....that was an environments.yaml from a day or two ago...errr : network: 7af5fb70-7cde-4707-9564-2d770426b3de [21:48] bdx, ok cool. just mentally wiring up all the various assets ;-) [21:48] totally. [21:48] i see product streams endpoint. are you using it to feed glance images? [21:49] beisner: yea [21:49] and simplestreams ... are they in blobs? [21:49] beisner: should I use that as a tools url endpoint? [21:50] beisner: errr....im not sure ....omp [21:53] bdx, i've got to run, kiddo sports. here is all i define in my environments.yaml. simplestreams are in a blob (url above), and images are in glance. everything else is automagic. http://paste.ubuntu.com/12692489/ [21:54] beisner: awesome, I'll give you an update tomo on what I was able to figure out! Thanks for your help! [21:54] bdx, yw. see ya! === blahdeblah_ is now known as blahdeblah [22:54] great, I crashed my bouncer [22:54] bdx: How's it going? [23:01] derekcat: glad to hear that fixed it! [23:03] There you are marcoceppi [23:05] marco-airplane: whats up [23:06] bdx: any progress? [23:06] still no luck bootstrapping openstack provider....I'm tearing down my dev stack atm to reprovision with some new network hardware.... [23:06] bdx: ack [23:07] marcoceppi, silly bouncer === Spads_ is now known as Spads