=== CyberJacob is now known as CyberJacob|Away [00:27] did the latest juju release fix the local ssh problems? [00:31] stokachu: it shuld have [00:32] what problem are you having ? [00:32] Permission denied (publickey,password). [00:32] ERROR exit status 255 [00:32] this was from a bootstrap, then juju deploy wordpress [00:33] using juju debug-log [00:34] ive also started seeing agent-state-info: 'hook failed: "install"' [00:34] with the latest juju-core [00:35] stokachu: what was the command you typed ? [00:36] davecheney: sudo juju debug-log [00:36] and sudo juju deploy wordpress [00:36] stokachu: you don't need to use sudo [00:36] for the local povider you need to sudo *ONLY* for juju bootstrap and juju destroy-environment [00:36] this should be clearly marked in the documentation [00:36] if it is not [00:36] please let me know and I will raise an issue === paulczar is now known as zz_paulczar [00:37] i believe debug-log does not work on the local provider [00:37] but the logs are in ~/.juju/local/something/seomting [00:37] we loopback mount them there [00:38] re-running with just juju deploy wordpress [00:45] davecheney: http://paste.ubuntu.com/6356219/ [00:45] ran with `sudo juju bootstrap --upload-tools; juju deploy wordpress` [00:46] stokachu: yea, that's the bug I hit, that's not really a bug. [00:46] stokachu: le sigh [00:47] #1247299 [00:47] <_mup_> Bug #1247299: apparmor blocks cgroup-lite from mounting [00:47] ah [00:47] rick_h_: is this the thing that slipped into 12.04.3 while we were in SF ? [00:47] stokachu: I hit that because I was installing build-essentials in my install hook, which grabs cgroups and that fails in lxc [00:47] davecheney: not sure when this hit. I've just started working on my own personal charm and hit that bug this weekend [00:48] rick_h_: i guess wordpress does something similar i havent looked [00:48] but it's marked invalid since it's got a lxc workaround [00:48] davecheney: however, I'd say it's a bug against the juju lxc approach then perhaps? [00:48] davecheney: I was going to bring it up on Monday when more folks were around [00:49] http://www.mail-archive.com/lxc-users@lists.sourceforge.net/msg03673.html [00:49] rick_h_: it is monday ... [00:49] I know it's not everyone's approach, but I like to do my python stuff in a virtualenv and need build tools and -dev packages to be able to pip install some things into it [00:49] thats what i used [00:49] davecheney: not here :P [00:49] i ran into this when using vagrant-lxc and maas [00:49] had to create a separate aa profile [00:50] stokachu: right, you turned off apparmor [00:50] app armour has done more to improve the security of linux by scaring people away than any other 'security technology' [00:50] I think juju should work with our setup ootb better [00:50] i didnt turn it off [00:50] rick_h_: i agree [00:50] and it did right til last week [00:50] stokachu: sorry, I thought you did what this therad mentions > Alternatively, you could set "lxc.aa_profile = unconfined" which would [00:50] i added "mount -> /tmp/*/*," to /etc/apparmor.d/lxc/lxc-default-with-loops [00:50] > turn off apparmor entirely for the container. [00:50] stokachu: ah, ok. Sorry, missed which approach you took. [00:50] :) [00:51] i initially turned it off but decided against it [00:51] davecheney: so yea, suggestions on how best to move the bug from lxc to juju appreciated. I'm not really sure how/who best to bring up the point. [00:51] rick_h_: thumper is around [00:51] but I was :( when I hit roadblocks trying to charm my own stuff this weekend [00:51] he wrote the local provider [00:51] oooh, I like bugging thumper :) [00:51] and I thought that he knew about this bug [00:52] * thumper isn't round [00:52] been exercising and everything [00:52] and can blame all things on him wahoo! [00:52] lol [00:52] thumper: I'd like my stuff to work kthx :P [00:53] * davecheney hands thumper his rimshot [00:53] thumper: #1247299 is biting a bunch of us and lxc says it's not their problem. How do we make it juju lxc's problem? [00:53] <_mup_> Bug #1247299: apparmor blocks cgroup-lite from mounting [00:53] * thumper looks [00:54] my problem showed up when trying to mount ephermeral images from /tmp [00:54] thumper: and mine from trying to install build-essentials [00:55] ugh [00:55] i just want to deploy joomla :( [00:55] lol [00:55] stokachu: we've got to start boxing up a cake to send to thumper :) [00:56] * thumper doesn't want a cake [00:56] but I'll accept a 24kg kettle bell [00:56] lol [00:56] rick_h_: hah my girl can make gluten free cakes [00:57] rick_h_: so installing build-essential triggers this problem [00:57] ? [00:57] thumper: yea, did for me [00:57] thumper: and moving to aws caused no issues [00:57] stokachu, rick_h_: can I get you both to comment on the bug with issues and reproducability? [00:57] I'll put it on my short term todo list [00:57] sure [00:57] ta [00:57] thumper: sure thing [01:00] thumper: ok updated, thanks :) [01:00] sp [01:00] np [01:01] marcoceppi: what do I need to run to create the framework of a charm? [01:01] marcoceppi: I want to make a charm that installs build-essential and compiles hello_world.cpp [01:01] and logs the output [01:02] thumper: install charm-tools from ppa:juju/stable then run `juju charm create ` [01:02] thumper: working on pushing up my demo charm and getting a log copy for the bug [01:02] marcoceppi: ta [01:02] marcoceppi: why are you working sunday night? [01:03] stokachu: juju debug-log and juju ssh still do not work for local provider. You can find the logs in ~/.juju/local/log and you can ssh with juju ssh [01:03] thumper: because the charm queue is still backed up [01:03] those are more things on my todo list... [01:03] it keeps me up at night in the fetal position [01:03] marcoceppi: ok thanks that was my other problem [01:04] stokachu: np! [01:04] we really /really/ should document that in the docs [01:04] * marcoceppi creates a task [01:04] marcoceppi: ah sweet, juju ssh wordpress/0 works [01:04] that at least unblocks me for the time being [01:05] stokachu: yeah, you can still do everything like the other providers, just have to do it in a slightly different way [01:05] local provider is special ;) [01:05] lol cool man [01:09] so with cgroup's do we need to add a profile to allow `mount -> /sys/fs/cgroup/*` [01:11] thumper: bug updated with comment, link to the charm, and the unit log showing the error [01:11] rick_h_: ta [01:12] https://github.com/davecheney/charms/tree/master/raring/docker [01:12] ^ docker charm, awesome, or heresy ? [01:14] davecheney: awesome heresy! [01:14] (Y/Y) ! [01:14] I don't know. I need to look up docker again. I don't get it, I thought it was just a pretty api around lxc. Everyone is going more gagga so I must have missed something [01:15] marcoceppi: seriusly, the docker install process is a piece of shit [01:15] curl some brogramming thing [01:15] add some packages [01:15] lol [01:15] recompile your modules [01:15] etc [01:15] davecheney: oh, you mean just like rvm and every other cool package out there? [01:15] so I made it into a charm [01:15] curl pipe to bash as root! What could do wrong? [01:15] FFFFFFFFFFFFFFFFFFFFFFFFFFUUUUUUUUUUUUUUUUUUUUUU cached charm! [01:16] davecheney: yea, doing local charm dev needs a --no-cachy-cachy option [01:16] davecheney: your charm lacks configuration and hooks. Also please don't make charmers the maintiner [01:16] so i think the lxc issue has to be solved within the lxc-ubuntu-cloud template along with apparmor [01:16] rick_h_ davecheney -u flag should do the trick? [01:16] marcoceppi: yea, but then I do a diff and see my revision went from 2 to 10 [01:17] and before merging go back and change it to 3 [01:17] where the lxc-ubuntu-cloud template sets a aa_profile customize directive for an apparmor profile we'd write [01:17] rick_h_: echo "revision" >> .bzrignore [01:17] nobody cares about that file anymore [01:17] rick_h_: to be fair, i should be yusing --upgrade-charm [01:17] marcoceppi: I thought that was the point of -u, to auto increment that [01:17] but it didn't make sense in this instance [01:18] marcoceppi: i forgot that as well [01:18] marcoceppi: I guess it doesn't make sense that it would work withuot the file since there's nothing to mark upgraded? [01:18] rick_h_: it is, but it only matters for local deployments. Everything else doesn't care about revision [01:18] rick_h_: if you delete the file juju deploy will just create it again [01:18] marcoceppi: right, so to work on the charm you need that file to exist to dev on it. [01:18] marcoceppi: ah, didn't realize that [01:19] rick_h_: right, but as you may well know, outside of local dev, the revision file gets changed dynamically by the store [01:19] I <3 the store sooo much [01:19] % juju ssh docker/0 -- sudo docker run -i -t ubuntu /bin/bash [01:19] ^ is this to meta ? [01:19] marcoceppi: please lets not talk about cs revisions [01:19] marcoceppi: also wanted to bug you about charm helpers sometime. I tried to use them this weekend. I noticed that in the juju-gui charm we've got a charmhelpers.py to use, but it's a ton larger now and has a bunch of files that are crazy (juju-gui file?) with their own deps/etc. [01:19] it's only monday [01:19] i don't need to cry on monday [01:20] lol [01:20] davecheney: eh, it could work. I saw someone's charm actually allowed you to send a docker config file via config-changed [01:20] marcoceppi: I ended up giving up and just doing shell vs trying to get the helpers into the charm [01:20] davecheney: it's still sunday here, I'm already crying ;) [01:21] rick_h_: yeah, I have plans to reign in charm-helpers. Documenting what it does is first on my list, packaging is a close second [01:21] rick_h_: there's a charm school video on it if you want to watch how it works for an hour [01:21] marcoceppi: k, I wasn't sure how that workd work. The install hook would need to do the install and you couldn't use it there then? [01:22] rick_h_: well, currently charm-helpers is embedded in the charm [01:22] rick_h_: so it would already be available [01:22] if you're using lp:charm-helpers [01:22] marcoceppi: ok, so the current *correct* way is to download and embed it then? Keeping it up to date via? [01:22] rick_h_: there's an update tool [01:22] marcoceppi: yea, I figured embedding was easy when it was a single file, but it has no upgrade path and such then [01:23] rick_h_: you basically create a .charm-helpers file and there's an executable you can run when you want to sync stuff [01:23] marcoceppi: oh, must have missed that when going through the code [01:23] It was born out of a need to keep remote deps down for IS. But now that the library is more mature it's time to properly package it in a ppa [01:23] so people don't have to do this [01:24] with a goal to get it in to the archive [01:25] cool, well I wanted to try it out this weekend. Heh, between that and the cgroups issue spent a couple of hours working on the charm and managed to get one config value in and working [01:25] wheeee [01:25] woo who! [01:25] rick_h_: http://www.youtube.com/watch?v=6kWfLujVwNI [01:26] some light video watching for you ;) [01:26] marcoceppi: yea, I thought I was cool trying to go through the source code and looking at the gui charm since I knew it used it. :P [01:26] marcoceppi: I wonder if I could make something like juju-docker plugin that would pipe the Dockerfile to your docker/0 unit ? [01:26] davecheney: subordinate would be cool too [01:27] davecheney: s/would be cool/might be another way to do that as well, I don't know though/ [01:27] I should probably just go use docker and get an idea for it [01:27] It seems cool, but I don't see what it solves that vagrant doesn't already do [01:32] jamespage: adam_g: Do you guys want eyes on your openstack charm in the queue? I know we've typically left those up to your review but I'd be happy to move it along [03:27] marcoceppi: simplest explaiantion of docker [03:27] checking your container into git [03:27] want to move your containre to a new provider [03:27] just check it out over there === freeflying is now known as freeflying_away === freeflying_away is now known as freeflying === thumper is now known as thumper-afk [06:13] morning all [06:13] hey wallyworld_, sorry I'm late, still want to chat? [06:21] jam: hi, just got back from getting kid from school, he was meant to have cricket training but it got called off so i got a phone call to go get him. we can talk now if you want [06:21] wallyworld_: sure, I'm there now [06:29] Hello, I just went through the juju demo, installing wordpress and mysql on an Amazon EC2 instance. There were no error messages, but when I go to that URL I get 502 Bad Gateway [06:29] anyone here have an idea of what I should do to correct this? [06:32] ah, it was just a delay [06:32] nevermind === CyberJacob|Away is now known as CyberJacob === freeflying is now known as freeflying_away === freeflying_away is now known as freeflying [12:26] hm, I have juju-core 1.16.2 [12:26] yet bootstrap is *uploading* 1.17.0.1 tools [12:26] andreas@nsn7:~$ juju version [12:26] 1.17.0-raring-amd64 [12:26] but [12:26] andreas@nsn7:~$ juju bootstrap -v --constraints mem=1500M [12:26] verbose is deprecated with the current meaning, use show-log [12:26] 2013-11-04 12:24:12 INFO juju.provider.openstack provider.go:127 opening environment "canonistack2" [12:26] 2013-11-04 12:24:15 INFO juju.environs.tools tools.go:85 reading tools with major.minor version 1.17 [12:26] ... [12:27] oh, wait [12:27] juju-core 1.16.2-0ubuntu1~ubuntu13.04.1~juju1 [12:27] so the package is 1.16.2, but it calls itself internalle 1.17.0? [12:27] and there are no 1.17.0 tools in this cloud I suppose [12:31] oh, never mind, my mistake [12:31] ahasenack: it should attempt to bootstrap with the same client minor version [12:32] marcoceppi: it's my mistake, I had another "juju" binary in the PATH :( [12:32] ahasenack: ah === gary_poster|away is now known as gary_poster [13:12] marcoceppi: https://code.launchpad.net/~sidnei/charms/precise/haproxy/restore-services-from-relation/+merge/193773 sorry, the branch you merged had broken tests. i should have set it to 'work in progress'. also, probably good practice to run the tests before merging for those that have it now. [13:12] sidnei: I typically do, didn't realize this has tests [13:13] sidnei: let me roll back [13:13] marcoceppi: wait, no [13:13] sidnei: wait, tests pass [13:13] marcoceppi: this is a follow up mp that fixes the tests [13:14] marcoceppi: indeed, my bad. the uncommitted fix in my local branch broke the tests, i thought i had pushed it. [13:14] sidnei: do you want me to roll back or not? [13:14] marcoceppi: anyway, the previous fix was incomplete. please merge this additional mp [13:15] it was tested by ahasenack === rogpeppe1 is now known as rogpeppe [13:52] sidnei: merged [13:52] marcoceppi: thanks [13:57] anyone able to help troubleshoot an env that says it's bootstrapped and not bootstrapped at the same time? http://paste.ubuntu.com/6358869/ [14:00] mthaddon: if you ran juju bootstrap, and the instance didn't fire up, or went away, there's an "is_bootstrapped" file in the object store that tells juju that it's bootstrapped [14:00] mthaddon: juju destroy-environment to clean that file up, then you can bootstrap again [14:01] marcoceppi: is there a bug about this do you know? this is pretty troublesome for mojo [14:01] mthaddon: not that I know of [14:01] * marcoceppi looks [14:02] ah, the bootstrap node is in ERROR state [14:02] mthaddon: https://bugs.launchpad.net/juju-core/+bug/1212177 [14:02] <_mup_> Bug #1212177: bootstrap detection stops too early

[14:02] thx muchly === freeflying is now known as freeflying_away === zz_paulczar is now known as paulczar === teknico1 is now known as teknico === gary_poster is now known as gary_poster|away === gary_poster|away is now known as gary_poster [16:01] bcsaller, did you end up getting the bundles working? [16:01] with quickstart? [16:31] marcoceppi: should the execution_environment() function in the charm helpers be valid to call when you are in an install hook? [16:32] Azendale: which charm helpers? [16:33] marcoceppi: I'm looking at the one in etherpad-lite. How can I know/find the version? [16:36] Azendale: there's a charm-helpers1 and charm-helpers2. If there's a folder called "charmhelpers" that's embeded in the charm it's charm-helpers 2 [16:37] marcoceppi: ok, then, it's charm-helpers2 [16:37] execution_environment seems valid enough [16:37] the relation_ commands would be problematic [16:37] should probably be wrapped in a try: block for those specific relation calls and set to either False or an empty array [16:38] actually [16:38] these all look safe [16:38] each relation method seems to do the right thing [16:39] relation_get() is the one that is failing in the install hook [16:40] relation_get should return None based on the code [16:40] the try block catches ValueError to make it none [16:41] Azendale: what error is being raised? [16:41] marcoceppi: Not ValueError. Let me look [16:42] marcoceppi: it raises CalledProcessError [16:42] Azendale: could you show me the output again? [16:42] https://bugs.launchpad.net/charms/+bug/1247636 has a copy of it [16:42] <_mup_> Bug #1247636: etherpad-lite fails to deploy, install hook running get-relation [16:43] Azendale: hum, maybe the catch should be updated to catch subprocess.CalledProcessError [16:44] ah yes, I thought this looked familiar [16:44] Azendale: you brought this up earlier (or someone did) [16:44] marcoceppi: yes, I did [16:44] oh, there's our conversation! [16:45] marcoceppi: yep. I got a chance to get back to this this morning [16:45] I'm torn as for what the solution should be. [16:46] I think there needs to be a try block in the execution_environment method [16:46] I still think it erroring out at relation_get because no JUJU_REALTION_ID is provided is valid [16:49] marcoceppi: So, it looks like execution_environment() is intended to get all the valid information it can about the environment. Maybe make it check? [16:49] Azendale: yeah, it should try for the relation_get call and set the value to None during exception === paulczar is now known as zz_paulczar [18:38] hey all, I just hit this exact same problem on juju running saucy...after starting up dbus manually and a ton of headache, I can get the charm to deploy. Is there anything I need to be doing differently with the saucy bits? [18:38] http://askubuntu.com/questions/364714/juju-deploy-of-charm-mysql-in-maas-provider-failing-after-successful-bootstrap === zz_paulczar is now known as paulczar === thumper-afk is now known as thumper [19:44] jcastro: mmm, check this out http://rtomayko.github.io/ronn/ronn.1.html [19:45] WHAT. [19:46] wish it wasn't a ruby gem. I want to use this with charm tools, so you can juju charm info and get a man page output from the readme [19:47] maybe I'll make like Ronn as a Service or something [19:47] heh [20:00] marcoceppi: Is there somewhere I should be branching charm-helpers2 from? I think I found a pretty good fix to the problem it was having with running get-relation in an install hook [20:01] Azendale: lp:charm-helpers [20:19] marcoceppi: I branched the charm helpers, but it looks like there is a change in the latest one that looks a lot like my change -- so I think a new version of the charm helpers might just fix the problem all together. What's the process for update the version of charm helpers a charm uses? (or is there somewhere to read about this?) [20:23] Azendale: there's a charm-helpers-sync tool, it's in http://bazaar.launchpad.net/~charm-helpers/charm-helpers/devel/view/head:/tools/charm_helpers_sync/README [20:23] it'll allow you to update the latest version of charm-helpers in the charm [20:23] then test, bzr commit, push to a branch and propose for merging [20:24] marcoceppi: ok, thanks :) === rogpeppe2 is now known as rogpeppe === paulczar is now known as zz_paulczar === zz_paulczar is now known as paulczar === paulczar is now known as zz_paulczar [22:35] oi, I'm sure I'm missing something simple, but I `juju upgrade-juju` from 1.14 to 1.16.2 and now all my agents are down... lol [22:43] bic2k: how long ago did you start the upgrade? [22:43] bic2k: it can take some time to get everything settled again [22:43] marcoceppi: more than an hour [22:44] bic2k: try sshing in to one of your nodes [22:44] marcoceppi: syslog shows it cannot connect to the API locally? [22:44] marcoceppi: I can SSH in to various nodes still, including machine-0 [22:44] bic2k: right, the nodes are up, are the agents all running? [22:46] marcoceppi: looks like jujud is running on the nodes still, still using 1.14 from the looks of it [22:46] that's probably why they're all reporting down [22:46] bic2k: has machine 0 been upgraded to 1.16.2? [22:47] marcoceppi: it has the updated jujus in /var/lib/juju/tools but is still running 1.16.2 [22:47] davecheney: who should I bug on core about upgrade-juju [22:47] marcoceppi: I mean it is still running 1.14.1 [22:47] bic2k: do the other nodes have 1.16.2 agents in /var/lib/juju/tools? [22:47] this might be a real simple update the init scripts by hand if so [22:48] but I'd like to collect logs from you if that's the case [22:48] marcoceppi: sure, just tell me what logs you want and I'll get them filled on a bug for you. [22:48] 1.16.2 is on all nodes/machines [22:48] bic2k: /var/log/juju/machine-*.log if you could pastebin that for me, I'd like to look at one of them [22:53] marcoceppi: I PM'ed the log [22:53] marcoceppi: lots of errors up top [22:53] I'll get the machine-0 log as well [22:53] I suspect there are some interesting points in there as well [22:53] yeah it looks like it did the right thing, but then failed to connect to the API [22:54] bic2k: can you pastebin one of the juju*machine* init files in /etc/init/ on a node? [22:54] marcoceppi: coming right up [22:55] * marcoceppi rolls the wheel of ping [22:56] bah, everyone is eod. thumper thoughts? [22:57] actually, this one is safe for public anyways: http://pastebin.com/qEKMk3hK [22:57] bic2k: what does /var/lib/juju/tools/machine-2/jujud point to? 1.16.2 or 1.14.0 ? [23:00] marcoceppi: machine-2 directory is symlinked to the 1.14.1 and unit-xxxxx-0 points to 1.16.2 [23:01] bic2k: yeah, outside of stabbing in the dark, I'm not sure where to go. Could you open a bug about this against juju-core with the logs you sent me (seems you've already stripped sensative information) as well as a tree of /var/lib/juju/tools [23:02] marcoceppi: sure, any ideas on what to do in the meantime? [23:02] marcoceppi: kinda deadlocked on management right now [23:02] bic2k: I can only think of things to poke and try [23:03] I dont' want to dig too deep in to the "oh lets try this crack idea" on a "production" environment [23:03] it'll make it hard to step out what I did. [23:03] I'm not even sure if you could revert back to 1.14.2 - I was going ot say you could update the sym link on boostrap to point to 1.14 but with schema changes in mongodb I dont' think that'll work sanely [23:04] marcoceppi: hmm, I feel like if I could fix that API connect error on machine-0 things would work themselves out... just not sure how to manage that service on that machine. Doesn't appear as a proper upstart job [23:05] bic2k: oh, it totally shouuld be [23:05] bic2k: there should be two upstart jobs, juju-db and juju-something-or-another [23:06] marcoceppi: oh weird, `service jujus-machine-0 status` works now [23:06] bic2k: maybe it was still in the process of churning over? [23:06] bic2k: is your bootstrap a micro? [23:07] marcoceppi: still on 1.14.1 [23:07] bleh [23:07] marcoceppi: smalls [23:10] bic2k: so jujud-machine-0 is still running on 1.14.1? [23:10] marcoceppi: yup [23:10] :\ [23:10] marcoceppi: logs show that it told the nodes to upgrade [23:10] marcoceppi: and it has a 1.16.2 in tools, so it staged the update... just didn't pull the trigger [23:11] bic2k: so, try running juju upgrade-juju again... [23:11] curious what that will do [23:18] marcoceppi: sure, let me finish filing this bug report and capturing some more logs first [23:18] bic2k: awesome, thanks for your patience [23:18] marcoceppi: thanks for walking me through this :-) === CyberJacob is now known as CyberJacob|Away [23:26] marcoceppi: filed, https://bugs.launchpad.net/juju-core/+bug/1247993 [23:26] <_mup_> Bug #1247993: upgrade 1.14.1 to 1.16.2 leaves machine agents not upgraded [23:26] marcoceppi: rerun of juju upgrade-juju with verbose shows it has no upgrades [23:26] marcoceppi: will wait a few minutes to see if it triggered anything [23:31] bic2k: any progress? [23:31] marcoceppi: na, it things it is already upgraded from the looks of it [23:32] * bic2k "thinks" not things... damn happy fingers [23:32] gotchya [23:34] is jcastro on? I think i may have found the issue with the quantum-gateway charm. It gave me a debug message about not understanding the shared-db-relation-changed hook and then it never writes any of the credentials to the nuetron.conf file [23:36] when i do "juju bootstrap" why is one of my maas nodes allocated to me? what does juju bootstrap do? [23:37] bjf: bootstrap creates a node in your cloud environment that does all the orchestration [23:37] it's how juju is asyncronous. So when you type juju deploy, you queue that event on the bootstrap node and it does it when ready [23:38] marcoceppi, is it unreasonable to use the maas server for juju orchestration? [23:38] marcoceppi, and should that node have been started up and running ? [23:38] bjf: no. Juju can use maas as well as lxc, aws, hp cloud (openstack) and others [23:38] yes, it should have [23:39] marcoceppi, ok, thanks, i'll go see what's up with that