TeTeThi, I'm following http://askubuntu.com/questions/65359/how-do-i-configure-juju-for-local-usage and I run into a problem at bootstrap already. the virtual net cannot be set up10:23
TeTeThttp://pastebin.ubuntu.com/851168/ http://pastebin.ubuntu.com/851170/10:23
TeTeTI try this in a vm, maybe the network setup does not work with the vm's network emulation?10:23
=== TheMue_ is now known as TheMue
koolhead17TeTeT: i tried it on a physical machine :P11:05
TeTeTkoolhead17: hmm11:07
SpamapSTeTeT: what release of juju? (apt-cache policy juju)15:03
TeTeTSpamapS: Installed: 0.5+bzr457-0ubuntu1 from preicse15:05
TeTeTSpamapS: is it supposed to work on a vm managed through libvirt and using the default virbr0 connected interface? Or does it need a real bridge to the ethernet card of the system?15:06
SpamapSTeTeT: it should work fine inside a VM yes15:07
TeTeTSpamapS: sudo virsh net-start default just does not work, maybe because of vm inside of a vm constraint?15:10
SpamapSTeTeT: no, the networking part of virsh is pretty simple15:14
SpamapSTeTeT: those "virbr" bridges are just bridges with no physical components.15:14
TeTeTSpamapS: ok. any idea how to get this working?15:15
SpamapSTeTeT: whats the problem with virsh net-start default?15:16
TeTeTSpamapS: error: internal error Network is already in use by interface eth015:16
benjiI'm having a problem running on EC2; the bootstrap appears to work then I run juju status, lie to ssh about verifying the fingerprint and then juju status hangs, never to return15:17
SpamapSTeTeT: sounds like you have it configured to be a "real" bridge instead of a virtual one.15:19
SpamapSbenji: does 'juju ssh 0' work?15:19
* benji tries15:20
benjiSpamapS: nope, I get "2012-02-21 10:20:05,876 INFO Connecting to environment..." and then a hang15:20
TeTeTSpamapS: doubt it, the vm has the normal 192.168.122.x address for virbr0 on the host15:20
SpamapSbenji: interesting...15:22
benjiSpamapS: this strace output looks like it might mean something to someone that knows what's going on: http://paste.ubuntu.com/851476/15:23
SpamapSbenji: hmmm.. you are running with default-series: precise ?15:24
benjiSpamapS: yep15:24
SpamapSbenji: have to go afk for a bit15:24
SpamapSbenji: try oneiric.. if that works.. we have a bug in precise15:24
benjiwill do15:24
benjiSpamapS: it works fine with oneiric15:34
SpamapSbenji: ok, do you want to open a bug report against juju? I can do it too if you don't want to.15:35
benjiSpamapS: I can15:36
SpamapSbenji: *thank you*!15:38
* hazmat upgrades to precise15:39
hazmatTeTeT, did you already have a bridge setup? it looks like its just the default libvirt bridge there15:41
TeTeThazmat: nah, I didn't setup an extra bridge.15:41
TeTeThazmat: oh, the reboot didn't change anything for me15:41
TeTeThazmat: which is mentioned in the askubuntu article15:42
_mup_Bug #937889 was filed: Hang on "juju status" with EC2 and Precise <juju:New> < https://launchpad.net/bugs/937889 >15:57
m_3benji SpamapS: I had problems with precise juju instances at the end of last week too... it couldn't install packages b/c of dep problems (txzookeeper iirc)15:57
m_3didn't get a chance to really debug it tho, but it's easily reproducable with just a bootstrap16:00
jcastroSpamapS: m_3: did you guys get a mail from wordpress for the server blog?16:03
benjiSpamapS: https://bugs.launchpad.net/juju/+bug/93788916:04
_mup_Bug #937889: Hang on "juju status" with EC2 and Precise <juju:New> < https://launchpad.net/bugs/937889 >16:04
m_3jcastro: looking now16:04
m_3benji: thanks, I'll +1 it16:04
jcastrom_3: oh so it worked? awesome, didn't know if it was set up to send mail, nice work!16:05
m_3jcastro: don't see it16:05
m_3easiest is probably postfix/gmail to send... I'd bet any direct sends'll be already blacklisted16:06
* jcastro nods16:06
jcastroour first issue, mail!16:06
m_3jcastro: I've got that config snapshotted somewhere if you want me to dig16:06
jcastrothis will be fun16:06
m_3jcastro: yup!16:07
jcastrom_3: hey so, you think you can charm up lp:summit this week?16:08
jcastroit'd be a nice win16:08
m_3jcastro: lemme look at it... I'm buried in stuff atm, but it wasn't too bad iirc16:10
SpamapSjcastro: still powering through my email16:11
jcastroSpamapS: it's ok it probably didn't send16:11
robbiewjcastro: you got blog access now16:20
jcastroyep, thanks16:20
jcastroman, wordpress needs FTP to import blogs16:20
jcastrom_3: everytime we run into a problem I'm going to say "octo"16:20
m_3jcastro: and I'll be happy to "+116:33
m_3" that16:33
m_3jcastro: zk charm reviewed and ready to promulgate16:34
* m_3 rings the promulgate bell16:34
jcastrooooh cute!16:36
* jcastro will blog that one16:37
jcastroanything special I should care about, or is it in the readme?16:37
m_3jcastro: readme's on the way still... it's almost eod for jamespage too so prob won't happen until tomorrow16:40
jcastrook no worries16:40
* jamespage is going to charm hbase tomorrow16:41
jamespagem_3: I was thinking of using dotdee to manage the config for hbase - have you seen any use of it in charms so far?16:43
m_3jamespage: have not16:44
m_3I think that's a great idea16:44
jamespagem_3: it would mean that you could generate the part of the config file associated with the event without having knowledge of the rest of the configuration16:45
m_3some charms have used a combo of config snippets and sed... but dotdee would be a little less manual16:45
m_3I've been experimenting with calling cheetah from the command line... (exporting vars into the env)... but it's... eh16:46
SpamapSsed is a fail IMO16:47
SpamapSwe should be building config files from scratch or using dotdee16:47
m_3sed can actually be a simpler/cleaner solution for small config changes16:49
m_3and easier to maintain b/c it's just pure diffs16:49
m_3but yes, for large-scale or complex config... it sucks16:49
jcastrom_3: hey the IP for that blog instance won't change will it?16:51
m_3yeah, it totally will... lemme attach an elastic IP real quick... hang on16:52
jamespagem_3: I did some cheetah stuff for tomcat7 I think16:56
m_3jcastro: ok, can you point the server blog url directly to that IP addr?  I can add a url like xxx.markmims.com in front of it if we need to16:57
jcastrohey should juju ssh blog/0 work?16:58
SpamapShazmat: is that reboot support I see landing in lp:juju ?!16:58
jcastrom_3: hey so does assigning that IP break the existing aws URL?17:01
m_3jcastro: you might need to specify environment... juju ssh -efido blog/017:02
m_3jcastro: checking on everything now17:02
hazmatSpamapS, yup, and upstartification, and all kinds of yummy goodness17:04
jcastrom_3: I get a connection timed out error17:04
m_3jcastro: wow... it looks like aws just removed the old dns entry17:05
jcastroand the ec2 url just stopped working for me altogether17:05
m_3gave it a new one that matches the elastic ip17:05
jcastrohah, awesome17:06
m_3but now it seems like juju's lost17:06
jcastroyep, I was going to ask, did we just find a new bug?17:06
m_3I've done this before without this problem... lemme poke around and see what happened17:07
jcastroAnd we're off to a great start!17:07
m_3in fact, it's up without this problem in another environment17:07
jcastrom_3: save your history, this will be a good post.17:07
m_3nice, now juju status reports different addresses for the service unit and the machine17:08
m_3it's trying to ssh to the old one that the service unit shows (and not getting through)17:09
m_3jcastro: you can still get in using the explicit machine id... 'juju ssh -efido 2'17:10
jcastrojamespage: thanks for the promulgation!17:20
jamespagejcastro, no problemo!17:20
_mup_Bug #937949 was filed: juju status shows addresses that are out of sync <juju:New> < https://launchpad.net/bugs/937949 >17:21
m_3jcastro: ^^17:21
marsHi guys, I have a question about installing from Launchpad private PPAs: is there a good charm cookbook recipe for doing this?17:29
marsThis looks good for a start: http://charms.kapilt.com/~openstack-ubuntu-testing/precise/nova-volume/hooks/nova-volume-common17:29
m_3mars: https://code.launchpad.net/~james-page/charms/precise/zookeeper/trunk is another example17:33
m_3it's pretty straightforward once you have packages _in_ the ppa17:33
m_3the charm's install hook just add-apt-repository and then apt-get update, then apt-get install17:34
m_3getting packages into the ppa is a whole other ballgame :)17:34
marsm_3, thanks, that is a nice way to do public archives17:35
marsPrivate ones require a bit more work, what with the custom URL and all.  AFAIK add-apt-repository doesn't handle them17:36
marsThe two recipes can probably be merged to what I want: install from a private PPA, public PPA, or archive.17:38
m_3mars: ah, gotcha... sorry17:39
m_3yeah, any ppas outside of launchpad seems like the tough thing with that would be key exchange17:39
m_3but I really don't know17:39
m_3we try to make sure any downloaded payloads can be cryptographically verified... for the charms in the charm store17:40
m_3for other charms, anything goes... pulling from github for node.js, npm, gems, etc is pretty common17:41
SpamapSm_3: whats the txzookeeper problem exactly?17:51
m_3SpamapS: boostrap a precise environment... ppa or distro doesn't matter17:52
m_3then ssh directly to the instance (juju ssh hangs)17:53
m_3dig through the log... packages aren't installed b/c of a dep problem (I vaguely remember it being txzk...)17:54
m_3I can reproduce and dig out the logs if you want, just lemme know17:55
m_3SpamapS: ^^17:57
m_3SpamapS: added your key to ubuntu@ec2-174-129-55-132.compute-1.amazonaws.com17:58
SpamapSm_3: that looks like just plain broken images18:00
=== grapz_afk is now known as grapz
m_3SpamapS: it's strangely similar to my desktop problem atm: http://paste.ubuntu.com/851649/18:03
SpamapSm_3: yours looks like an out of sync mirror..18:04
m_3oh, nice... /me fixing that!18:04
m_3very scared I have to reinstall18:04
SpamapSm_3: definitely not18:05
SpamapSm_3: just point at us.archive.ubuntu.com18:05
jcastroSpamapS: m_3: incoming mail about a charm contest, please review by end o business today, so I can launch this badboy18:05
m_3jcastro: roger roger18:05
m_3SpamapS: yeah, taking my apt-cacher-ng out of the picture18:06
m_3SpamapS: no change18:08
SpamapSm_3: have you tried dist-upgrade ?18:09
SpamapSm_3: or apt-get -f install ?18:10
m_3yup, same18:10
SpamapSm_3: that does not make any sense.18:10
m_3variations of -f or --fix-xxx didn't seem to do much18:11
SpamapSm_3: you have something damanged.. what version of gnome-control-center?18:11
m_3I hadn't been using any mirrors.... other than apt-cacher-ng18:11
m_3gnome-control-center 1:3.2.2-2ubuntu818:12
SpamapSm_3: 1:3.3.5-0ubuntu2 is the latest. It should be installing that18:12
m_3it was an upgrade from oneiric and not a fresh install18:12
SpamapSmine too18:12
SpamapSmine goes back to 10.10 :)18:12
m_3the juju one is more important though18:13
SpamapSyes I'm looking into the juju one18:13
SpamapSI think thats just broken images18:13
m_3jcastro: comments in on the charm contest18:38
SpamapShazmat: https://code.launchpad.net/~clint-fewbar/juju/use-packages-yaml/+merge/9404019:36
SpamapShazmat: fixes running juju w/ precise instances19:37
SpamapShazmat: oddly enough.. I used lbox propose, but I don't see it talking to rietveld. :-P19:38
* SpamapS lunches19:38
hazmatSpamapS, it needs a -cr flag to make it go to reitveld19:40
SpamapSah, next time.. its trivial anyway19:40
SpamapSbenji: hey we found a resolution for the bug you reported this morning21:46
SpamapSI should say, a few hours ago.. might not have been morning21:47
benjiIt was morning somewhere.21:47
SpamapSbenji: thanks for trying out precise. :) The problem was that libc6 was updated between alpha1 and now, so debconf was prompting for some questions21:57
SpamapSAnd the real underlying problem was that we were doing apt-get in runcmd instead of listing the package in cloud-init's 'packages' line21:58
m_3SpamapS: did you merge and kick off a ppa build?21:59
benjiSpamapS: interesting; good turn-around time on the fix21:59
m_3SpamapS: nevermind... I see22:03
hazmatSpamapS, that seems a little strange only because by default will use nightlies builds on ec2, i suppose that's not the case for openstack22:08
SpamapShazmat: no, we use released builds by default on ec222:19
SpamapSdef get_current_ami(ubuntu_release="oneiric", architecture="i386", persistent_storage=True, region="us-east-1", daily=False, desktop=False, url_fetch=None):22:20
SpamapS    data["version"] = daily and "daily" or "released"22:20
SpamapShazmat: thats the more conservative approach22:20
SpamapShazmat: trouble is, there's no way to override it22:20
SpamapSthough I suspect that will come with the full implementation of constraints22:20
hazmatSpamapS, we should be using the nightlies, we tell cloud-init to do an update/upgrade22:22
hazmatso we're just wasting bandwidth22:22
hazmatand it also means we get newer kernels22:22
SpamapShazmat: true!22:22
SpamapSless repeatable deploys with that strategy22:23
SpamapScan't tell you how many times I had version skews drive me *NUTS* in tracking down problems at 3am22:23
hazmatSpamapS, is the reality any different if we're doing an upgrade on the machine prior to setting up juju?22:23
SpamapShazmat: I'm suggesting that the upgrade is also a problem22:24
SpamapShazmat: I think ultimately we just need ways to say "fire all install hooks" on a service so that you can re-assert versions22:25
* hazmat nods22:25
hazmatversion skew pincer movement, wiped out entire roman legions, didn't even need the elephants at cannae22:25
hazmatyeah.. i can see it both ways22:26
hazmatif your reproducing or adding units to the service, you definitely want the same versions, if your deploying fresh, i'd see wanting to have the latest stable-updates applied22:27
hazmatas a goal22:27
hazmatSpamapS, sounds worth some more discussion on list to poll a larger audience22:28
SpamapShazmat: perhaps add-unit should not do the update/upgrade22:35
SpamapShazmat: another thought is to defer updates/upgrades to charms always22:35
hazmatSpamapS, but that could be at a different version delta than the original unit22:35
hazmatSpamapS, ie. if their off releases, and the first unit did the update/upgrade, than the second unit is stuck at the base image version.. doesn't really make sense22:36
SpamapShazmat: right, so really, perhaps update/upgrade should be pushed off to charms.22:37
hazmatSpamapS, i think its probably more of use to use the nightly unless its an add-unit in which case we use the previously used image, and we don't update/upgrade.. BUT.. there are lots of management tools and probably colo services that might also bbe doing package management22:37
SpamapShazmat: *or* disconnected from deploy/add-unit22:37
SpamapShazmat: I think we should just document how it works now, and think about how to improve the "update the whole service" story22:38
hazmatSpamapS, that's fair, although i think it could use some discussion on the wider list as well to help advance the story22:39
SpamapShazmat: I asked earlier.. but saw no response.. did I see rebooting landing in lp:juju ?22:44
jimbakerSpamapS, hazmat replied at 10:04 MST: SpamapS, yup, and upstartification, and all kinds of yummy goodness22:51
jimbakernice to see those features land, as i mentioned in #juju-dev at the time :)22:52
SpamapSahh ok cool!22:52
SpamapSThats like.. huge22:52
jimbakerSpamapS, indeed!22:52
_mup_Bug #863526: Juju agents do not handle reboots <production> <juju:Triaged> < https://launchpad.net/bugs/863526 >22:53
SpamapSSo, thats kind of a meta-bug, but does a reboot work now?22:53
* SpamapS tries it22:53
hazmatSpamapS, you can kill any juju agent and it should do the right thing22:53
hazmatmad props to fwereade_ who did the heavy lifting22:53
hazmatwhich reminds me, i should circle back to the branches i had waiting on that22:54
* SpamapS bootstraps and then reboots to see what happens22:54
hazmatSpamapS, that might not work ;-)22:55
hazmatSpamapS, its more that agents can be restarted, and killed at arbitrary points and do the right thing22:55
* SpamapS would really like to close some 'production' bugs22:55
SpamapShazmat: any reason that might not work?22:55
hazmatSpamapS, the session expiration stuff hasn't landed, as it was waiting on the restart work as a mechanism22:56
SpamapSso if the reboot doesn't happen in like, 3 seconds, something bad happens?22:56
hazmatSpamapS, if the zk server advances the clock and expires the session, the agents might not handle that.. i dunno there is some support there for killing old sessions when the process comes backs .. so it might work22:57
hazmatbut if the agent is alive when the session expiration happens then they don't do anything for it22:57
SpamapShazmat: well in this case, I'm rebooting zookeeper too...22:57
hazmatSpamapS, right, but zk will effectively advance the clock on all extant sessions when it comes back up... like i said it might work, i just can't guarantee it yet22:58
SpamapShazmat: I don't understand what advance the clock means, and I don't understand why an expired session does anything. :-P22:59
SpamapSdoesn't the agent just start a new session?22:59
hazmatSpamapS, it will when it starts up, but not if the session is expired while its up23:01
SpamapSActually the agents are failing on start23:01
SpamapSjuju.errors.JujuError: No session file specified23:01
hazmatSpamapS, hmm. they should all have session files specified if the env was started with the latest trunk23:02
SpamapS    JUJU_ZOOKEEPER=localhost:2181 python -m juju.agents.machine -n --logfile=/var/log/juju/machine-agent.log23:03
SpamapS    --pidfile=/var/run/juju/machine-agent.pid', 'JUJU_ZOOKEEPER=localhost:2181 python23:03
SpamapSbootstrap seems to not use the upstart yet?23:04
hazmatSpamapS, quite possible, its a large set of changes that just landed on trunk.. i'll do some additional qa testing now23:06
SpamapS./juju/providers/common/tests/data/cloud_init_branch_trunk:runcmd: [sudo apt-get install -y python-txzookeeper, sudo mkdir -p /usr/lib/juju,23:06
SpamapSmissed some apt-get's23:06
hazmatSpamapS, you want the cake or egg  treatment ;-)23:07
SpamapShazmat: the egg goes on my face.. and the cake.. well23:08
hazmatbut so much tastier23:08
SpamapShazmat: stand down23:11
SpamapSmy local version of juju was the distro version23:11
hazmatcool, i do remember testing that earlier (killing machine agent), it looks ok locally.. still not 100% sure about the restart capability23:15
SpamapShazmat: but there are branches that will solve that in flight?23:23
hazmatSpamapS, yes23:24
hazmatheading out to check out a user group meeting, g'night23:24
SpamapSThe system is going down for reboot NOW!23:25
* SpamapS crosses fingers23:25
SpamapShazmat: have fun23:25
SpamapSinitially it looks to have worked quite nicely23:26
SpamapSHeh.. it helps that we reach runlevel 2 at 7 seconds.23:26
SpamapS2012-02-21 23:25:19,466:2636(0xb73896c0):ZOO_INFO@zookeeper_close@2304: Closing zookeeper sessionId=0x135a235ffb30001 to []23:27
SpamapS2012-02-21 23:25:47,188:576(0xb74ed6c0):ZOO_INFO@log_env@658: Client environment:zookeeper.version=zookeeper C client 3.3.323:28
SpamapSlooks like it worked fine for machine/provisioning agent23:28
SpamapSsweet... and they can be restarted with 'service juju-machine-agent restart'23:30
SpamapSI just noticed23:52
SpamapSam I not being asked to verify ssh keys now?!23:52
SpamapSis that landed?23:52
SpamapSbecause if it is.. woot!23:52
SpamapSif I"m just dumb and fixed my .ssh/config to not be asked.. then ignore moe23:52
SpamapSignore me too.. but really ignore moe.. that jaerk23:52

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!