=== andrewsmedina_ is now known as andrewsmedina
_mup_juju/enhanced-relation-support r11 committed by jim.baker@canonical.com00:50
_mup_Usage and impl details00:50
=== lifeless_ is now known as lifeless
m_3bac: might be easier to hit me up wth email this week for questions about the buildbot review02:38
_mup_juju/enhanced-relation-support r12 committed by jim.baker@canonical.com03:33
_mup_Completed impl plan and details03:33
=== andrewsmedina_ is now known as andrewsmedina
=== stub1 is now known as stub
=== Leseb_ is now known as Leseb
=== Leseb_ is now known as Leseb
jcastroSpamapS: charm school rehearsal!14:50
jcastroSpamapS: 3 minutes!15:12
ivokscan someone help me out here? what's exactly failing here? http://paste.ubuntu.com/860691/16:10
ivoksjuju destroy-environment otoh works :)16:24
james_wdoes someone have a pointer to this summit charm I'm hearing about?16:39
=== Leseb_ is now known as Leseb
=== tobin is now known as joshuaBRB
=== joshuaBRB is now known as tobin
robbiewm_3: ^^^^^^?18:50
robbiewivoks: hazmat jimbaker or bcsaller are your best bets for help there18:51
bcsallerivoks: the last line of your paste: 2012-02-28 11:08:53,092 DEBUG Environment still initializing. Will wait.18:53
bcsallerivoks: sounds like its still doing the bootstrap18:53
bcsallertakes a while18:53
ivoksbcsaller: it's not18:55
ivoksbcsaller: i've digged into it a bit more18:55
ivoksbcsaller: none of the juju services are started within the instance18:56
hazmativoks, are you on/ssh'd into the machine?18:56
ivokshazmat: i'm not now, but yes, i can ssh into it18:56
hazmativoks, which version of juju are you running?18:56
ivokshazmat: in an hour :)18:56
ivokshazmat: all precise18:57
ivokshost and guests18:57
hazmativoks, from ppa or the distro version?18:57
hazmativoks, cool, if you can log in and get the cloud-init logs that should be helpful18:57
ivokshazmat: i've noticed it complains about bad syntax for juju-admin18:58
ivoksand also both juju services said that --nodeamonize (or --nodaemon) is bad option18:58
hazmativoks, that would imply a client vs node version mismatch18:59
hazmatie different juju versions18:59
ivokshazmat: i'm reimplementing openstack as we speek, so i'll be able to reproduce this in 60-90 minutes (or less)18:59
hazmatthere was some large changes that landed last week, but they caused some incompatbilities between versions18:59
hazmativoks, reimplementing or redeploying? ;-)19:00
hazmativoks, cool19:00
ivokshazmat: both :/19:01
ivokshazmat: hardware and sotfware :D19:01
ivokshazmat: ok, i have system up and ready19:31
hazmativoks, the file i'd be interested in is.. /var/log/cloud-init-output.log19:32
ivoksgive me a second, i need to bootstrap it19:32
ivokshazmat: http://paste.ubuntu.com/860965/ <- that's from console19:39
ivokshazmat: well, same thing is in the log19:40
m_3james_w: summit's still in progress... it's a combo of pgsql, memcache, and a fork of lp:~michael.nelson/charms/oneiric/apache-django-wsgi/trunk with summit-specific setup19:43
james_wm_3, interesting, I'd like to take a look when it's nearing completion19:43
james_wI have another django thing I want to charm, so I'm keen for some convergence19:44
m_3james_w: sure, I'll ping you for summit help... I understand most of what's going on so far19:44
m_3but there are a couple of places where it assumes interactive installation19:44
james_win the init-summit stuff?19:46
m_3james_w: I'll push up to lp:~mark-mims/charms/oneiric/summit/trunk and ping you when there's more to look at19:46
m_3I'm planning on keeping with the puppet-based django charm for now... might back off of that later, but haven't decided yet19:46
james_wI think it's good as an example, but a set of helpers (puppet modules?) to make writing django charms easier would be better19:50
ivokshazmat: machine-agent.log shows: juju.state.errors.MachineStateNotFound: Machine 0 was not found19:53
ivokshazmat: i didn't look into juju internals, but does it ask nova-compute for the state of the node? i just noticed "AttributeError: 'dict' object has no attribute 'state'" in nova-compute's log19:55
hazmativoks, sorry was in a meeting19:58
ivokshazmat: no worries19:59
hazmativoks, can you pastebin the cloud-init/user-data .. its at /var/lib/cloud/instance/user-data.txt19:59
hazmatthis sounds vaguely familiar19:59
ivokshazmat: http://paste.ubuntu.com/861000/20:00
hazmatSpamapS, m_3 does that ring any bells.. juju-admin: error: unrecognized arguments: 2007-01-19 2007-03-01 2007-08-29 200720:00
hazmatits like its getting junk back from the metadata server20:01
hazmativoks, what version of openstack?20:01
ivokshazmat: 2012.1~e4~20120224.1291320:01
hazmativoks, if you do curl  what do you get back?20:02
ivoksand then bunch of dates20:03
hazmatthat's the problem20:03
ivokshazmat: http://paste.ubuntu.com/861002/20:03
hazmatsmoser, adam_g does that sound familiar.. metadata server returning back junk20:03
hazmati wonder if i can reproduce with devstack20:04
SpamapShazmat: yes I've seen that.. IIRC its from broken metadata service.20:04
ivoksfwiw, this is keystone only auth20:04
smoserwhy are you using 1.0 ?20:05
smoseri think it doesn't matter actually, but i wouldn' tuse that.20:05
ivokser... 1.0 of what?20:05
smoserin that url20:06
hazmatsmoser, because its stable and well known20:06
smoserdont use that and it works.20:06
smoserhazmat, 1.0 is probably missing all sorts of things you want anyway.20:06
ivoksthat gives the same outpu20:06
smoserits basically unknown to openstack20:06
smoseruse that20:07
hazmatsmoser, but its returning back its knowns, and it returns back 1.0 as a valid value20:07
ivoksnow that's right20:07
smoserhazmat, yeah, you're right. it does list it in the index.20:07
smoserwhich is obviously wrong.20:07
hazmatie. it should work, but its a bug in ostack imo20:08
hazmatwe can update it to a newer version.. but really we don't need much else besides pub/priv address and instance id from the md server20:08
ivokshazmat: is there anything i can do on a live system... i guess i need to change the path somewhere?20:12
smoserivoks, please open a bug on nova.20:12
ivokssmoser: ok20:12
ivoksnot sure how to phrase it :D20:13
smosermetadata service reports to support 1.0 but serves incorrect data20:13
smoserthen show your wget's20:13
hazmativoks, pls paste a link to the bug here, i'm going to explore it a little more20:16
_mup_Bug #942868: metadata service reports to support 1.0 but serves incorrect data <OpenStack Compute (nova):New> < https://launchpad.net/bugs/942868 >20:17
hazmativoks, thanks20:20
ivokshazmat: np20:20
ivoks2012-02-28 15:22:47,958 INFO Connected to environment.20:22
ivoksfinally :)20:23
hazmatsmoser, does devstack work on precise?20:41
hazmatthere's a bunch of oneiric only labeling on it20:41
ivokszul's packages work20:42
ivoksfrom the archives20:42
smoserhazmat, yes, it works.20:43
hazmativoks, smoser thanks20:47
smoserhazmat, by works, i mean maybe20:47
smoserit has worked. and has been broken.20:47
_mup_Bug #942868 was filed: metadata service reports to support 1.0 but serves incorrect data <juju:Confirmed> <OpenStack Compute (nova):New> < https://launchpad.net/bugs/942868 >20:48
zulhazmat: yes it does20:48
ivokshazmat: note that bootstraping juju with e4 will fail20:48
ivoksis there a way to pass additional info to cloud-init while doing juju bootstrap (like apt proxy)? i've seen a bug about this issue...20:55
hazmativoks, there isn't, although that one in particular is worth exposing i think20:59
jcastroSpamapS: 51 charms, did you promulgate something?21:07
SpamapShazmat: I'd love to see the full power of cloud-init exposed to people21:20
SpamapShazmat: by and large, we should abstract the things that make sense across all providers, but being able to slide your own cloud-config in would be quite useful.21:20
niemeyerSpamapS: This would render charms dependent on such cloud-init config..21:21
SpamapSniemeyer: I hadn't thought about it until you said charms, but really.. subordinate charms almost give us the same capability as cloud-init..21:24
SpamapSniemeyer: the only thing missing is the ability to do interesting things to the system in early-boot21:25
SpamapSwhich I don't think most people want21:25
niemeyerSpamapS: Agreed21:26
SpamapSwhat I do think people want is the ability to have their data on a RAID of EBS volumes, or the ability to install a bunch of extra stuff... both of those will work fine with subordinates21:26
SpamapSThough the EBS thing will still require manual intervention to attach the EBS volumes.. that would be true of cloud-init too actually.21:26
niemeyerSpamapS: That'll eventually be handled internally by juju itself21:34
SpamapSniemeyer: I've bee kicking some ideas around in my head about how it might work. I feel like we're still a world away from addressing it though.21:35
niemeyerSpamapS: As far as focus goes, agreed21:36
SpamapSniemeyer: like, I don't even know what to ask for. ;)21:36
niemeyerSpamapS: That's been on the table since first tech sprint, though21:36
niemeyerSpamapS: It's about management of volumes and relationship between volumes and units21:38
niemeyerSpamapS: This must be a first-class feature21:38
SpamapSniemeyer: yeah, I feel like the simples thing to do is to just track the volumes that get attached to EBS rooted machines.. and offer a way to say "add-unit --volume-id d-203494950"21:39
SpamapSniemeyer: but I know thats glossing over a lot of details21:39
jcastroSpamapS: launchpad workflow email on the list, I mention you explicitly to explain something so if you could that would be swell. :)21:41
hazmatwe need juju eatings its own tail first21:42
hazmatthen we can launch juju environment infrastructure services like volume management21:42
hazmatthey would be provider specific in the case of volume management21:42
hazmatie. perhaps ceph for maas/orchestra, ebs for ec221:43
hazmatand then have per unit volumes against the storage service, or in some cases against direct/ephemeral disk21:43
SpamapShazmat: at a simple level, those are all doable as subordinates now.21:44
SpamapSwhen it lands21:44
* SpamapS notes that Ubuntu 12.04 beta1 is dropping any minute.. w/o subordinates.. :(21:45
hazmatSpamapS, subordinates get a little messy here, and subordinates live in their principal's environnment21:45
hazmatso they'd be coming in after the principal, the sequence is wrong.. its much simpler if its in core imo.. yes it could be hacked between cooperating charms with intimate details..21:45
SpamapShazmat: the subordinate can be related to other services though right? So the ceph-client subordinate will be able to ask services where to mount their ceph-backed-block device21:46
hazmatSpamapS, first subordinate branches just landed21:46
hazmatSpamapS, yes21:46
SpamapSOrdering, once again, rears its ugly head21:46
hazmatSpamapS, but what's the lifecycle.. the subordinate can be added to the principal at any point in the principal's lifecycle.21:46
hazmatso the principal already has a bunch of data, and then your copying data around volumes and coordinating the unit's underlying service and relations to during that migration.. its pretty messy imo21:47
SpamapShazmat: the principle needs a way to delay full configuration until certain relations are established.. in this case.. the subordinate "where do I put my data" relation. :)21:47
SpamapShazmat: forget subordinates, this is true of other services too like nova api that have two options (local or remote database storage)21:48
hazmatSpamapS, again this is much simpler if something as intrinsic as storage is in the core21:48
hazmatSpamapS,  required relations are orthogonal, but also of import21:48
hazmatdeploying subordinates when the principal isn't a known running state, complicates their communication imo.. a unit doesn't respond to relation events till its running/started21:49
SpamapSI see the two use cases as equals until we have any kind of real discussion about storage.21:49
SpamapShazmat: so, the answer I have for the nova problem and ceph (it has this problem too) is to just have a config option giving the relations hints about when to store data.21:50
SpamapSSo for ceph, you just tell ceph "start-nodes=4"" and it delays data storage until there are 4 peer units.21:52
hazmatSpamapS, so i think we've walked around two solutions for required.. one is per the subordinate implementation, don't deploy actual units till the required relations are satisifed.. ie. in much the same way that subordinates units are actualized till the subordinate relation is established21:52
hazmatthe other is a unit don't join provider relations till its requires are satisifed21:53
hazmatwhich is actually a bit better/needed imo, else you get additional ordering issues around when is something fully operational.. ie deploy with dep relations doesn't nesc. solve the ordering around consumers of the service attempting to use it pre its initialazation.. but waiting till its dependencies are solved and running does.21:54
hazmatSpamapS, i don't think config option is the right solution21:54
SpamapShazmat: no, its the "current" solution21:55
hazmatfair enough21:55
_mup_juju/enhanced-relation-support r13 committed by jim.baker@canonical.com21:55
_mup_Reworked docs on changes to relation commands21:55
jcastroniemeyer: hey we did this already right "[niemeyer] drive dicussion about interface documentation on juju mailing list"?22:05
* jcastro is cleaning up spec work items22:05
niemeyerjcastro: Kind of..22:05
niemeyerjcastro: I did this before, maybe a couple of times even, but I wouldn't consider the problem as solved22:06
jcastromy bp is that we have a discussion, a solution or not isn't in my scope. :)22:06
niemeyerjcastro: LOL22:07
hazmatniemeyer, did you see clint's email to the list re env vars.. i asked him to post there for your feedback22:08
niemeyerhazmat: I missed that.. looking22:09
jcastrook I'll put "inprogress" until I am more desperate to close it22:09
hazmatjcastro, it wouldn't be hard to do some bespoke interface analysis for documentation22:14
hazmatits kinda of hacky.. but we could track all the keys being get/set22:15
jcastrohazmat: hey speaking of, we need to figure out how to style the generated sphix docs22:15
hazmatvia charm introspection22:15
jcastrosay something magical like "bootstrap can do all that for us easily"22:15
hazmatjcastro, ugh.. that means another rt... better start now if you know what you want..22:15
jcastroheh ok22:15
hazmatjcastro, i'd look around/google for sphinx themes22:16
hazmatand see if there's something reasonable out there22:16
jcastroI'd like to see if anyone else in the project is styling their sphix22:16
jcastromaybe we can steal22:16
hazmati don't think bootstrap whole sales is going to work with sphinx without some dev time22:16
hazmatjcastro, yup.. that's a good plan22:16
hazmatjcastro, i kinda of like the sphinx theme the celery folks did22:17
jcastroI don't see other ones styling it, I am debating if it's even worth the effort right now22:17
james_wjcastro, I'm pretty sure http://developer.ubuntu.com/packaging/html/ is sphinx22:17
hazmatoh.. nevermind that's the old one22:17
jcastrojames_w: oh nice, where'd you get that, dholbach?22:19
hazmatjcastro, actually it looks we don't need an RT we can just mod the source..22:19
hazmatthe makefile for the cron is the same22:20
james_wjcastro, yeah, he'll know for sure22:20
hazmathere's some.. http://pythonic.pocoo.org/2010/1/8/new-themes-in-sphinx-1-022:20
hazmatthese are the built in ones.. http://sphinx.pocoo.org/theming.html22:21
hazmatjames_w, cool.. it kinda of looks ugly but we grab the colors ;-)22:21
hazmatm_3, ping22:24
m_3hazmat: hey22:26
hazmatm_3, hey just wanted to check and see how charmrunner is working out22:27
m_3hazmat: working through some recursion problems with the planner22:27
hazmatm_3, oh.. do tell?22:27
hazmatm_3, which charm?22:27
m_3on gnaglia for some reason... lemme get you the logs... link coming22:27
hazmatm_3, typically that means a semantic in the charm error but not always22:28
m_3also need to coerce max_time into float for watch22:28
* m_3 grrrr latency22:29
hazmatm_3, i switched the project maintainer to juju-jitsu hackers.. if you need to merge something22:29
m_3hazmat: ^^22:30
m_3I'd like to take it down soon so lemme know when you're done digging through the workspace22:30
m_3hazmat: thanks for the ownership... I'll push the float commit22:31
niemeyerhazmat, SpamapS: Responded22:31
hazmatm_3, what's the channel for test notifications?22:31
m_3hazmat: ##charmbot-test22:32
hazmatah it is a double hash22:32
m_3trying to be polite with made-up channels :)22:33
m_3I'll figure out the problem (like you said, prob just syntax somewher in interface)22:33
m_3but that' pretty much status atm22:34
hazmatm_3, so besides ganglia its been okay?22:38
hazmatm_3, are you copying the juju-record output into the workspace?.. is that logs.zip?22:38
hazmatits more than just the logs but fair enough22:39
m_3hazmat: just walking through the list of fails... ganglia was the third type of fail22:39
m_3I need to re-run to test other timeout fails with new max_time... that was my next step22:40
m_3yeah, it's coming along nicely in general though22:41
hazmatm_3, yeah.. the service watcher timeout code was a little suspect i think22:41
hazmatit had too many ways of going at it22:42
m_3hazmat: there was another problem with the runner generating and reading plans from different places, but I'll push that back too22:42
hazmatm_3, sounds good22:42
m_3there's a bit of strange behavior in the watcher beesides timeout, but one at a time22:42
m_3cool... I'm gonna tear down the ganglia run then22:44
hazmatniemeyer, responded22:46
niemeyerhazmat: re-re-responded22:48
hazmatjamespage, wow that's seriously insane about jenkins master/slave23:33
hazmatthat's like the craziest design impl i've heard all year.23:34
niemeyerhazmat: Do you realize the irony there? ;-)23:34
hazmatniemeyer, yeah.. i'm a slave, no i'm the master ;-)23:35

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!