/srv/irclogs.ubuntu.com/2011/08/01/#ubuntu-ensemble.txt

SpamapSjcastro: hahahahah no I made that for the Cloud Days presentation like 3 months ago. ;)01:16
jcastroSpamapS: I didn't know about it before01:24
jcastroI am going to steal it01:24
SpamapSjcastro: the one on the wiki is svg ... make sure to steal *that* :)01:26
jcastroWe should ask someone on design to prettyfy it01:26
SpamapSyeah that would be cool01:26
SpamapSI'm hesitant to make the marketing assets more polished than ensemble itself tho ;)01:27
jcastrowell, I see people are using it and reporting the annoyances01:27
jcastrorobert collins just filed a few01:28
niemeyerGood morning all06:34
niemeyerfwereade: Morning07:49
_mup_ensemble/trunk r288 committed by gustavo@niemeyer.net07:53
_mup_Merged fix-status-scope branch from Clint [r=bcsaller,fwereade]07:53
_mup_Fixes ensemble status to work with multiple scope filters as per the07:53
_mup_help documentation. Tests were also adapted to reflect the arguments07:53
_mup_one should expect from argparse.07:53
=== jamespage1 is now known as jamespage
kim0Morning all08:08
niemeyerkim0: Morning!08:17
kim0niemeyer: hey .. oh you're up so early :)08:17
niemeyerkim0: Yeah, I'm actually about to step back to try to sleep some more08:17
kim0have some nice rest then08:18
niemeyerJust couldn't sleep for a while, and decided to get up and do something useful08:18
niemeyerThanks,  see you soon :)08:18
kim0:)08:18
fwereadeniemeyer: hey! totally missed you, don't expect people on IRC at this time :p08:22
fwereadenice w/e?08:23
fwereadeand hey kim0 ;)08:23
kim0fwereade: Hey Morning o/ :)08:23
raphinkI think one should expect people on IRC at about any time08:35
raphinkgiven there's people who live all around the globe (and even have internet access)08:36
fwereadeI have a recollection that "fix released" currently means "merged into trunk" rather than actually "released" -- can anyone confirm?10:16
fwereadeif so I can mark a couple of bugs "fix released", and tidy up kanban a little10:17
kim0raphink: yeah pretty much :) 10:25
noodles775Hi! Does anyone know if m_3 's psql branch is near ready for inclusion in principia ? https://bugs.launchpad.net/principia/+bug/80384110:40
_mup_Bug #803841: Formula needed (postgresql) <new-formula> <Ensemble Formulas:In Progress by mark-mims> < https://launchpad.net/bugs/803841 >10:40
kim0SpamapS: oh nice work on the graphic at http://askubuntu.com/questions/55179/what-is-the-purpose-of-the-bootstrapping-instance-ensemble :)11:15
kim0wonder if that should be added to the bootstrapping section of the user-tutorial11:15
hazmatg'morning11:56
fwereadehazmat: heyhey11:58
hazmatfwereade, how's your day so far?12:04
* hazmat tries to figure out if there's a way to see what the service-config settings are for a service12:04
fwereadehazmat: ah, not too bad... I think I may have actually just run out of work for the moment, I was about to take a look at some reviews12:05
fwereadeonce I get confirmation from andres that cobbler-launch-machine works for him I'll have anothe 3 branches to propose12:05
hazmatfwereade, nice re cobbler, that's awesome12:06
fwereadeand you; nice weekend?12:06
fwereadehazmat: there are a few more things we need to think about but I've been concentrating on getting parity with the spike branch12:06
hazmatfwereade, quiet weekend, don't remember much of it ;-) spent some time cutting a computer case with with a rotary blade so it could fit some 5x3 drive enclosures, also picked up a 4g hotspot device till my new internet connection is setup (my old isp sucked and 'accidentally' dropped me when i made an unrelated request).12:08
hazmatfwereade, do you have a local cobbler instance for testing or is it based on extraction from the spike w/ mocking?12:08
fwereademy local cobbler is, er, partially working12:09
fwereademost stuff I can test directly, but actually completing a netboot install of oneiric has had a long series of hilarious and bizarre problems12:10
fwereadeI might grid my lions and have another go shortly but it's a dispiriting prospect ;)12:10
fwereadebtw, hazmat, I was going to take a look at lp:~hazmat/ensemble/security-groups, but I'm a bit confused about its status12:27
fwereadeit claims to depend on lp:~hazmat/ensemble/security-connection... which lp thinks exists, but bzr doesn't12:29
hazmatfwereade, it depends on lp:~hazmat/ensemble/security-connection-redux12:54
hazmatfwereade, i'm not sure what happened to security-connection branch, but lp doesn't like it.. the revisions are the same though12:55
fwereadehazmat: heh, ok, thanks :)12:58
* hazmat pokes at fwereade fds testing work13:11
* fwereade wishes to make clear he has no idea what he's doing, and really appreciates fixes and explanations ;)13:11
hazmatfwereade, i'm curious to see if we manually run the gc how the numbers change13:12
fwereadehazmat: good idea13:12
hazmatfwereade, alternatively to scanning all fds, it might have been easier to just scan the /proc/pid/fd13:13
hazmatoh.. that's what's its doing13:14
hazmat'self'13:15
fwereadehazmat: that's what I do, is there some code lying around still doing things the dumb way?13:15
fwereadehey, I don't think I fixed ... er ... the branch it was originally flagged on, whatever it was13:15
hazmatfwereade, yeah my mistake, looks fine, i hadn't seen the 'self' syntax before13:16
fwereade(and it turns out I did fix lp:~fwereade/ensemble/storage-file-objects :))13:16
fwereadehazmat: cool :)13:16
hazmatone other issue, if  a test fails and had temp files/dirs they stick around post test run13:17
hazmatnot related to the branch, just a general test issue13:17
fwereadehazmat: I view that as a feature, myself: when a test fails I really like to be able to look at the problematic temp files ;)13:19
hazmatfwereade, so this is actually pointing out alot of failures in things that are okay13:21
hazmatfwereade, the addCleanup calls from lib/mocker.py run after teardown13:22
hazmatwhich take care of self.makeDir, self.makeFile afaicr13:22
* hazmat double checks13:22
fwereadehazmat: gaah! nice catch :D13:22
hazmatmocker isn't deferred aware so the ordering is a little odd at times13:22
fwereadehazmat: I assumed teardown happened in tearDown, but you know what they say about assumptions :)13:23
fwereade(they make an ass of u and mptions... hmm, doesn't quite work like that)13:23
hazmatfwereade, i made a one line change to fix it against spurious tests, in the modified _run, if methodName=="tearDown": self.addCleanup(self._diff_fds)13:26
hazmatspurious failures that is13:26
fwereadehazmat: awesome, how much nicer does it come out?13:26
hazmatfwereade, not catching normal operations, reduces the problem almost entirely afaics13:27
fwereadehazmat: and I guess we're pretty safe from having further cleanups added at that point13:27
hazmatfwereade, doing a full run now13:27
fwereadehazmat, brb13:28
fwereadeb13:32
m_3noodles775: currently the basic pg formula works.  authentication is wide open and there's no replication.13:36
hazmathmm, trial has an addCleanup that masks mocker's addCleanup 13:37
noodles775k, thanks m_3. 13:37
m_3noodles775: I should get the acls working before inclusion into principia13:37
RoAkSoAxfwereade: howdy!!13:38
fwereadeRoAkSoAx: heyhey!13:38
fwereadeRoAkSoAx: nice weekend?13:38
noodles775m_3: Yep, sounds sane :) I've subscribed to the bug so I'll know when I can try it. Thanks mor working on it!13:38
RoAkSoAxfwereade: yeah!! busy though!! how was yours?13:39
fwereadeRoAkSoAx: nice, very peaceful, cath and laura were still in the UK for most of it13:40
m_3noodles775: awesome, I'll keep it updated13:41
RoAkSoAxfwereade: cool ;)13:41
RoAkSoAxfwereade: anywas, what's your latest branch?13:41
fwereadeRoAkSoAx: hmm :)13:42
fwereadeRoAkSoAx: shadow-trunk is not yet updated with either lp:~fwereade/ensemble/cobbler-kill-machine or lp:~fwereade/ensemble/bootstrap-verify-storage13:43
fwereadebut maybe best to skip bootstrap-verify-storage for now: I think it's good, but it could interact with launch-machine (which is merged)13:43
fwereadeand I'd like to be sure of cobbler-launch-machine's status13:44
RoAkSoAxfwereade: ok13:46
RoAkSoAxfwereade: i'll work on top of shadow-trunk first then13:47
fwereadeRoAkSoAx: awesome13:47
fwereadeRoAkSoAx: if it looks healthy, I'll merge in 2 more and make 3 merge proposals13:47
RoAkSoAxfwereade: awesome!13:50
fwereadeRoAkSoAx: it actually works?13:50
fwereadehazmat: I'm sorry, but I'm really confused about your security-* pipeline: would you please tell me exactly what order I should be looking at them in? I'm driving myself slightly insane13:52
RoAkSoAxfwereade: havent started yet13:53
RoAkSoAxwill let you know as soon as I do13:53
RoAkSoAxfwereade: im updating all the cobbler-side pieces first13:53
fwereadehazmat: don't worry, I think I've figured it out14:03
niemeyerGood morning!14:36
niemeyerHmm.. funny to say that twice in the same day14:37
niemeyerAram, Aram2: Morning to both of you14:38
Aram2hi :-).14:38
Aram2I'm having some issues running the tests. http://pastebin.com/GPkiK7uZ14:38
* niemeyer looks14:39
Aram2I assume it doesn't parse ~/.ensemble/environments.yaml ?14:39
niemeyerAram2: Hmmm14:40
niemeyerAram2: That's strange.. have you touched any code?14:40
Aram2no.14:40
Aram2latest rev from bzr.14:40
niemeyerAram2: Let me try to run this test14:40
niemeyerAram2: In theory, this test should be mocked14:40
niemeyerAram2: Sorry, I actually mean that this part of the logic should be mocked14:40
Aram2I see.14:41
niemeyerAram2: Test passes here14:41
niemeyerHmm14:41
* niemeyer reads the test14:41
Aram2how can I run only this tests and not all?14:42
niemeyerAram2: Just take that last string at the end of your paste and put it after ./test14:42
Aram2also, I am using bzr branch lp:ensemble , I assume this is the branch I am interested about.14:43
Aram2ah, ok.14:43
niemeyerAram2: It is indeed14:43
Aram2yeah, test fails.14:43
Aram2ensemble works though.14:43
niemeyerAram2: Phew.. that's actually good14:44
niemeyerAram2: If it passed it'd mean the failure would come from the interaction between tests, and would be harder to debug :-)14:44
niemeyerAram2: Hah14:46
niemeyerAram2: This test is actually broken, nevermind14:47
Aram2heh.14:47
niemeyerAram2: It's just that pretty much all of us have the environment set up14:47
niemeyerAram2: The real env, I mean14:47
niemeyerAram2: Not ~/.ensemble14:47
Aram2aha.14:48
niemeyerAram2: Many EC2 tools depend on the AWS_ACCESS_KEY_ID variable14:48
niemeyerAram2: and it's partner (AWS_SECRET_ACCESS_KEY)14:48
niemeyerAram2: Our tests should not depend on it to run, though14:48
Aram2yeah...14:49
niemeyerAram2: For the moment, you can just disable this test14:49
_mup_Bug #819329 was filed: Tests depend on AWS_ACCESS_KEY_ID being set <Ensemble:Confirmed> < https://launchpad.net/bugs/819329 >14:50
hazmatniemeyer, good afternoon ;-)14:51
niemeyerhazmat: Hey!14:52
* hazmat hands off review queue crown to fwereade 14:54
* fwereade peers at it, and tries to put it on his foot14:55
niemeyer:-)14:58
jcastrorobbiew: heya do you have those 5 reasons narrowed down? I need them for a slide14:58
robbiewyeah...but they aren't "blessed" yet14:58
robbiewneed to run them by sabdfl with jono14:59
* jcastro nods14:59
robbiewthey are in the Messaging whatever14:59
jcastrook14:59
robbiewone of the bazillion ensemble google docs14:59
niemeyer:)15:02
niemeyerAram2: How's it going overall, btw?  Have you started doing any coding, or still at the understanding phase?15:03
Aram2Aram2: I have been busy a little bit... I have played with ensemble, read he current code and now I'm trying to make Go and Python work together nicely in some way.15:04
Aram2it would be nice if you could call one from the other but unfortunately you can'15:04
Aram2t :-).15:04
niemeyerAram2: Right, agreed15:05
niemeyerAram2: Note that for this experiment, you don't really have to make them play together15:05
niemeyerAram2: The question is more how it'd look like if some pieces were ported over15:05
Aram2yeah.15:06
_mup_ensemble/states-with-principals r302 committed by kapil.thangavelu@canonical.com15:13
_mup_use constant for otp identity key in domain state dict, machine state tracks otp data reference.15:13
fwereadeniemeyer, hazmat: security-otp-principal15:36
fwereadeintended only to narrow the window of opportunity for a potential attacker, rather than anything stronger?15:36
hazmatfwereade, i'm in progress on a reply on the mp, the nutshell is your assessment is correct, there is a window till the otp is consumed that an attacker can gain access to the persistent principal credentials if they can get ahold of the otp prinicipal/data enroute.15:37
fwereadehazmat: thanks15:38
fwereadeis there any way to review "approve so long as you file a bug about this shortcoming when you merge it"? :p15:39
niemeyerIt would be mitigated by encryption, though15:39
fwereadeniemeyer: sorry, what's encrypted?15:39
niemeyerfwereade: Nothing..15:40
niemeyerfwereade: It would :)15:40
fwereadeniemeyer: ah :)15:40
niemeyerFor now this is already good progress on top of what we do now 15:40
hazmatfwereade, atm i don't see a good way to manage that risk alternatively without a dedicated security agent.. we effectively create the structure with acls referencing the intended principal credentials within the cli which need to get past through at least one other process (which creates the intended connected client/agent) before they get to their destination15:40
fwereadeniemeyer: absolutely, I don't want to reject it, just to make sure that we track the fact that it's a potential issue15:40
niemeyerfwereade: I know, I'm just explaining as well, rather than attempting to change your feeling about it :)15:41
hazmatniemeyer, encryption of the otp data? the decrypt key needs to be passed as well15:41
fwereadeshall I file a bug then, and we can worry about it later?15:42
hazmatniemeyer, fwereade i'm totally open to alternate ideas on this15:42
fwereadehazmat, niemeyer: I think that at some stage in the future a security agent may be a good way around this (we could implement something that really did delete all unhashed records of the otp when it was first accessed) but for now I'm happy with it15:44
niemeyerhazmat: No, encrypt of the channel used to communicate back with zk with the key15:44
niemeyerhazmat: I think what you have is good progress already15:45
hazmatniemeyer, fwereade its unclear to me if alternate solutions would work well without such an interception gap without implementing a new zk auth mechanism15:45
niemeyerhazmat: We can easily hack ZK to implement real OTPs, or other mechanisms in the future15:45
hazmatniemeyer, sounds good15:45
fwereadecool, I'll approve and file15:45
niemeyerfwereade: Agreed15:45
_mup_Bug #819379 was filed: otp mechanism is vulnerable to interception <Ensemble:New> < https://launchpad.net/bugs/819379 >15:47
hazmatfwereade, thanks for the reviews!15:48
fwereadehazmat: a pleasure :)15:48
SpamapSkim0: thats an older graphic, though updated with new understanding. :)15:53
robbiewm_3: call time?16:02
m_3robbiew: yup16:02
* hazmat enjoys googling for ensemble bugs16:06
niemeyerLunch time.. biab16:08
fwereadeneed to pop out for a little while, bbs16:14
* niemeyer pops back17:04
RoAkSoAxfwereade: so now the orchestra config should provide a storage-url?17:34
fwereadeRoAkSoAx: yes17:37
fwereadeit seemed foolish to assume it was *necessarily* on the same server as cobbler itself17:38
RoAkSoAxfwereade: what's the formatting? http://W.X.Y.Z/abc?17:38
fwereadeRoAkSoAx: yep17:38
RoAkSoAxfwereade: well kind of but given that the cobbler server was the "orchestra" server it made sense to keep it there17:38
fwereadeRoAkSoAx: fair point :)17:39
RoAkSoAxfwereade: yes cause the idea is to make orchestra automatically configure the webdav server17:41
RoAkSoAxfwereade: and have it up and running post installation and ready to serve ensemble17:41
RoAkSoAxfwereade: let's do this, if no storage-url is provided, then assume a default, which would be the <orchestra-server/webdav>17:42
niemeyerRoAkSoAx: Agreed, at least defaulting to it sounds like a good plan17:42
fwereadeRoAkSoAx: then we can trim that back :)17:43
fwereadesure, happy to add that17:43
RoAkSoAxniemeyer: indeed17:43
RoAkSoAxfwereade: yeah! that will allow us more flexibility in case someone would like to chagne the storage somewhere else17:43
fwereadeRoAkSoAx: ...but still nothassle people by asking them to configure it until they need it17:43
fwereadeperfect17:43
fwereadecheers :)17:43
RoAkSoAxfwereade: indeed17:44
dakerm_3, what's the status of the postgres formula ?17:48
fwereadeI think it's the end of my day now; RoAkSoAx, I'll probably be back on later to merge that change through the plumbing and into shadow-trunk (and everywhere else it needs to be)17:55
RoAkSoAxfwereade: btw.. AFAIK, a "name" on a cobbler system cannot be changed17:55
RoAkSoAxfwereade: cool I'll continue my testing17:55
RoAkSoAxfwereade: and the changes on setting the provider-state using the UID doesn't really seem relevant because , as far as I can see, to be able to edit a system, you need to obtain the name of the UID17:57
RoAkSoAxso at the end it seems to be the same17:57
m_3daker: in progress... eta few hours18:05
dakerok18:06
m_3daker: have machine-based acls working (pg_hba)18:07
* negronjl is away: out to lunch18:33
* niemeyer finds his way through the maze of branches creates by fwereade ;-)18:51
niemeyers/creates/created18:51
fwereadeRoAkSoAx: belatedly: I just changed a system name, to prove I could19:07
fwereadeRoAkSoAx: there's a rename button in the system list19:07
fwereadeRoAkSoAx: I think you *can't* change a name while something else is holding a reference to it though19:08
RoAkSoAxfwereade: yeah maybe19:09
RoAkSoAxfwereade: btw...19:09
RoAkSoAxfwereade: the way how you obtaining the keys for clout-init user-data is broken19:09
RoAkSoAxas it makes clout-init to fail19:09
fwereadeRoAkSoAx: bah :(19:09
fwereadeRoAkSoAx: I thought I was doing it just the same as in EC2, didn't expect *that* of all things to be a problem19:10
RoAkSoAxfwereade: http://paste.ubuntu.com/656595/19:10
fwereadegoodness me19:11
SpamapSnegronjl: does redis support master<->master replication?19:11
fwereadeI guess it's travelling a very different path to the one it takes on EC219:12
_mup_ensemble/webdav-storage-prereq r318 committed by gustavo@niemeyer.net19:16
_mup_Merged generic-state-ops.19:16
niemeyerPlease ignore this.. just a temp branch for review19:16
niemeyerfwereade: Hmm19:17
fwereadeniemeyer: Hmm?19:18
fwereade(sorry, irresistible symetry)19:18
niemeyerfwereade: Sorry, just double checking19:20
_mup_ensemble/webdav-storage-prereq r318 committed by gustavo@niemeyer.net19:21
_mup_Merged generic-state-ops.19:21
fwereadeniemeyer: I'm fretting that I've done something unforgivably dumb now :/19:21
niemeyerfwereade: No, I'm just a bit stuck on webdav-storage19:21
niemeyerfwereade: I can't find the proper review base19:21
fwereadeniemeyer: is that one of the ones with 2 parents?19:22
niemeyerfwereade: I get a 600 lines diff on it, including things I already reviewed19:22
fwereadeI seem to recall it requires cobbler-instance-ids as well19:22
niemeyerfwereade: Yes, but I'm taking that into account19:22
niemeyerfwereade: Yeah.. I'm merging both of these to form a base19:22
niemeyerThere's still more, apparently19:22
fwereadeniemeyer: hmm, not that I recall...19:23
fwereadeniemeyer: I'm being called, bbs, I'll see if I can see what I've done19:23
niemeyerfwereade: Hmmm.. part of the diff is within one of the pre-req branches, actually19:24
niemeyerfwereade: Maybe it's just the diff that is bogus..19:24
niemeyerfwereade: Let me try something else19:24
niemeyerfwereade: Alright, I think I got it19:26
niemeyerfwereade: Don't move too fast.. I'll review it quickly before the diff disappears.. ;)19:26
fwereadeniemeyer: back, shout if I can help at all19:27
niemeyerfwereade: Super, thanks19:27
niemeyerrobbiew: ping19:34
negronjlSpamapS: no.  afaik redis only supports master-slave19:46
SpamapSnegronjl: I wonder if we can take the redis-master and redis-slave formulas and just make a 'redis' formula..19:47
negronjlSpamapS:  i'll give that a shot19:47
SpamapSnegronjl: also can redis slaves be promoted to masters?19:48
SpamapSnegronjl: if thats the case (and not too complicated) then it might make sense to have them work in a ring.19:49
negronjlSpamapS:  afaik.  yes.  still reading a bit about redis19:49
robbiewjcastro: ping20:17
jcastrorobbiew: pong20:18
=== daker is now known as daker_
fwereadeok, I think that really is it for me for now :)20:28
fwereadenn all20:28
=== heckj_ is now known as heckj
hazmatniemeyer, the extra security work integrated into domain object creation (service, unit, machine) seems to have some impacts on total test run time, about 25-30% increases, for tests that heavily use the api.. the setup/teardown deltas are under 2%, its just the extra cost of manipulating the additional nodes afaics21:07
niemeyerhazmat: Hmm21:12
niemeyerhazmat: If you're comfortable the API is the right one, we can move forward with this no matter what21:13
niemeyerhazmat: and worry about optimization down the road21:13
niemeyerhazmat: We can do tricks like disabling the auth steps under controlled scenarios21:14
niemeyerhazmat: But I'd be happier to run them all the time until we're sure they are good21:14
hazmatniemeyer, yeah.. i tried seeing what i can do about optimization now, but i'm not seeing any quick gains, i've got most of the api encapsulated into a class, that i could switch toggle by test21:15
hazmatwhich might get things a bit closer21:15
hazmatniemeyer, sounds good re goodness21:15
* niemeyer steps out for watching a movie..22:43
=== heckj_ is now known as heckj
_mup_Bug #819562 was filed: ensemble.formula.tests.test_bundle fails on buildds <Ensemble:New> <ensemble (Ubuntu):New> < https://launchpad.net/bugs/819562 >23:55
_mup_Bug #819563 was filed: ensemble.formula.tests.test_bundle.BundleTest.test_executable_extraction fails on buildds <Ensemble:New> <ensemble (Ubuntu):New> < https://launchpad.net/bugs/819563 >23:57

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!