/srv/irclogs.ubuntu.com/2011/08/18/#ubuntu-ensemble.txt

niemeyerhazmat: http://paste.ubuntu.com/668723/00:00
jimbakerm_3, but my initial feeling is  that the current model has an elegant simplicity to it00:00
niemeyerDinner00:01
jimbakerniemeyer, enjoy what must be a late dinner!00:01
m_3jimbaker: understand... it might be a design decision to keep the states/events at a certain level00:03
=== otubo[AFK] is now known as otubo
niemeyerhazmat: ping02:00
hazmatniemeyer, pong03:42
niemeyerhazmat: Hey man03:42
niemeyerhazmat: Just finishing some docs for gozk03:42
hazmatniemeyer, hola.. long dinner with an out of town friend03:42
niemeyerhazmat: It's in a pretty good shape I think03:42
niemeyerhazmat: Nic03:42
niemeyere03:42
hazmatniemeyer, awesome.. 03:43
* hazmat checks out the paste03:43
niemeyerhazmat: I have a better test, actually..03:43
* niemeyer pastes03:43
niemeyerhazmat: http://paste.ubuntu.com/668835/03:44
hazmatniemeyer, interesting03:45
hazmatlooks so you did the diversion of session events off watches, nice03:46
niemeyerhazmat: Yeah, and there's a non-obvious handy aspect there.. I'm just finishing the docs and will paste it03:47
hazmatniemeyer, one caveat, perhaps obvious, if the reinit is used to restablish sessions the extant watches against the session are somewhat hosed because the zk c bindings are tracking watches against the handle, and on reinit the handle is hosed though the session restored03:47
hazmats/handle is hosed/handle is new03:47
niemeyerhazmat: Not sure I get how that's an issue03:48
jimbakernice simple test03:48
hazmatniemeyer, its not an issue, just something for a document03:48
hazmatniemeyer, you where saying there's a handy non-obvious property there?03:48
niemeyerhazmat: But why does it even have to be in the doc?  It feels like an internal implementation detail03:48
niemeyerhazmat: Yeah.. events stringify properly through String()03:49
niemeyerhazmat: and os.Error is actually a interface { String() string }03:49
niemeyerhazmat: Which means it becomes comfortable to handle problematic situations03:49
niemeyerhazmat: if !event.Ok { return event }03:50
hazmatniemeyer, well reconnect/reinit a session, with watches associated to a session, and old extant watches effectively being dead is always obvious.03:50
niemeyerhazmat: The event itself is the error03:50
hazmatisn't that is03:50
niemeyerhazmat: That's exactly what the test I pasted does.. old watches do not die unless the session expires03:50
niemeyerhazmat: If people get an expired session, the watch will fire03:51
hazmatniemeyer, in that cases its the same session and same handle03:51
hazmatif you ReInit with an open watch.. things are different, even though its the same session03:51
niemeyerhazmat: Sure.. if people obtain a new zk value, it's a new zk value..03:52
hazmatniemeyer, very cool though.. good to have this 03:52
hazmatniemeyer, its a c library implementation detail/deficiency, just the association of callbacks to the handle, the watch callbacks should be session associated across reinit boundaries imo03:53
niemeyerhazmat: I don't feel too strong about this..03:54
niemeyerhazmat: ReInit is a bizarre detail03:54
niemeyerhazmat: In practice, zk does the heavy lifting internally03:55
hazmatniemeyer, this looks pretty good03:56
hazmatniemeyer, on a separate, unrelated note, i found a gevent / twisted integration, basically a new reactor for twisted that uses gevent and automatically associates twisted protocols to i/o greenlets.. allows intermixing the gevent and twisted style code.. http://wiki.inportb.com/wiki/Projects:Python:Geventreactor03:57
niemeyerhazmat: Thanks, really glad you like it.. I hope we can sort out some of our issues03:58
niemeyerhazmat: http://paste.ubuntu.com/668840/03:58
niemeyerhazmat: This is part of the docs03:58
niemeyerhazmat: Holy crap03:58
niemeyerhazmat: This is sick :)03:58
hazmatsuper awesome :-)03:59
hazmatniemeyer, the implementation is pretty clean, i haven't run it against anything yet, i'm curious how well it will handle inlinecallbacks.. but definitely interesting04:00
hazmatniemeyer, so re docs.. look good.. one point of concern, is that there are numerous session channel events which are effectively transient which mean nothing to the app, it might be worth noting that, there really there to allow apps to respond to connectivity changes for self-throttling.. but on a size limited buffer their effectively no-ops to most apps.04:02
niemeyerhazmat: This takes two second class citizens in the Python world and put them together.. if you want to create a bomb, this is pretty effective.04:03
hazmati guess its captured with the panic if not read from session channel event, but it doesn't really distinguish that most of these events are frivolous to most apps04:03
hazmatniemeyer, lol04:03
hazmatniemeyer, if we pool our super powers.. we get cows :-)04:03
niemeyerhazmat: That milk radioactive acid04:03
hazmatniemeyer, i guess it does, its just implicit in that non important things aren't delivered to watchers, which implies they are delivered to the session channel04:04
niemeyerhazmat: Good point re. docs, will add04:05
_mup_ensemble/expose-cleanup r319 committed by jim.baker@canonical.com04:05
_mup_Mock completely current tests against shutdown04:05
niemeyerhazmat: Indeed04:05
hazmatotoh.. twisted has pluggable reactors.. this is just another pluggable reactor..  a libev one.. the addition of greenlets to it.. is definitely radioactive, but intriguing04:06
niemeyerhazmat: It's intriguing for sure, but I'm not interested on it in the way you are.. We really can't even think of using something like this for Ensemble04:07
jimbakerwhy would inlineCallbacks be impacted by the choice of reactor implementation? other than a slow reactor is going to trampoline slowly?04:07
niemeyerhazmat: We have to walk towards a platform that enables us to produce very a solid framework04:08
hazmatjimbaker, it shouldn't be.. it should just work04:08
niemeyerhazmat: This is going into the exact opposite direction04:08
jimbakerhazmat, exactly. there's no magic there04:08
niemeyerhazmat: Made docs more clear04:09
hazmatniemeyer, cool04:11
hazmati'm going to head to bed... have a good night folks04:12
niemeyerhazmat: Night man.. will just finish this and will head to bed too04:13
TREllisanyone tried playing with openstack & ensemble yet?08:11
TREllisjust getting a "Connection was refused by other side: 111: Connection refused" upon bootstrapping atm08:12
=== otubo is now known as otubo[AFK]
hazmatmorning12:34
hazmatTREllis, we found a few issues with it in one of our dependencies, txaws, its definitely a priority to have it working smoothly b4 oneiric12:35
=== otubo[AFK] is now known as otubo
TREllishazmat: ah, I saw one Clint put a patch for txaws and applied it manually to check, but I still get problems, perhaps my config is dodgy? lemmie pastebin it12:49
TREllishazmat: http://paste.ubuntu.com/669198/ my environments.yaml12:50
TREllishazmat: since I replaced the IP with a hostname and the txaws patch I get: http://paste.ubuntu.com/669200/12:51
TREllisactually, hostname or IP, doesn't make a difference12:54
hazmatTREllis, yeah.. i think that's about where we got on it12:58
hazmatTREllis, we're going to need to do some additional debugging 12:58
TREllishazmat: ok thanks, added as a comment to the bug13:00
niemeyerGood morning software lovers14:32
m_3morning!15:08
heckjmorning west coaster!15:09
kim0heh any idea how long does our cloudfoundry formula actually take to deploy15:24
niemeyerkim0: No..?15:27
hazmatkim0, there's an open bug about it.. there's some env var that the ruby needs that is unknown15:27
hazmatkim0, it will hang as i recall else.. i thought it was just rabbitmq, but i think that's been resolved15:28
hazmati haven't seen any confirmation that its working though15:28
hazmatbut the sympton was it just hanged on install15:29
kim0botchagalupe is saying it's 25mins min! wanted to know if we beat that15:29
kim0I know it's mostly packaging, not really an ensemble thing though15:30
kim025mins still sounds too much to me15:30
m_3kim0: agree, that sounds kind of strange... even if they have to entirely bootstrap a gem env15:31
niemeyerkim0,m_3: It'd be nice to have more details about what's going on there15:32
niemeyerIt's certainly quite unexpected.. there's no reason for it to take 25 mins15:32
m_3kim0: you have links to the latest or should we wait for negronjl 15:32
kim0m_3: haven't looked, but we could dig it up from negronjl's 15:33
kim0m_3: it seems to have a bug now though15:33
m_3kim0: ok, I'll schedule some time to branch and play with it this afternoon15:33
kim0m_3: cool!15:33
kim0I'll love to show it to the world .. finishing is 5mins or so hopefully :)15:34
m_3kim0: it's gotten a lot of press lately15:34
kim0yeah definitely hot15:34
niemeyerkim0: Exactly.. it'd be nice to expose that15:38
kim0if the bug is fixed today sometime .. I'll screencast it first thing tomorrow morning 15:39
niemeyerkim0: That's awesome, thanks a lot15:51
_mup_ensemble/expose-cleanup r320 committed by jim.baker@canonical.com16:00
_mup_Mock out sleep16:00
hazmatfwereade, niemeyer, bcsaller, jimbaker.. g+ meeting?16:05
bcsallerlooking for invite16:05
fwereadehazmat: invite away16:06
niemeyerYep!16:06
jimbakermeeting sounds good16:07
hazmatinvites out16:09
hazmatjimbaker, you see invite16:11
hazmat?16:11
jimbakerhazmat, i don't16:11
niemeyerThat's not working..16:11
niemeyerI'll restart..16:11
jimbakeri refreshed g+ a few times (which i don't expect to do), nothing yet16:11
_mup_Bug #828885 was filed: 'relation-broken' hook not firing when relation is set to 'error' state <Ensemble:New> < https://launchpad.net/bugs/828885 >16:20
niemeyerif !event.Ok { return event }16:21
niemeyerc.Assert(event, Matches, "ZooKeeper connected; path created: /path")16:24
negronjljcastro: ping17:39
jcastronegronjl: howdy17:40
negronjljcastro:  When you get a minute, I'd like to catch up with you re: NoSQL17:40
negronjlis kim0 also attending ?17:40
jcastroI have time now17:40
negronjlG+ ?17:40
jcastrosure17:40
negronjlgive me a sec17:41
negronjljcastro: invite sent17:43
niemeyerbcsaller, hazmat: I'm available for our call17:47
bcsallerniemeyer, hazmat: ready when you are17:48
hazmatniemeyer, bcsaller invitations out17:50
=== otubo is now known as otubo[AFK]
jcastrolynxman: heya, can you update the ensemble in macports to be a more recent snapshot?18:17
jcastrowe're doing a talk at scale out camp next wednesday and it'd be nice to have something more up to date18:17
_mup_ensemble/expose-cleanup r321 committed by jim.baker@canonical.com18:59
_mup_Refactoring of complex mocking18:59
_mup_ensemble/expose-cleanup r322 committed by jim.baker@canonical.com19:09
_mup_Finished previous refactoring19:09
=== otubo[AFK] is now known as otubo
_mup_ensemble/expose-cleanup r323 committed by jim.baker@canonical.com19:35
_mup_Verify deletion of security group in mocking19:35
* niemeyer breaks for coffee20:44
=== otubo is now known as otubo[AFK]
_mup_ensemble/expose-cleanup r324 committed by jim.baker@canonical.com20:54
_mup_Explicit MATCH for mock params, remove debugging20:54
_mup_ensemble/expose-cleanup r325 committed by jim.baker@canonical.com21:06
_mup_PEP8 & PyFlakes21:06
_mup_ensemble/expose-cleanup r326 committed by jim.baker@canonical.com21:18
_mup_docstrings, comments21:18
_mup_ensemble/machine-agent-uses-formula-url r314 committed by kapil.thangavelu@canonical.com21:20
_mup_additional tests for local file urls if the file is not present.21:20
_mup_ensemble/machine-agent-uses-formula-url r315 committed by kapil.thangavelu@canonical.com21:33
_mup_replace mocker any with value match functions21:33
fwereadeniemeyer, hazmat: EC2LaunchMachine *theoretically* handles stuff in machine_data like image_release_name, and various other params that get passed through to get_current_ami21:38
fwereadeniemeyer, hazmat: ...but I don't see any mechanism by which they could actually be injected at the moment21:39
fwereadeniemeyer, hazmat: (ok, yes, they get put in machine_data... but nothing *does* put them in machine_data AFAICT, so it all seems to be dead code)21:39
fwereadeniemeyer, hazmat: off the top of your heads, am I missing something?21:40
niemeyerfwereade: They're probably not being used21:40
niemeyerfwereade: get_current_ami was built as an experiment, outside of ensemble21:40
fwereadeniemeyer: then I can kill them? :)21:41
fwereadeniemeyer: get_current_ami itself is actually fine, it has (I think) sensible defaults for when you don't specify a default image id in the config21:41
niemeyerfwereade: If they're working correctly, doesn't feel like a good idea..  we have missing functionality in that area, and this is a utility we can make good use of21:41
fwereadeniemeyer: get_current_ami is used elsewhere, and is useful so far as I know21:43
fwereadeniemeyer: tbh it's not really a hassle to keep them in, it'll just be an unused parameter somewhere21:43
fwereadeniemeyer: but on the other hand it wouldn't be a hassle to add them when we needed it21:44
niemeyerfwereade: That's fine.. as long as they're not causing problems and not getting on your way to implement something, I suggest keeping things as they are21:44
fwereadeniemeyer: we'd need code changes to actually make use of them now21:44
niemeyerfwereade: We need to support firing different releases, which touches that area21:44
fwereadeniemeyer: ok, I'll add an unused .image_options or something to the machine_data replacement, and make sure it works for when we get around to using it21:45
niemeyerfwereade: Ah, I see.. you _are_ in fact touching on that area, and it's getting on your way21:46
niemeyerfwereade: Sounds fine to remove it then21:46
fwereadeniemeyer: cool :)21:46
fwereadeniemeyer: I promise it won't be any harder to add in the future than it would be right now ;)21:47
hazmatfwereade, the only injection mechanism atm is via config, we don't have pass through parameters to get machine options from start machine21:49
hazmathmm.. or do we21:49
fwereadehazmat: we do, if we stick the right magic in machine_data21:49
niemeyerhazmat: Injecting machine details through config is really a hack, though21:50
fwereadehazmat: but we don't have any mechanism for actually adding them to machine_data, so it's kinda moot21:50
niemeyerhazmat: Sounds fine to kill it for the moment21:50
hazmatniemeyer, agreed most of the time folks are asking for it exposed on a per command basis, not an environment default21:51
niemeyerRight21:51
hazmatniemeyer, actually things like region are useful for determining ami and are environment settings properly21:51
niemeyerhazmat: Sure.. that should continue to exist21:52
niemeyerfwereade: ^21:52
fwereadeniemeyer, hazmat: region remains necessary, and default-image-id remains usable21:52
niemeyerfwereade: default-image-id is also a hack21:53
fwereadeall we lose is the ability to stick things like image_release_name into machine_data, which we don't do anyway, and should do in a different way when we do come to do it21:53
fwereadeniemeyer: agreed, but it's a hack that people may well be currently using21:53
niemeyerfwereade: We can support it for now, but it's dying soon21:54
fwereadeniemeyer: sounds good21:54
niemeyerfwereade: They may be using, but there are features that we're adding very soon that render it unusable21:54
hazmatniemeyer, to be replaced with?21:54
fwereadeniemeyer: but I think it deserves a targeted death, not just a drive-by21:54
niemeyerhazmat: To be replaced with release-level selection21:54
niemeyerhazmat: The way to customize a base image with Ensemble is through formulas, not through custom machine ids21:55
fwereadeniemeyer: I'd been assuming something along those lines21:55
niemeyerhazmat: People should definitely not be publishingn formulas which require others to tweak their setup to use custom images21:55
fwereadeniemeyer: so, say, mongodb will want a 64-bit arch21:55
hazmatyeah.. 64-bit image selection ended up being the primary use for default-image-id.. but it should be automatic just based on size21:56
fwereadeniemeyer: and we'll need to have an ensemble format to define these properties anyway, because we'll need to specify them differently on ec2 and on orchestra21:56
hazmatthe only ec2 oddball is tiny with 64bit21:56
fwereadeniemeyer: so the time to thread them through is when we've actually defined what can be specified21:57
niemeyerfwereade: We'll probably start introducing these bits pretty soon21:58
SpamapShrm.. seeing something very strange when trying to bootstrap on canonistack..21:58
niemeyerfwereade: This is part of the store work21:58
fwereadeniemeyer: sounds good to me21:58
SpamapSwhen doing a bootstrap w/ EC2.. it never seems to check for the existence of the s3 bucket..21:58
SpamapSbut with canonistack, it tries to find it, gets a 404, and fails21:58
fwereadeniemeyer: in that case, maybe it would be more sensible to retain the capability, just change how it's done21:59
fwereadeniemeyer: it's just one extra field/parameter after all21:59
niemeyerSpamapS: I have vague memories of S3 failing in non-obvious ways in some case, with 403.. there's a chance they're behaving differently there21:59
hazmatSpamapS, i suspect its the error parsing against the file not found22:00
niemeyerfwereade: Sounds good as well22:00
hazmatSpamapS, like we expect to find a certain value in the error denoting a file not found22:00
SpamapSniemeyer: whats confusing is that no such request is even made to ec222:00
hazmatand having looked at s3store.py i doubt its compatible on error message content22:00
SpamapSactually I can't say that for sure..22:01
SpamapSI just realized the S3 URL is HTTP, not HTTPS22:01
niemeyerSpamapS: That'd be a serious bug.. it should certainly check22:01
niemeyerSpamapS: I mean, EC2 not checking the bucket22:01
niemeyerEnsemble not checking the bucket on EC2, that is22:02
hazmatSpamapS, specifically line 73 of providers/ec2/files.py i'd stick a pdb in there to verify22:02
hazmatwe check the bucket access on s3  prior to starting any machines on ec222:03
hazmatits the error that comes back if the file doesn't exist that seems to be getting not trapped corectly into something ensemble recognizes i suspect22:03
SpamapSahh so its looking for 'NoSuchBucket' when a 404 should be sufficient22:04
niemeyerSpamapS: Man..22:07
niemeyerSpamapS: This is starting to feel like a can of worms22:07
niemeyerIt's unfortunate that these interactions are entirely different :-(22:08
SpamapSwell there is no published standard for AWS22:09
SpamapSso it takes somebody reporting that something is different for an implementation to get fixed22:09
SpamapSits worth noting that nova-objectstore is *not* recommended except for testing/single node setups.22:09
hazmatSpamapS, i doubt swift does much better  here22:10
niemeyerSpamapS: When one mimics an implementation, there's only one way to make it look the same..22:10
niemeyerSpamapS: http://paste.ubuntu.com/669591/22:10
niemeyerSpamapS: I'd expect people to do that kind of experimentation when implementing it22:10
hazmatSpamapS, looking at the swift s3 it definitely doesn't..22:11
hazmatSpamapS, both implementations do however maintain the same http error codes as s322:11
hazmatjust not the same error content responses22:11
SpamapSwhich makes sense22:11
SpamapShonestly, code > content ;)22:12
hazmatSpamapS, yeah.. i think switching this to just verifying error codes should be sufficient22:12
SpamapSit may be worth reporting to swift and nova-objectstore that they should conform to AWS22:13
SpamapSbut we probably should work w/ them before they fix that22:13
SpamapSbig question, why am I not getting a real traceback?22:15
SpamapShazmat: that failure is not coming from line 7322:15
niemeyerSpamapS: Agreed, we should try to make it work regardless22:15
SpamapSor anywhere in providers.ec2.files :-P22:15
niemeyerSpamapS: I'm just bitching22:16
SpamapSniemeyer: yeah, not much we can do unless we want to try and champion AWS compatibility as a standard ;)22:16
SpamapSanyway, have to run.. ttyl guys22:16
hazmatSpamapS, re tracebacks.. twisted tends to make that hard.. what traceback do you get?22:16
niemeyerSpamapS: Cheers22:16
hazmatSpamapS, ttyl22:17
=== otubo[AFK] is now known as otubo
lynxmanjcastro: sure, it always take a bit to update though22:32
lynxmanjcastro: as a matter of fact, the port I submitted for Ensemble is still not in the public repo :(22:33
lynxmanjcastro: http://www.macports.org/ports.php?by=name&substr=ensemble22:33
niemeyerhazmat: I'm attempting to force a session expiration without much success..22:43
niemeyerhazmat: Have you succeeded in testing this on txzk?>22:43
hazmatniemeyer, hmm.. expiration specifically..22:44
hazmatniemeyer, i could do closes easily, by connecting another client with the same session, and closing it one, so it closes the other22:44
* hazmat checks for expiration22:45
hazmatniemeyer, yeah.. that generates an expiration22:45
jcastrolynxman: oh man that sucks. :-/22:45
hazmatniemeyer, http://bazaar.launchpad.net/~ensemble/txzookeeper/trunk/view/head:/txzookeeper/tests/test_session.py#L15122:46
niemeyerhazmat: Yeah, I recall you mentioned the trick22:46
niemeyerhazmat: I was trying to check a real scenario22:46
lynxmanjcastro: yeah, they're up their elbows on work with the release of Lin :/22:47
niemeyerhazmat: Reduced the timeout, killed the server22:47
lynxmanjcastro: Lion I mean :)22:47
niemeyerhazmat: Restarted the server.. session seems reestablished fine22:47
niemeyerhazmat: It's working too well.. ;)22:47
jcastrolynxman: ok I'll say "in progress" or something22:47
hazmatniemeyer, yeah.. so timeout specified on init is not the actual session timeout22:47
hazmatniemeyer, the timeout is negotiated between client and server22:47
niemeyerhazmat: Oh, crap.. you're right.. it's even called recvTimeout22:47
niemeyerOh well.. I'll see if the trick works22:48
lynxmanjcastro: sounds cool :)22:48
_mup_ensemble/expose-cleanup r327 committed by jim.baker@canonical.com23:17
_mup_Add mocking around shutdown taking too long23:17
hazmatniemeyer, can you confirm that i didn't miss anything from our security discussion in austin.. https://pastebin.canonical.com/51455/23:23
adam_ganyone bootstrapped an oneiric AMI lately? or at least since monday?23:33
adam_g  * softwareproperties/ppa.py:23:33
adam_g    - show PPA description and confirm before adding23:33
adam_g      (security-o-catch-all spec)23:33
* hazmat tries it out23:53
hazmathmm23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!