| * niemeyer => bed.. night all! | 03:13 | |
| _mup_ | ensemble/resolved-state-api r198 committed by kapil.thangavelu@canonical.com | 12:15 |
|---|---|---|
| _mup_ | refactor resolved state api to have a more 'friendly' api feel, via relation-set-resolved taking relation-name instead of internal id. | 12:15 |
| hazmat | niemeyer, on the resolved state refactor.. the relation resolved set now takes unit names, but the inverse, get still returns internal ids, but its primary consumer is the unit agent. | 12:16 |
| niemeyer | hazmat: Hmm.. that feels a bit icky | 12:17 |
| hazmat | niemeyer, on the icky front, it also means pulling in some ensemble.unit imports into state.service | 12:18 |
| niemeyer | hazmat: Ouch | 12:18 |
| niemeyer | hazmat: That's a big ickiness :) | 12:18 |
| hazmat | niemeyer, we need to dereference to properly validate | 12:18 |
| niemeyer | hazmat: How do you mean? | 12:18 |
| hazmat | niemeyer, well given a rel name by itself, doesn't tell us the workflow state of the relation, which we need to verify if we want to provide feedback to the user | 12:19 |
| hazmat | i've got the feedback aspects of the set worked out, and added some additional unit relation accessor to the unit state. | 12:20 |
| hazmat | but the notion that set/get are not inverses is slightly odd | 12:20 |
| niemeyer | hazmat: That seems to indicate a more serious problem in our abstractions | 12:21 |
| niemeyer | hazmat: ensemble.unit should be using ensemble.state | 12:21 |
| niemeyer | hazmat: Not the other way around | 12:21 |
| hazmat | niemeyer, oddly enough no circular imports at the moment, unit.workflow is self contained atm | 12:21 |
| hazmat | i had to pull in lifecycle for the unit tests though | 12:21 |
| hazmat | in general i agree state should be foundational layer | 12:22 |
| niemeyer | hazmat: Maybe it should be moved into the state then.. | 12:22 |
| hazmat | niemeyer, it has no state manipulations. | 12:22 |
| niemeyer | hazmat: There's stuff in ensemble.hooks which is fiddling with nodes directly too | 12:22 |
| niemeyer | hazmat: That's another one we have to fix | 12:23 |
| niemeyer | hazmat: Hmmm | 12:23 |
| hazmat | niemeyer, the only stuff in hooks that manipulates state is via relationcontext atm | 12:23 |
| niemeyer | hazmat: Yes | 12:24 |
| hazmat | niemeyer, i don't see anything wrong with ensemble.hooks consuming a state api | 12:24 |
| niemeyer | hazmat: Sorry, I'm on crack | 12:25 |
| niemeyer | hazmat: I was thinking of ensemble/state/hooks.py, which is already in state | 12:25 |
| hazmat | ah.. right | 12:26 |
| niemeyer | Anyway, workflow.. hmmm | 12:26 |
| * hazmat pushes his branch | 12:26 | |
| niemeyer | yield retry_change(self._client, self.zk_state_path, update_state) | 12:26 |
| hazmat | niemeyer its at lp:~hazmat/ensemble/resolved-state-api if your curious | 12:26 |
| niemeyer | data, stat = yield self._client.get(self.zk_state_path) | 12:27 |
| niemeyer | Indeed workflow should be better integrated in the state | 12:28 |
| niemeyer | hazmat: Checking it out | 12:28 |
| niemeyer | hazmat: Yeah, I think we should talk about some cleaning up there indeed | 12:43 |
| niemeyer | hazmat: The issues we're seeing in this branch are not a problem in the branch itself | 12:43 |
| hazmat | niemeyer, i'm concerned as well about meeting the deadlines for the features entailed by the branch | 12:43 |
| niemeyer | hazmat: It's just a side effect of some pending polishing after some organic growth which took place | 12:44 |
| hazmat | given some refactoring work to fix things up | 12:44 |
| hazmat | niemeyer, yeah.. the testing situation in test_service is also need a fix up | 12:44 |
| niemeyer | hazmat: Yeah, maybe we should just merge your previous version with this specific aspect uncovered, to get the feature itself in, and then polish a bit the organization | 12:45 |
| hazmat | the relation endpoint stuff never cleaned up some of the tests, so there are like half-dozen ways to set thigns up | 12:45 |
| niemeyer | hazmat: For instance, it's bizarre that is_relation_running is a method | 12:45 |
| hazmat | a function you mean and not a method? | 12:45 |
| niemeyer | hazmat: We have a RelationStateManager, and a RelationState | 12:45 |
| niemeyer | hazmat: Sorry, yeah | 12:45 |
| hazmat | niemeyer, so we should have time at uds then to discuss some of these issues | 12:47 |
| niemeyer | hazmat: I'm not sure there's a lot to debate there.. we just have to integrate that aspect better into the state | 12:48 |
| niemeyer | hazmat: A UnitState should be aware of its workflow state.. a RelationState should be aware of its workflow state | 12:48 |
| niemeyer | hazmat: The workflow stuff itself seems nice on itself.. it's just too isolated from the rest | 12:49 |
| niemeyer | hazmat: The question is: how to merge these concepts nicely? | 12:49 |
| hazmat | niemeyer, the workflow just needs a get/set api on the state to push the workflow for zk persistence. There is an analogous api for unit relation settings data (get/set on the unit relation) although in practice that's only used by tests | 12:52 |
| niemeyer | hazmat: I think it needs a bit more than that.. e.g. "is running" is something we should be able to inquire in a UnitState | 12:53 |
| niemeyer | hazmat: Otherwise it becomes txzookeeper2 | 12:54 |
| hazmat | niemeyer, in that case we need to fold workflow aspects into state | 12:54 |
| hazmat | niemeyer, so i need a path forward, i can revert back and tackle the rest review minus this comment or i can push it forward (the set change is largely done). | 13:32 |
| hazmat | if revert, i'll file an issue regarding this for future | 13:32 |
| niemeyer | hazmat: The former sounds like a better plan | 13:33 |
| hazmat | niemeyer, okay.. sounds good | 13:33 |
| niemeyer | hazmat: If we're compromising somewhere, the id issue sounds relatively minor compared to the other inconsistencies being introduced | 13:33 |
| niemeyer | hazmat: It'd be good to sort the overall problem soonish, though | 13:34 |
| niemeyer | hazmat: This will only get worse | 13:34 |
| hazmat | niemeyer, agreed re id minor to the others | 13:35 |
| hazmat | niemeyer, service.py/test_service.py are also getting a bit large.. i think the notion of encapsulating the get/set/clear/watch api into some of staterequestprotocol will help clean that up | 13:36 |
| niemeyer | hazmat: I think we already have a good part of it encapsulated into YAMLState | 13:36 |
| niemeyer | hazmat: This is effectively read/write/delete/watch | 13:37 |
| niemeyer | hazmat: read/write are ready | 13:37 |
| _mup_ | Bug #772331 was filed: Incorporate workflow more closely into state <Ensemble:New> < https://launchpad.net/bugs/772331 > | 13:37 |
| _mup_ | Bug #772332 was filed: Use a StateRequestProtocol utilizing yamlstate encapsulate get/set/del/watch apis in unit state <Ensemble:New> < https://launchpad.net/bugs/772332 > | 13:39 |
| hazmat | niemeyer, agreed, i just want to take a next step that will allow us to move the state based request/response protocols into their own classes. and expose via composition on the unit | 13:40 |
| niemeyer | hazmat: YAMLState is already its own class | 13:41 |
| hazmat | niemeyer, yamlstate is a good start and would be used by a protocol impl, but the usage scenarios are different, yaml state represents a concurrent r/w with dict merge, the protocol implementations are geared more towards request/response.. internally i'm sure they could utilize a yamlstate, but they have additional notions of the api/validation/transformation they want to expose | 13:43 |
| niemeyer | hazmat: Indeed, and that's what the methods in the respective state classes do | 13:44 |
| niemeyer | hazmat: Note that the purpose of the state layer is precisely to give semantic meaning to what otherwise is just storage | 13:44 |
| hazmat | niemeyer, sure, and the protocols implemenation would be in the state package, part of my reasoning a concern with some of the practical parts of how big the unit state class and its tests are getting. | 13:45 |
| niemeyer | hazmat: Yeah, I understand, and I agree there are improvements to be made.. just trying to provide some feedback on them | 13:45 |
| niemeyer | hazmat: We shouldn't end up with UnitState.resolved.set(data), UnitState.resolved.get() => data | 13:46 |
| _mup_ | ensemble/resolved-state-api r199 committed by kapil.thangavelu@canonical.com | 14:43 |
| _mup_ | revert the last commit, removing relation name args to set relation resolved | 14:43 |
| _mup_ | ensemble/resolved-state-api r200 committed by kapil.thangavelu@canonical.com | 14:55 |
| _mup_ | remove extraneous assertidentical, simplify dict merge, use equality state matching for is_x_running methods. | 14:55 |
| === niemeyer is now known as niemeyer_lunch | ||
| === niemeyer_lunch is now known as niemeyer | ||
| _mup_ | ensemble/resolved-state-api r201 committed by kapil.thangavelu@canonical.com | 17:43 |
| _mup_ | use constant instead of boolean to make the usage clearer for retry_hooks param to unit/relation resolved. | 17:43 |
| _mup_ | ensemble/ensemble-resolved r253 committed by kapil.thangavelu@canonical.com | 17:54 |
| _mup_ | use new retry hook constants | 17:54 |
| _mup_ | ensemble/expose-provisioning r215 committed by jim.baker@canonical.com | 17:56 |
| _mup_ | Collect together requested opened ports by machine for eventual provisioning changes | 17:56 |
| _mup_ | ensemble/ensemble-resolved r254 committed by kapil.thangavelu@canonical.com | 18:09 |
| _mup_ | address syntatic review comments | 18:09 |
| _mup_ | ensemble/refactor-to-yamlstate r199 committed by bcsaller@gmail.com | 18:52 |
| _mup_ | Changes per review | 18:52 |
| _mup_ | Dict copy returned again rather than YAMLState | 18:52 |
| _mup_ | Re-enabled tests of dict copies. | 18:52 |
| _mup_ | Fix remaining set_data as string calls (converted | 18:52 |
| _mup_ | to dict) | 18:52 |
| _mup_ | ensemble/refactor-to-yamlstate r200 committed by bcsaller@gmail.com | 18:55 |
| _mup_ | merge trunk | 18:55 |
| niemeyer | bcsaller: re. | 18:58 |
| niemeyer | """ | 18:58 |
| niemeyer | I allow YAMLState.read(required=True) which will throw an | 18:58 |
| niemeyer | StateNotFound exception which the higher level API can then translate | 18:58 |
| niemeyer | """ | 18:58 |
| niemeyer | bcsaller: The point was specifically about the way in which the code is organized | 18:58 |
| niemeyer | bcsaller: Not about the outside interface | 18:59 |
| niemeyer | bcsaller: test + get leaves a race condition at the + | 18:59 |
| niemeyer | bcsaller: The suggestion was to just do try: get except: do whatever | 18:59 |
| bcsaller | gustavo: YAMLState needed to use the product of that get though and didn't currently take its state via __init__ | 19:00 |
| niemeyer | bcsaller: I don't understand.. just do it, and catch an error in case it fails? | 19:00 |
| niemeyer | bcsaller: Every try/except involves using the product of the get | 19:00 |
| bcsaller | I understood it as "check existence or throw an error" to "get state and use it if possible or throw an error". I made it so yamlstate can get the state and use it | 19:01 |
| bcsaller | the get outside of yamlstate didn't avoid the race if I didn't use that data | 19:02 |
| bcsaller | and yamlstate was set up to use the data after read | 19:02 |
| niemeyer | bcsaller: | 19:03 |
| niemeyer | try: | 19:03 |
| niemeyer | state = YAMLState(client, path) | 19:03 |
| niemeyer | except zookeeper.NoNodeException: | 19:03 |
| niemeyer | # not found | 19:03 |
| niemeyer | bcsaller: ? | 19:03 |
| bcsaller | YAMLState.__init__ isn't async and doesn't check the node | 19:03 |
| bcsaller | nor does read normally require it, but there were cases where the existing apis sometimes expect it or check for a specific exception | 19:04 |
| bcsaller | in those cases this lets read raise an error if the node didn't exist | 19:04 |
| niemeyer | bcsaller: Ok, sorry.. | 19:04 |
| bcsaller | which for YAMLState isn't normally an issue as it will try to create it on write | 19:04 |
| niemeyer | state = YAMLState() | 19:04 |
| niemeyer | try: | 19:04 |
| niemeyer | state.read() | 19:04 |
| niemeyer | yield state.read() # actually | 19:05 |
| niemeyer | except zookeeper.NoNodeException: ... | 19:05 |
| niemeyer | ? | 19:05 |
| bcsaller | except read didn't throw the error | 19:05 |
| bcsaller | I made that behavior optional | 19:05 |
| bcsaller | write would just try to create/merge to the path when it was done | 19:05 |
| niemeyer | bcsaller: Ok, but the equivalent.. | 19:05 |
| bcsaller | so the exception was eaten | 19:05 |
| niemeyer | bcsaller: Hmmm | 19:06 |
| niemeyer | bcsaller: I see what you mean, sorry | 19:06 |
| niemeyer | bcsaller: I misunderstood the comment | 19:06 |
| niemeyer | bcsaller: Sounds fine | 19:07 |
| bcsaller | there is a question if that type of test is needed at all now though, if the behavior is to create the node in write | 19:07 |
| bcsaller | gustavo: maybe I can make the docstring clearer? if you have a suggestion | 19:07 |
| niemeyer | bcsaller: Maybe it's not.. the problem was just the race | 19:07 |
| niemeyer | bcsaller: I haven't seen the code, which is perhaps the issue | 19:07 |
| niemeyer | bcsaller: (changes were not pushed) | 19:08 |
| bcsaller | gustavo: pushed | 19:08 |
| bcsaller | includes a trunk merge, I was rechecking it for any errors | 19:09 |
| niemeyer | bcsaller: pulling | 19:09 |
| niemeyer | bcsaller: Yeah, it's fine as it is, sorry for jumping to conclusions | 19:10 |
| bcsaller | np | 19:10 |
| bcsaller | keeping me honest ;) | 19:11 |
| niemeyer | bcsaller: You've mentioned changes in YAMLState and it felt like the actual race wasn't addressed, which clearly is not the case | 19:11 |
| niemeyer | bcsaller: So, removing the test.. hmmm | 19:11 |
| niemeyer | bcsaller: Do you recall where the relation data is first initialized? | 19:11 |
| bcsaller | you mean hook._setup_relation_data or state.relation._add_relation_state? | 19:12 |
| bcsaller | (I didn't specify that dotpath very well there, left out the class, but I think you know what I mean) | 19:14 |
| * bcsaller fetches coffee | 19:14 | |
| niemeyer | bcsaller: No, I mean the relation state itself | 19:14 |
| niemeyer | bcsaller: We're assuming it exists | 19:14 |
| niemeyer | bcsaller: So someone else must have created it | 19:14 |
| niemeyer | bcsaller: We can't change that logic without understanding that | 19:15 |
| niemeyer | bcsaller: Btw, it looks like none of the changes in protocol.py are required | 19:15 |
| niemeyer | bcsaller: It's probably just left over from the previous version | 19:16 |
| niemeyer | bcsaller: and the changes to YAMLState are also pending tests | 19:17 |
| bcsaller | I did add one test for the required flag raising an exception | 19:17 |
| * bcsaller checks it over again | 19:18 | |
| niemeyer | bcsaller: Heh, yeah I'm on crack again | 19:20 |
| _mup_ | ensemble/expose-provisioning r216 committed by jim.baker@canonical.com | 19:21 |
| _mup_ | Remove requested opened ports upon a service no longer being exposed | 19:21 |
| niemeyer | Is it just me or the mouse can't click in Natty every once in a while? | 19:22 |
| jimbaker | niemeyer, i have seen something like this - just gets briefly frozen up | 19:23 |
| jimbaker | in terms of user input | 19:23 |
| niemeyer | Yeah | 19:23 |
| jimbaker | timely event loop processing, it's an art ;) | 19:24 |
| niemeyer | Not just briefly actually | 19:24 |
| niemeyer | Can barely use the trackpad | 19:24 |
| jimbaker | for me i have seen pauses up to maybe 15s, especially when i have initially started. pretty rare however. everything else iirc seems to work in terms of normal refreshing | 19:26 |
| jimbaker | just the input processing | 19:26 |
| niemeyer | Is there a reset-mouse? :) | 19:28 |
| _mup_ | ensemble/unit-agent-resolved r256 committed by kapil.thangavelu@canonical.com | 19:28 |
| _mup_ | pass transition variables to action. | 19:28 |
| niemeyer | Switching to the terminal usually works, but not this time | 19:28 |
| hazmat` | niemeyer, restarting unity from a vterm didn't help? | 19:29 |
| niemeyer | No, generally kills the whole thing | 19:30 |
| hazmat` | niemeyer, its worked for me pretty consistently (vterm, kill compiz & gnome-screensaver, unity --replace, back to desktop) | 19:30 |
| jimbaker | hazmat`, that definitely helped with the true freezes. need to attach that on a sticky note to my laptop ;) | 19:31 |
| niemeyer | hazmat`: Yeah, that seems to have worked | 19:31 |
| hazmat` | multi az/multi region mongodb replica set http://blog.mongodb.org/post/4982676520/mongodb-on-ec2-best-practices | 19:51 |
| hazmat` | trunk has surpassed 1k tests, jimbaker you get a prize ;-) | 19:58 |
| === andreas__ is now known as ahasenack | ||
| hazmat` | bcsaller, fwiw, i didn't get any errors on current trunk | 20:11 |
| jimbaker | bcsaller, thanks for fixing refactor-to-yamlstate, that's working for me now | 20:24 |
| bcsaller | jimbaker: woot | 20:24 |
| _mup_ | ensemble/unit-agent-resolved r257 committed by kapil.thangavelu@canonical.com | 21:19 |
| _mup_ | fold fire_hooks into transition-variables | 21:19 |
| _mup_ | ensemble/unit-agent-resolved r258 committed by kapil.thangavelu@canonical.com | 21:23 |
| _mup_ | unit workflow retry actions take an optional flag for fire hooks, workflow definition specifies alias for retry transitions. | 21:23 |
| _mup_ | ensemble/unit-agent-resolved r259 committed by kapil.thangavelu@canonical.com | 21:34 |
| _mup_ | lifecycle methods sans hook tests | 21:34 |
| _mup_ | ensemble/refactor-to-yamlstate r201 committed by bcsaller@gmail.com | 21:35 |
| _mup_ | revert minor changes and unused import | 21:35 |
| hazmat` | niemeyer, got a moment? | 21:51 |
| _mup_ | ensemble/unit-agent-resolved r260 committed by kapil.thangavelu@canonical.com | 22:08 |
| _mup_ | allow unit resolved watch to stop watching. | 22:08 |
| _mup_ | ensemble/unit-agent-resolved r261 committed by kapil.thangavelu@canonical.com | 22:09 |
| _mup_ | allow unit resolved watch to stop watching. | 22:09 |
| _mup_ | ensemble/unit-agent-resolved r261 committed by kapil.thangavelu@canonical.com | 22:16 |
| _mup_ | allow unit relation resolved watch to stop watching. | 22:16 |
| _mup_ | ensemble/trunk r214 committed by bcsaller@gmail.com | 22:18 |
| _mup_ | Merge refactor to YAMLState. [r=niemeyer] [f=770448] | 22:18 |
| _mup_ | This branch refactors relation settings to use YAMLState internally. | 22:18 |
| _mup_ | Changes relation_set commands to take a dict rather than a YAML serialized dict . | 22:18 |
| niemeyer | hazmat`: Sorry, missed your question, and about to step out for a break with Ale | 22:20 |
| niemeyer | hazmat`: I'll be back in 1h or so, though | 22:20 |
| niemeyer | hazmat`: Can we talk then? | 22:21 |
| === niemeyer is now known as niemeyer_bbl | ||
| _mup_ | ensemble/expose-provisioning r217 committed by jim.baker@canonical.com | 22:59 |
| _mup_ | Minimal support for eventual consistency in provider firewalls | 22:59 |
| hazmat` | niemeyer_bbl, tomorrow | 23:00 |
| hazmat` | i'm out to a hadoop meetup | 23:00 |
| jimbaker | hazmat`, have fun | 23:03 |
| hazmat` | jimbaker, thanks | 23:03 |
| jimbaker | reminds me, i promised to give a talk on ensemble + zk for our local hadoop/big data meetup in june | 23:04 |
| niemeyer_bbl | hazmat`: Sounds good, enjoy! | 23:21 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!