| _mup_ | ensemble/new-hook-semantics-5-docs r201 committed by jim.baker@canonical.com | 00:23 |
|---|---|---|
| _mup_ | Addressed review points on formula doc | 00:23 |
| _mup_ | ensemble/new-hook-semantics-5-docs r202 committed by jim.baker@canonical.com | 06:03 |
| _mup_ | Better intro for formula section | 06:03 |
| _mup_ | ensemble/yaml-state-node r192 committed by bcsaller@gmail.com | 07:23 |
| _mup_ | read/write version of YAMLState | 07:23 |
| _mup_ | ensemble/yaml-state-node r193 committed by bcsaller@gmail.com | 08:59 |
| _mup_ | test multiple writes with and w/o reads | 08:59 |
| _mup_ | ensemble/config-state-manager r196 committed by bcsaller@gmail.com | 10:24 |
| _mup_ | new YAMLState api changes | 10:24 |
| _mup_ | ensemble/trunk r199 committed by kapil.thangavelu@canonical.com | 13:30 |
| _mup_ | merge state-machine-enhancements [r=niemeyer][f=755062] | 13:30 |
| _mup_ | Adds some additional capabilities to the statemachine used for workflows. | 13:30 |
| _mup_ | Transition aliases, which allowing grouping and addressing of common | 13:30 |
| _mup_ | transitions from different states. It also introduces success transition | 13:30 |
| _mup_ | ids that a transition can use to effect an additional transition upon | 13:30 |
| _mup_ | success. | 13:30 |
| niemeyer | hazmat: Morning! | 13:31 |
| TeTeT | kim0: hi, my query for the high level overview is primarily motivated by the questions and mis-conceptions I hear from the UEC students, especially when it comes to virtualization vs cloud computing | 13:31 |
| niemeyer | hazmat: Feeling better? | 13:31 |
| hazmat | niemeyer, a bit, more medications to start my day. | 13:31 |
| niemeyer | Hey, morning TeTeT, kim0 | 13:31 |
| kim0 | TeTeT: Yeah .. those believing "put your app on the cloud, and it'll scale like google" :) | 13:31 |
| kim0 | niemeyer: hazmat TeTeT morning :) o/ | 13:31 |
| TeTeT | kim0: think we should address this early on to make sure the scenario for use of ensemble is set right, e.g. expectation management from the start | 13:32 |
| TeTeT | hi niemeyer | 13:32 |
| TeTeT | morning hazmat | 13:32 |
| niemeyer | hazmat: Ouch.. Did you have that kind of issue often when you were a non-veg? | 13:32 |
| TeTeT | kim0: that, and 'oh, this isn't like vmware at all' | 13:32 |
| kim0 | TeTeT: I agree, Would be awesome if you can start contributing to that too :) | 13:32 |
| kim0 | it seems you hear lots of feedback | 13:32 |
| kim0 | aw I can understand someone's disappointment comparing uec to vsphere | 13:33 |
| TeTeT | kim0: sure, I'll prepare something. I want to present UEC / ensemble / landscape to our key customer in Germany any time anyway | 13:33 |
| hazmat | niemeyer, indeed i did, bad allergies some seasons. mostly a factor of location afaict | 13:33 |
| kim0 | Awesome | 13:33 |
| niemeyer | hazmat: Aw :( | 13:33 |
| TeTeT | kim0: so for that introduction I'd better come up with something written anyway, which we can then recycle for the docs, I hope | 13:33 |
| kim0 | Absolutely .. sounds good | 13:34 |
| niemeyer | hazmat: I'll send good energy waves towards you then ;-) | 13:34 |
| hazmat | niemeyer, thanks :-) | 13:34 |
| kim0 | oh can you do that | 13:35 |
| * kim0 points the antenna | 13:35 | |
| niemeyer | kim0: ;-) | 13:35 |
| kim0 | I just played with debug-log .. and love it .. except I want to split out text into columns per service unit | 13:36 |
| kim0 | anyone has a magic one liner | 13:36 |
| niemeyer | kim0: I can certainly try, at least | 13:36 |
| kim0 | if it'll take you more than a couple of mins, I might want to bang my head a bit at it | 13:37 |
| kim0 | niemeyer: ideally .. the line order would be preserved by printing empty lines on columns of service units not executing anything | 13:37 |
| kim0 | i.e. looking at the resulting text table .. you'd easily see the order of events triggering | 13:38 |
| niemeyer | kim0: No trivial one-liners that I can think of | 13:38 |
| * kim0 wonders if he spotted a missing unix kung foo tool | 13:38 | |
| kim0 | okie .. I'll take a shot then | 13:39 |
| kim0 | probably needs some custom python | 13:39 |
| hazmat | kim0, there's also several filtering/inclusion options if you want to isolate a particular | 13:41 |
| hazmat | kim0, do you have an example of what sort of columnization you'd want to see? | 13:42 |
| kim0 | hmm yeah .. those would be great too .. but I'm really more into looking at the big picture now .. just need a prettier big picture to look at :) | 13:42 |
| kim0 | Yeah sure | 13:42 |
| kim0 | cloumn per service uit | 13:42 |
| kim0 | unit* like | 13:42 |
| kim0 | mysql/0 | wordpress/0 | 13:42 |
| kim0 | joined | -- | 13:42 |
| kim0 | -- | starting db-relation-changed | 13:42 |
| kim0 | something like that I guess would be great | 13:43 |
| kim0 | of course should be tab aligned | 13:43 |
| kim0 | any way .. I'll try it myself a bit | 13:43 |
| kim0 | guess distributed logs weren't that popular 20 years ago :) | 13:43 |
| hazmat | yeah.. i think thats possible, but possibly we'll end up going very wide. the stdout of the hooks might also interrupt the visual flow | 13:44 |
| kim0 | hazmat: yeah, which is another thing | 13:45 |
| kim0 | currently we have no way to log at different levels from inside hooks right | 13:45 |
| hazmat | kim0, there is an ensemble-log command available to hooks, but i don't think we currently utilize it, we just capture stdout/stderr of hooks.. at a set log level | 13:46 |
| kim0 | ah .. that'd be great | 13:46 |
| kim0 | to use :) | 13:46 |
| kim0 | wonder if I should add that to the examples/ formulas | 13:46 |
| hazmat | kim0, sounds good, probably we should also verify its working.. | 13:47 |
| kim0 | hazmat: if ensemble-log is used today, can we filter to only see its output and not stdout/stderr ? | 13:47 |
| hazmat | kim0, also i've noticed that install, start hooks aren't commonly reported, this is as per the async watches and creator thread on this list.. after we fix that up, we should be able to use logs to show people hook sequencing/interaction for a tutorial. | 13:48 |
| * hazmat checks on ensemble-log | 13:48 | |
| kim0 | hazmat: oh that's what I was doing (log sequencing for a tutorial) :) so it's not working now .. I see | 13:49 |
| hazmat | kim0, minus the install/start its correct | 13:51 |
| hazmat | looking at the log implementation, it looks like its broken to me | 13:51 |
| kim0 | yummy | 13:52 |
| hazmat | hmm. maybe not.. the test looks correct | 13:52 |
| kim0 | good then | 13:53 |
| kim0 | can someone exaplin the workflow for editing (or creating new) formula .. | 13:54 |
| kim0 | I suppose I deploy, and it breaks horribly | 13:54 |
| kim0 | how do I fix, and try again | 13:54 |
| kim0 | I hope I dont have to tear down everything for ever changed line | 13:55 |
| kim0 | every* | 13:55 |
| hazmat | kim0, looks good actually re ensemble-log | 13:57 |
| kim0 | hazmat: great :) thanks | 13:57 |
| kim0 | I will start a branch with nice log messages | 13:57 |
| hazmat | kim0, we'll have two commands that can assist with this, ensemble formula-upgrade to replace a formula that's in place, and ensemble resolved to correct for any hook errors. | 14:17 |
| hazmat | kim0, also ensemble debug-hooks for interactive debugging. | 14:18 |
| kim0 | yeah I guess debug-hooks helps with what I need .. will play with that too | 14:19 |
| hazmat | kim0, there's one missing piece to debug-hook that's still in review... i've kinda let it slide while i was working on upgrades. | 14:19 |
| hazmat | kim0, re ensemble resolved.. on a hook error, there's a change to an error workflow state within the unit agent (either at a unit or relation level, depending on the hook that fails). | 14:19 |
| hazmat | the ensemble resolved command will transition back to a known good state, optionally refiring the hook | 14:19 |
| kim0 | hmm that sounds magical :) | 14:20 |
| hazmat | formula upgrades are only allowed from a unit that's running (ie. not in an error state). | 14:20 |
| kim0 | we'll talk more about it in the meeting I guess | 14:20 |
| _mup_ | ensemble/unit-agent-formula-upgrade r233 committed by kapil.thangavelu@canonical.com | 15:49 |
| _mup_ | upgrade unit agent tests, remove duplicate test, add some state assertions, clean up doc strings. | 15:49 |
| niemeyer | jimbaker: ping | 15:50 |
| _mup_ | ensemble/trunk-merge r185 committed by kapil.thangavelu@canonical.com | 15:51 |
| _mup_ | merge trunk | 15:51 |
| jimbaker | niemeyer, hi | 15:53 |
| niemeyer | jimbaker: nm, already providing a new review | 15:54 |
| jimbaker | niemeyer, sounds good, hopefully the reworked intro looks good | 15:55 |
| niemeyer | jimbaker: Actually, can you please roll back that big introduction you added? | 15:55 |
| niemeyer | jimbaker: It doesn't :( | 15:56 |
| niemeyer | jimbaker: There are several issues in there | 15:56 |
| jimbaker | niemeyer, introductions are always tough | 15:56 |
| niemeyer | jimbaker: and I'm afraid we're getting into a review loop | 15:56 |
| jimbaker | so it makes sense that it's problematic | 15:56 |
| niemeyer | jimbaker: Can you please roll that back and address just the specific issues mentioned in the review? | 15:56 |
| niemeyer | jimbaker: So we can merge | 15:56 |
| niemeyer | jimbaker: Feel free to provide another change suggestion later on | 15:56 |
| jimbaker | everytime i look at the overall intro for ensemble, i think how much it needs to be reworked too | 15:57 |
| niemeyer | jimbaker: Some examples: | 15:57 |
| niemeyer | jimbaker: We don't offer total ordering guarantees to the user | 15:57 |
| niemeyer | jimbaker: It's not true that machines and service units have a 1-1 relationship | 15:57 |
| niemeyer | jimbaker: None of the text in the first bullet point is about scalability | 15:57 |
| jimbaker | niemeyer, this makes sense | 15:57 |
| niemeyer | jimbaker: It's not true that every event in Ensemble is translated into a hook execution | 15:58 |
| niemeyer | jimbaker: Etc | 15:58 |
| niemeyer | jimbaker: I'd really like to merge the main changes which this branch was trying to implement at this point, which is fixing the hook documentation | 15:58 |
| niemeyer | jimbaker: Rather than stepping through that unrelated review which was introduced post-reviewing | 15:58 |
| jimbaker | it sounds like i still wasn't careful enough in the descriptions, despite my attempt to do so | 15:58 |
| jimbaker | so definitely needs another branch to make even more precise | 15:59 |
| jimbaker | or precise at all | 15:59 |
| niemeyer | jimbaker: Cool, feel free to push those into another branch | 15:59 |
| niemeyer | jimbaker: It feels like a good direction, but feels bad to have that in mid-review | 15:59 |
| jimbaker | ok, i will roll that back. should i simply remove the intro as a whole, just have it start at "this section describes how formulas..." | 16:00 |
| jimbaker | ? | 16:00 |
| niemeyer | jimbaker: No, put the version you had previously, and apply the review | 16:00 |
| niemeyer | jimbaker: There were very simple changes to do only | 16:01 |
| jimbaker | niemeyer, ok, i guess i mixed up what you wanted. so you just wanted the one sentence removed, that's it? | 16:01 |
| niemeyer | jimbaker: Yes | 16:01 |
| _mup_ | ensemble/unit-agent-formula-upgrade r235 committed by kapil.thangavelu@canonical.com | 16:06 |
| _mup_ | doc work, from review comments | 16:06 |
| hazmat | niemeyer, on the formula-upgrade point 9, its seems the right solution then is to bypass the machine formula and just have the unit agent download directly to temp space and extract? there's a few other places where the unit agent pokes outside of its directory (for workflow state state files), and perhaps with on disk hook execution queue in the future. | 16:08 |
| niemeyer | hazmat: Workflow state should really be within its directory too | 16:09 |
| hazmat | niemeyer, i've been trying to segment the unit agent state files from the unit itself, ie outside of the unit. so we're saying we want to have all of these files within the unit? | 16:10 |
| niemeyer | hazmat: Hmm.. will the unit agent run within the unit? | 16:11 |
| niemeyer | hazmat: Once we have containers | 16:11 |
| hazmat | niemeyer, its outside the unit | 16:11 |
| _mup_ | ensemble/new-hook-semantics-5-docs r203 committed by jim.baker@canonical.com | 16:11 |
| _mup_ | Removed intro for incorporation into a future branch | 16:11 |
| hazmat | niemeyer, i was planning on using lxc-attach to execute hooks within the container | 16:11 |
| hazmat | it gave us additional separation from the container, and so i did the same with agent state files. | 16:11 |
| niemeyer | hazmat: It's not yet clear if that's a good idea or not | 16:12 |
| niemeyer | hazmat: In either case, units shouldn't be poking in the machine agent directory at least | 16:12 |
| hazmat | niemeyer, in that case we either have the machine agent download the formula or bypass the formula cache on the machine | 16:13 |
| niemeyer | hazmat: Yeah, I think it should bypass | 16:14 |
| niemeyer | hazmat: We have no idea about what the machine agent is doing.. it might decide to clear that directory and remove the formula under our feet | 16:14 |
| hazmat | niemeyer, it does make the formula-cache on the machine stale.. but if we can live with that.. | 16:14 |
| niemeyer | hazmat: It's as stale as the machine agent decides it to be | 16:14 |
| hazmat | true | 16:14 |
| hazmat | niemeyer, sounds good | 16:15 |
| niemeyer | hazmat: The point is that this directory is owned by a different process | 16:15 |
| niemeyer | hazmat: Also, what happens if two different units decide to download the same formula at the same time? | 16:15 |
| niemeyer | I'll head to lunch.. Ale is waiting | 16:15 |
| niemeyer | biab | 16:15 |
| === niemeyer is now known as niemeyer_lunch | ||
| jimbaker | niemeyer_lunch, i have pushed up the change. this removes the new intro and replaces it with the old intro, cutting the first two sentences (didn't seem to make sense to discuss service units at that point) | 16:21 |
| _mup_ | ensemble/unit-agent-formula-upgrade r236 committed by kapil.thangavelu@canonical.com | 16:22 |
| _mup_ | download formula upgrade to temp directory. | 16:22 |
| === deryck is now known as deryck[lunch] | ||
| * hazmat lunches | 16:36 | |
| kim0 | when using ensemble-log inside a formula .. do I log at DEBUG level ? | 16:47 |
| kim0 | ensemble-log --log-level DEBUG "foo bar" ? | 16:48 |
| kim0 | I hope to be able to get those messages with debug-log *without* getting stdout/stderr from hooks | 16:48 |
| === niemeyer_lunch is now known as niemeyer | ||
| kim0 | niemeyer: Trying to add proper ensemble-log messages in the example formula .. any comment on ^ | 17:06 |
| niemeyer | kim0: Hmm.. whether it's DEBUG or not depends on the nature of the message | 17:07 |
| kim0 | niemeyer: if it's DEBUG, can I still separate it from stdout though | 17:07 |
| niemeyer | kim0: Yes, IIRC ensemble-log doesn't print to stdout | 17:07 |
| kim0 | great .. so how do I get debug-log to filter out stdout/stderr | 17:08 |
| niemeyer | kim0: Hmmm.. I see what you want | 17:09 |
| niemeyer | hazmat: Do you recall which log-level stdout/err messages get logged as? | 17:09 |
| kim0 | isn't DEBUG the lowest level .. i.e. I hope there's some other way to filter other than by log level? | 17:12 |
| niemeyer | kim0: Yes, it's the lowest/highest level (depending how one looks at it) | 17:17 |
| niemeyer | kim0: I don't think I really understand what is the problem you're facing, though | 17:17 |
| kim0 | niemeyer: well I wanted to use ensemble-log from inside formulas, gather those log messages using debug-log, but do not wish them intermingled with stdout/stderr. If you tell me the way to do this, is to log at a loglevel != DEBUG, that's fine I guess .. was just wondering how to do it | 17:34 |
| niemeyer | kim0: Right, that's what you want to do it.. but what's the background on why you want to do it? | 17:37 |
| niemeyer | kim0: There's no reason for a hook to expose stdout/stderr besides for debugging/understanding purposes | 17:37 |
| _mup_ | Bug #766241 was filed: Add dummy provider support for port authorization/revocation <Ensemble:New> < https://launchpad.net/bugs/766241 > | 17:37 |
| === deryck[lunch] is now known as deryck | ||
| hazmat | niemeyer, debug | 17:48 |
| kim0 | yeah .. it's just that "set -eux" is too noisy! | 18:02 |
| hazmat | kim0, you can also explicit exclude just the hook output | 18:08 |
| niemeyer | kim0: Hmmm.. we shouldn't be using x I think | 18:08 |
| kim0 | hazmat: hmm how would I do htat | 18:09 |
| kim0 | niemeyer: yeah I'd think so | 18:09 |
| hazmat | kim0, ensemble debug-log -x hook-output | 18:09 |
| kim0 | hazmat: does that include stderr ? | 18:09 |
| hazmat | kim0, yes that's all hook output | 18:09 |
| kim0 | hazmat: oh thanks | 18:09 |
| hazmat | kim0, checkout out some of the other options on debug-log --help | 18:09 |
| kim0 | hazmat: help message doesn't really make it clear what "log channels" I can include/exclude | 18:11 |
| * hazmat nods | 18:12 | |
| hazmat | it needs some more examples | 18:12 |
| hazmat | in the help message | 18:12 |
| hazmat | and perhaps at least a partial listing of log channels | 18:12 |
| jimbaker | kim0, set -x is definitely expedient, and it really makes it possible to see what's going on. but if i could just observe the relation settings change in the debug log, that would probably eliminate a lot of debugging statements | 18:14 |
| kim0 | jimbaker: yeah, I'm just using ensemble-log at INFO level for now | 18:16 |
| jimbaker | hazmat, i think we should also add the relation settings info to ensemble status too, now that we have states there. we can always provide an options flag for what to show | 18:16 |
| kim0 | jimbaker: are you saying, relation setting changes don't go to logs now | 18:16 |
| kim0 | ah yeah | 18:16 |
| jimbaker | kim0, i don't believe they are otherwise observable. could be wrong :) | 18:17 |
| kim0 | nah :) | 18:17 |
| hazmat | jimbaker, yeah. it would be nice to have a cli flag to enable that | 18:17 |
| jimbaker | kim0, but clearly the relation settings, along with the specific hooks firing, are the key to understanding what actually happens | 18:17 |
| hazmat | kim0, er.. that should be debug-log -x hook.output | 18:17 |
| kim0 | oh ok | 18:18 |
| kim0 | hazmat: indeed at least popular channels should be listed on the help message please | 18:18 |
| kim0 | do we need a bug for showing relation-settings changes in logs .. or is someone on that | 18:19 |
| jimbaker | kim0, we need a bug for that and in status too i think | 18:20 |
| jimbaker | (so two bugs) | 18:20 |
| kim0 | so "ensemble status" shows relation settings for every service unit, right? | 18:20 |
| jimbaker | kim0, it does not yet. but i think it should | 18:21 |
| kim0 | yeah, I meant that's how you'd want it .. cool .. filing | 18:21 |
| jimbaker | the only consideration here is that there security issues | 18:21 |
| kim0 | ah passwords and stuff | 18:21 |
| jimbaker | but i think these be addressed later to mask it out properly. any system auditing always has these issues from what i have seen in the past | 18:21 |
| kim0 | hmm ok | 18:22 |
| jimbaker | not showing it in ensemble status is simply doing it by obscurity anyway | 18:22 |
| jimbaker | so the good thing about showing it in ensemble status sooner than later is to keep reminding ourselves of this problem ;) | 18:23 |
| jimbaker | kim0, you might be interested in this intro fragment, http://pastebin.ubuntu.com/596140/ - it's not going into the branch lp:~jimbaker/ensemble/new-hook-semantics-5-docs because of its lack of precision is currently misleading | 18:31 |
| jimbaker | but there's some useful text in there that we may want to incorporate into the ensemble docs, after some more editing | 18:32 |
| kim0 | jimbaker: thanks :) | 18:33 |
| jimbaker | kim0, re state diagrams - you should be aware of sdedit, if not already - see http://sdedit.sourceforge.net/example/index.html | 18:34 |
| jimbaker | there's integration with sphinx too | 18:35 |
| jimbaker | (much like the dot integration) | 18:35 |
| jimbaker | i could definitely see a script reading the debug log and producing this type of output | 18:36 |
| kim0 | jimbaker: thanks useful info :) | 18:37 |
| jimbaker | (well not loops, but the example is just an indication of what can be done, if there's data to support it) | 18:37 |
| _mup_ | Bug #766306 was filed: example formulas should use ensemble-log to log useful messages <Ensemble:New> < https://launchpad.net/bugs/766306 > | 18:41 |
| niemeyer | jimbaker: Its purpose is to allow the formula to | 18:41 |
| niemeyer | 255+ clean up established state | 18:41 |
| niemeyer | 256+ | 18:41 |
| niemeyer | 257+ The service unit can then clean up any established state. | 18:41 |
| jimbaker | niemeyer, thanks | 18:42 |
| niemeyer | jimbaker: No problem, +1 with that | 18:43 |
| jimbaker | niemeyer, cool, i will get it merged in shortly! | 18:43 |
| kim0 | why doesn't the bot here catch merge proposals :) | 18:44 |
| jimbaker | kim0, yeah, it's an annoying bug in our mup | 18:44 |
| * kim0 kicks _mup_ | 18:45 | |
| _mup_ | Bug #766315 was filed: status should show relation settings <Ensemble:New> < https://launchpad.net/bugs/766315 > | 18:46 |
| niemeyer | kim0: Strange problem indeed | 18:47 |
| _mup_ | Bug #766317 was filed: debug-log should show relation settings changes <Ensemble:New> < https://launchpad.net/bugs/766317 > | 18:48 |
| kim0 | filed the two bugs | 18:48 |
| kim0 | TeTeT: you might enjoy this text too http://pastebin.ubuntu.com/596140/ | 18:54 |
| TeTeT | kim0: looks great :) | 18:55 |
| _mup_ | ensemble/new-hook-semantics-5-docs r204 committed by jim.baker@canonical.com | 18:55 |
| _mup_ | Fixed -relation-broken description | 18:55 |
| _mup_ | ensemble/trunk r200 committed by jim.baker@canonical.com | 19:07 |
| _mup_ | merge new-hook-semantics-5-docs [r=niemeyer][f=756581] | 19:07 |
| _mup_ | Updates the documentation of hooks with respect to the new hook | 19:07 |
| _mup_ | semantic changes. Copy edits the formula section as a whole to clarify | 19:07 |
| _mup_ | how hooks actually execute, and under what conditions. | 19:07 |
| jimbaker | cool, 200th revision on trunk | 19:09 |
| kim0 | woohoo :) | 19:11 |
| _mup_ | ensemble/unit-agent-formula-upgrade r237 committed by kapil.thangavelu@canonical.com | 19:25 |
| _mup_ | add utility workflow function to tell if a unit is running and its state. | 19:25 |
| hazmat | jimbaker, we're closing on 1k unit tests as well | 19:26 |
| hazmat | niemeyer, jimbaker, bcsaller standup? | 19:26 |
| jimbaker | hazmat, sounds good on all counts! | 19:26 |
| bcsaller | yeah | 19:26 |
| jimbaker | (so to speak :) | 19:26 |
| niemeyer | Yep, let's do it | 19:26 |
| hazmat | bcsaller, jimbaker skype? | 19:27 |
| bcsaller | I'm on | 19:28 |
| niemeyer | Up | 19:28 |
| hazmat | bcsaller, cool, false skype presence info | 19:28 |
| _mup_ | ensemble/unit-agent-formula-upgrade r238 committed by kapil.thangavelu@canonical.com | 19:44 |
| _mup_ | clear upgrade before flag before retrieving service formula id, remove executor run lock acquisition when starting | 19:44 |
| _mup_ | ensemble/unit-agent-formula-upgrade r239 committed by kapil.thangavelu@canonical.com | 19:55 |
| _mup_ | missing test for is_unit_running | 19:55 |
| _mup_ | ensemble/yaml-state-node r194 committed by bcsaller@gmail.com | 20:03 |
| _mup_ | simpler YAMLState | 20:03 |
| _mup_ | reflecting the changes from review this version maintains less internal state | 20:03 |
| _mup_ | additional test of multiple writes | 20:03 |
| niemeyer | hazmat: There are still many appearances of 'cli' in the file.. we should probably try to put acronyms in capitals in user documentation | 20:38 |
| hazmat | niemeyer, hmm.. i never considered the possiblity that acronoym would be unknown. | 20:38 |
| hazmat | fair enough re caps | 20:38 |
| niemeyer | hazmat: Uh.. I'm not saying it's unknown.. it's just an acronym | 20:39 |
| niemeyer | hazmat: same way that API is generally capitals | 20:39 |
| niemeyer | and AWS, etc | 20:39 |
| _mup_ | ensemble/ensemble-resolved r240 committed by kapil.thangavelu@canonical.com | 20:44 |
| _mup_ | ensemble resolved cli | 20:44 |
| niemeyer | hazmat: :-) | 20:45 |
| hazmat | niemeyer, doh. ;-) | 20:45 |
| niemeyer | hazmat: The branch looks good | 20:45 |
| hazmat | niemeyer, yeah.. i still need to do an end to end test, but else it should be good. | 20:46 |
| niemeyer | hazmat: Was a bit overwhelming, and required some morning-level freshness, but mostly because there's a lot there.. was pretty straightforward | 20:47 |
| hazmat | niemeyer, cool. yeah it grew a bit bigger than i wanted in one branch, but it was getting hard to separate out | 20:47 |
| hazmat | niemeyer, fwiw, i'm planning on doing the resolved stuff much the same way | 20:48 |
| hazmat | unit_state.set_resolved_flag(retry).. unit_state.set_relation_resolved_flag(mapping) | 20:48 |
| hazmat | mapping being relation_state_id: retry_boolean | 20:48 |
| niemeyer | hazmat: Hmm | 20:49 |
| hazmat | and then watches on the agent for the unit, and a watch in the lifecycle for the relation | 20:49 |
| niemeyer | hazmat: Why do we need a flag on it? | 20:49 |
| hazmat | niemeyer, we don't, probably shouldn't have that in the name | 20:49 |
| hazmat | looks like i can deal with the ambiguous anonymous relations without issue here | 20:50 |
| niemeyer | hazmat: I mean.. isn't the fact we have an _error state already indicating a non-resolved state, and thus putting the unit back in running would be the action? | 20:50 |
| hazmat | niemeyer, i need to signal to the agent that it should take action | 20:50 |
| hazmat | the resolved stuff triggers the watch which triggers the agent to do some work | 20:51 |
| niemeyer | hazmat: Won't it notice you changing its state? | 20:51 |
| hazmat | the workflow transition even sans hook needs to take place on the agent | 20:51 |
| hazmat | niemeyer, no it is the owner of that data, directly changing the state, could be arbitrary, without the nesc. lifecycle invocations | 20:52 |
| hazmat | i could genericize it via the workflow state client | 20:53 |
| niemeyer | hazmat: Ok, sounds good | 20:53 |
| hazmat | to allow marking generic firing of transitions and recording that as intent to act upon | 20:53 |
| niemeyer | hazmat: Nah, your idea of a semantically rich request actually feels good | 20:53 |
| hazmat | niemeyer, cool | 20:54 |
| niemeyer | hazmat: Otherwise, you're right in that the agent can't do anything more intelligent with the request | 20:54 |
| niemeyer | hazmat: "Oh, I'm resolving, thus ..." | 20:54 |
| hazmat | niemeyer, yeah.. or like oh.. my state is stopped now, what does that mean.. where was was i, this is not beautiful house ;-) | 20:55 |
| hazmat | ^my | 20:55 |
| niemeyer | hazmat: Right :-) | 20:55 |
| _mup_ | ensemble/ensemble-resolved r241 committed by kapil.thangavelu@canonical.com | 21:03 |
| _mup_ | add a is_relation_running workflow utility function. | 21:03 |
| _mup_ | ensemble/yaml-state-node r195 committed by bcsaller@gmail.com | 21:06 |
| _mup_ | merge trunk | 21:06 |
| _mup_ | ensemble/unit-agent-formula-upgrade r240 committed by kapil.thangavelu@canonical.com | 21:11 |
| _mup_ | make sure the upgrade flag presence check is done before we clear the flag. | 21:11 |
| hazmat | niemeyer, any thoughts on the async watcher and creators stuff? | 21:11 |
| niemeyer | hazmat: Sorry, what's the context again? | 21:11 |
| niemeyer | hazmat: Ah, the email on the list | 21:12 |
| niemeyer | hazmat: I forgot to reply.. let me look at it again | 21:12 |
| hazmat | niemeyer, my mail to the list regarding usage patterns for watchers | 21:12 |
| hazmat | niemeyer, thanks | 21:12 |
| niemeyer | hazmat: Sorry about that | 21:12 |
| hazmat | i'm noticing a lot of tests i'm hacking in fire_transition("stop") to stop the background activity so the test doesn't fail because of the async work | 21:12 |
| _mup_ | ensemble/trunk r201 committed by bcsaller@gmail.com | 21:15 |
| _mup_ | Merge yaml-state [r=niemeyer] [f=758583] | 21:15 |
| _mup_ | 21:15 | |
| _mup_ | Adds a reusable utility for managing access to Zookeeper nodes containing YAML serialized state. The YAMLState object proxies as a Python dict for normal operation. Prior to access instance.read() should be invoked and instance.write() will update the Zookeeper state. | 21:15 |
| niemeyer | hazmat: Replied | 21:22 |
| niemeyer | hazmat: Yeah, that sucks | 21:22 |
| niemeyer | hazmat: I have an idea about a better system to make testing this easier | 21:23 |
| niemeyer | hazmat: That's the good news.. the bad news is that I don't know how to implement it yet | 21:23 |
| hazmat | niemeyer, yeah.. i started poking adding something like this but it was getting a little complicated untop of all the work in the upgrade branch.. there's some subtly there. | 21:31 |
| hazmat | there where one or two tests that error'd out with the change, but i think i'd like to approach in a more systematic way. | 21:31 |
| niemeyer | hazmat: Like this what, more specifically? | 21:32 |
| hazmat | niemeyer, i added support to the unit lifecycle start, that it only fired after the unit relations watcher had fired once. | 21:32 |
| niemeyer | hazmat: I think we may come up with a simpler way to test that kind of logic altogether | 21:32 |
| niemeyer | hazmat: Which would get rid of all sleeps/waits/etc | 21:33 |
| niemeyer | hazmat: Higher bandwidth might be nice for that conversation, though | 21:34 |
| hazmat | niemeyer, agreed, let's do it for tomorrow's standup? or before the standup? | 21:34 |
| hazmat | i'm making good head way on 'resolved' | 21:35 |
| niemeyer | hazmat: Might be good to have it separated | 21:35 |
| niemeyer | hazmat: It may be pretty fast, but it could also take abit | 21:35 |
| niemeyer | hazmat: Just as a heads up, when testing we're in control of both the reactor and the zookeeper deployment | 21:35 |
| niemeyer | hazmat: We should be able to figure if there's something pending to be executed, or if the system is stale for real | 21:36 |
| niemeyer | hazmat: No deferred calls in twisted's reactor after a ping roundtrip to zookeeper means the system is stale | 21:37 |
| niemeyer | hazmat: Nothing will happen | 21:37 |
| niemeyer | I think we can use that to solve those problems | 21:37 |
| hazmat | niemeyer, in general for sequencing it feels like it would be easier for tests if the background activity was done after the method completed. | 21:38 |
| hazmat | at least in a steady state | 21:38 |
| niemeyer | hazmat: The background processing is always done after the method completed, so I don't see what you really mean | 21:39 |
| hazmat | inspecting the reactor for latent deferreds, and then doing something with that feels kinda of magical as well | 21:39 |
| hazmat | niemeyer, well the method could delay its return till the initial invocation of the watcher, at which point its in a steady state, minus any additional test induced state chagnes | 21:39 |
| niemeyer | hazmat: Well.. it is much less magical to me than arbitrarily saying the event has to happen in a given timeframe when in fact it doesn't | 21:39 |
| niemeyer | hazmat: The initial invocation of the method has no special meaning, as I pointed out in the list | 21:40 |
| niemeyer | hazmat: It can be empty, and then can have data 5 milliseconds later | 21:40 |
| hazmat | the event happens fairly quickly, just typically not before the test exits... part of the problem is arguably that we need to separate our tests a bit more, and not have tests which have relations that are being setup in the background. | 21:40 |
| hazmat | niemeyer, sure, it can, but the test is explicitly setting up a state, ideally the methods its invoking should be done/current with the state when complete. the test is the state driver/manipulator for changes. | 21:42 |
| niemeyer | hazmat: Yeah, exactly.. that "ideally" is the problem.. there's no hard constraint about that | 21:43 |
| hazmat | a change in data 5ms seconds later in this case is one thats being done by the test, and the test should expect/test for that | 21:44 |
| hazmat | which is also arguably why the test shouldn't be subclassing something that sets up a relation if doesn't care about it. | 21:44 |
| niemeyer | hazmat: No questions about that.. doesn't affect the points though | 21:44 |
| niemeyer | hazmat: The problem is that, e.g., "creation relation" + "set watch on relation" will fire in one way | 21:47 |
| niemeyer | hazmat: but "set watch on relation" + "create relation" fires in a different way | 21:47 |
| niemeyer | hazmat: Our system was built in such a way that things work well in both cases | 21:47 |
| niemeyer | hazmat: Adding special behavior to the first time it fires fails in the second case. | 21:48 |
| hazmat | the system always reads the current state, and tries to match it. i agree adding semantics to the first invocation feels a bit iffy perhaps.. but the caller knows then that the async watch creator has completed to matching the current state when its done, and that makes some of the tests a bit easier.. i think its something we can solve in practice for a large set of the cases by having the test setup reflect only what the tes | 21:50 |
| hazmat | t needs. | 21:50 |
| hazmat | niemeyer, but the problem still exists beyond tests | 21:50 |
| hazmat | for example the debug-log and debug-hook stuff are setting up watches, and then going about their business, and some information/behavior is lost | 21:51 |
| hazmat | because by the time those watches have finished, the install and start methods have already executed | 21:51 |
| niemeyer | hazmat: It makes tests easier and unrealistic.. the first invocation has no special meaning. It can have state or not, and it should correctly yield the same state with A+B or B+A | 21:51 |
| hazmat | niemeyer, a phone call would be better | 21:52 |
| niemeyer | hazmat: There are tons of things which right now simply work because of that | 21:52 |
| niemeyer | hazmat: Indeed, sorry.. I wanted to give a heads up about the idea and ended up taking your attention | 21:52 |
| hazmat | and they'll continue to work that way, but certain cases need to know that the async behavior matches the state before continuing | 21:52 |
| hazmat | no worries, we can discuss in more detail tomorrow | 21:53 |
| niemeyer | Cool | 21:55 |
| niemeyer | I'll push some personal stuff now.. want to get a new mgo release out tonight still. I'm hoping that after some work over the following nights and on the weekend it should be ready to push the prototype needs. | 21:57 |
| _mup_ | ensemble/unit-agent-formula-upgrade r241 committed by kapil.thangavelu@canonical.com | 22:21 |
| _mup_ | a test was incorrectly setting up the workflow state. | 22:21 |
| _mup_ | ensemble/trunk r202 committed by kapil.thangavelu@canonical.com | 23:43 |
| _mup_ | merge unit-agent-formula-ugprade [r=niemeyer][f=755062] | 23:43 |
| _mup_ | This merge completes the upgrade-formula functionality for ensemble, | 23:43 |
| _mup_ | by implementing the unit agent support for upgrading formulas and | 23:43 |
| _mup_ | executing formula-upgrade hooks. It changes the unit workflow to add | 23:43 |
| _mup_ | a new transition from the started state, that effects an upgrade on | 23:43 |
| _mup_ | the unit. It also adds the notion of out-of-band hook execution on | 23:43 |
| _mup_ | the hook executor for the formula-upgrade-hook to be invoked while | 23:43 |
| _mup_ | the executor is stopped but still queuing new hook executions. | 23:43 |
| _mup_ | ensemble/trunk-merge r186 committed by kapil.thangavelu@canonical.com | 23:45 |
| _mup_ | merge trunk | 23:45 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!