[00:01] Starting to fade [00:04] I think signatures are fully working [00:17] <_mup_> ensemble/unit-agent-resolved-part-two r269 committed by kapil.thangavelu@canonical.com [00:17] <_mup_> unit agent resolving units. [00:57] <_mup_> ensemble/unit-agent-resolved-part-two r270 committed by kapil.thangavelu@canonical.com [00:57] <_mup_> fix a typo in the stop transition action [01:08] <_mup_> ensemble/unit-agent-resolved-part-two r271 committed by kapil.thangavelu@canonical.com [01:08] <_mup_> round out unit agent resolved test cases. [01:11] <_mup_> Bug #776014 was filed: unit agent needs support for resolving units < https://launchpad.net/bugs/776014 > [01:14] okay.. that's the last branch of the resolved work. [01:14] * hazmat calls it a night [03:20] <_mup_> ensemble/config-set r209 committed by bcsaller@gmail.com [03:20] <_mup_> wip to show kapil [03:29] <_mup_> ensemble/config-set r210 committed by bcsaller@gmail.com [03:29] <_mup_> fix typo [03:58] bcsaller, this is what i was thinking of for the test api https://pastebin.canonical.com/47020/ [04:00] that looks like a good start [05:44] <_mup_> ensemble/expose-provisioning r233 committed by jim.baker@canonical.com [05:44] <_mup_> More work on testing cleanup and handling scenarios occurring at that time [06:15] <_mup_> ensemble/expose-provisioning r234 committed by jim.baker@canonical.com [06:15] <_mup_> Refactored callbacks [06:15] <_mup_> ensemble/expose-provisioning r235 committed by jim.baker@canonical.com [06:15] <_mup_> Refactored callbacks [06:57] <_mup_> ensemble/config-state-manager r208 committed by bcsaller@gmail.com [06:57] <_mup_> reapply expose stuff, shouldn't be here, but this fixes the issue [06:59] <_mup_> ensemble/expose-provisioning r236 committed by jim.baker@canonical.com [06:59] <_mup_> Properly expose services if provisioning agent restarts [08:58] Morning everyone === TeTeT_ is now known as TeTeT [09:54] <_mup_> ensemble/config-get r209 committed by bcsaller@gmail.com [09:54] <_mup_> still dealing with bad merge from before, forward prop changes [10:06] <_mup_> ensemble/config-set r212 committed by bcsaller@gmail.com [10:06] <_mup_> check point config set [13:57] niemeyer: Morning o/ [14:02] g'morning folks [14:03] niemeyer, i think that's the last branch for resolved in review [14:11] hmm why is it that my merges to other projects like https://code.launchpad.net/~ubuntu-branches/ubuntu/natty/logrotate/natty [14:11] appear with my name on them, while to ensemble, the reviwers name is put instead [14:12] not that I care much, but new members will probably be too excited seeing their own name besides their formulas and stuff [14:19] kim0, did you commit the merge on the other projects? [14:19] hazmat: no I don't have upload rights [14:19] I think Daviey did for logrotate [14:20] kim0, we have this commit message notation syntax for when doing merges on behalf of other users [a=kim0]... does seem a little strange [14:21] that bzr would pull the commit as a merge in one case.. and pull the underlying commits in the otehr [14:21] yeah :/ [14:21] kim0, it looks like that was actually committed by a daemon, the committer and author metadata at the bzr level is different [14:22] ah.. ic.. you can pass --author to bzr commit [14:22] yeah.. we should start using that [14:23] hazmat: so that is something you guys would need to do, not the person creating the branch right [14:23] kim0, yup.. in the case of that logrotate branch it looks like some automated process kicked in and did the merge after review [14:24] cool .. @everyone please do that then while merging :) [14:24] thanks hehe [14:25] Another thingie, I'm starting a new little doc on writing and contributing an Ensemble formula [14:25] for that, I need a simplistic "hello world" style formula [14:26] any suggestions ? [14:26] I think someone mentioned there could be some user formula for adding/deleting users .. that sounds simply enough, but not sure how it'd work [14:27] maybe a motd formula! [14:27] kim0, that's machine config, not service deployment. [14:28] well yeah I understand .. mm [14:30] kim0, i agree its something we should have some facilities for, machine level policies [14:30] I mean it can be done today right ?! [14:30] it's just that we don't want to huh [14:31] so if an admin wants to create users (or change motd) .. do we say ensemble is not for that [14:34] hazmat, kim0: Good morning! [14:34] hazmat: That's awesome [14:34] hazmat: I'll have a full pass today [14:34] niemeyer, great [14:35] Auth actually works, btw! [14:35] niemeyer, i'm thinking i should do some work with ben on the service-config.. else i can dig back into the debug-hook stuff [14:35] hazmat: http://pastebin.ubuntu.com/602773/ [14:35] hazmat: Sounds good, it feels like he'd benefit from a hand on the hooks part [14:36] niemeyer, cool [14:36] re auth [14:37] hazmat: Yeah, it turned out pretty neat. This enables us to authenticate people in the repo [14:38] hazmat: Sending them back and forth to Launchpad for authentication [14:38] kim0, they can add users as part of managing a service [14:38] kim0, for example someone doing a heroku clone, might deploy a service formula per user, and the individual units could create a new user representing that customer on a machine for launching the appserver... [14:39] Yeah, got it .. so what's a good hello-world formula, that hopefully doesn't involve complex config files ..etc [14:43] kim0, one that prints 'hello world'.. really though if you want to just have it add a user its fine.. just overkill [14:43] kim0, i'd probably do one that uses a packaged service.. like take nginx or apache as an example.. as it provides enough functionality to grow the formula over multiple tutorial parts. [15:26] <_mup_> Bug #776426 was filed: Add debug hook cli flag for deploy and and add-unit. < https://launchpad.net/bugs/776426 > [15:53] i should be available when there, but just in case i'm silent: i need to take my toyota to the dealer === niemeyer is now known as niemeyer_lunch [16:50] bcsaller, just sent you an email regarding the remaining stuff i could think of for service-config.. give me a ping when you've got a moment [16:50] bzr cli parsing does seem a bit nicer than argparse [16:56] hazmat, i think subcommand support was never really completed in argparse. and that's where the comparison w/ bzr is pretty obvious [16:57] jimbaker, subcommands are fine, its the help/output formatting usage and customization that's atrocious imo [16:57] hazmat, but that is part of what it should be - good help for subcommands [16:59] makes sense that in combining functionality - help + subcommands, for example - would should the most gaps in something like argparse === niemeyer_lunch is now known as niemeyer [17:49] hazmat: I realized last night that some of the files on that branch were not commited. Hopefully you were able to pull them when you had a look this morning? [17:50] bcsaller, i had a look and setup a pipeline, but there where a lot of conflicts with trunk [17:50] bcsaller, any chance you could resolve those and push an updated version? i'd like to use a clean branch point to start with [17:51] I can try to remerge trunk in the last phase of the pipeline, sure. I didn't realize [17:51] there is no conceptual conflict so it doesn't make sense [17:53] hi all [17:54] bcsaller, i've found typically its better to merge at the start of the pipeline (i use a trunk-merge pipe) and then pump the changes through [17:54] yes, but the start of my pipeline was already merged [17:54] I don't know if thats the source of the issues [17:54] bcsaller, you can add a secondary merge-trunk pipeline after that [17:54] ahh [17:55] I'm happy to try that [17:58] bcsaller, out of curiosity what's the name of the hook (for config-changed)? [17:58] config-changed [17:59] later part of the spec has it [18:00] bcsaller, cool, i'm starting off with a fresh trunk branch for the moment [18:00] bcsaller, what's that assessment of parts that are ready or almost ready accurate? [18:00] s/was that [18:01] minus any of the merge issues they are fine, I'll try merging trunk again and seeing if it breaks things or not [18:09] <_mup_> ensemble/service-config-unit-lifecycle r215 committed by kapil.thangavelu@canonical.com [18:09] <_mup_> unit lifecycle.configure method for processing service configuration changes. [18:12] <_mup_> ensemble/expose-provisioning r237 committed by jim.baker@canonical.com [18:12] <_mup_> Comments and method rename to indicate (somewhat more) private api [18:17] i'm not entirely clear what the error handling looks like for configure errors should look like. [18:18] bcsaller, any thoughts on what that behavior should be? [18:18] <_mup_> ensemble/expose-provisioning r238 committed by jim.baker@canonical.com [18:18] <_mup_> Removed eventual consistency support for incorporation into a later branch [18:18] let me see if I covered that in the spec, one sec [18:19] " Errors in the config-changed hook force ensemble to [18:19] assume the service is no longer properly configured. If the service is [18:19] not already in a stopped state it will be stopped and taken out of [18:19] service" [18:20] stopped and taken out of service are slightly different, but easy enough.. it will mean missed hook executions though for relations, if its brought back online [18:28] <_mup_> ensemble/expose-provisioning r239 committed by jim.baker@canonical.com [18:28] <_mup_> Test comments [18:34] <_mup_> ensemble/config-get r210 committed by bcsaller@gmail.com [18:34] <_mup_> better merge resolution, less juggling [18:36] this branch is finally getting into a good state for review. i am relieved [18:36] just need a few more tests... [18:36] <_mup_> ensemble/service-config-unit-lifecycle r216 committed by kapil.thangavelu@canonical.com [18:36] <_mup_> unit workflow support for service configuration [18:38] <_mup_> ensemble/config-set r213 committed by bcsaller@gmail.com [18:38] <_mup_> Better ptests in Makefile, sensitive to new files properly [18:38] <_mup_> Bug #776596 was filed: unit workflow and lifecycle must support service configuration < https://launchpad.net/bugs/776596 > [18:42] bcsaller, just pushed the lifecycle/worfklow changes for review based on trunk, i'm going to switch tracks back to debug-hook, unless you'd like me to work on the watch_config_changed stuff? [18:42] you're fast [18:43] I'll look it over, but thats very helpful. Work on the debug hook [18:43] bcsaller, practice makes perfect.. there's some work to be done wrt to resolved, but that should probably land first [18:46] <_mup_> Bug #776605 was filed: Hooks need access to service options < https://launchpad.net/bugs/776605 > [19:00] * hazmat bangs head against merge === marrusl is now known as marrusl_afk [19:29] * niemeyer switches to reviewing mode [19:35] > +class ServiceUnitRelationResolvedAlreadyEnabled(StateError): [19:35] > + """The unit has already been marked resolved. [19:35] > Documentation needs updating. [19:35] how so? [19:35] hazmat: s/unit/relation/ [19:36] hazmat: [19:36] """ [19:36] It is actually making a request for the problem to be solved, not solving the [19:36] problem directly, as there are behavioral aspects to transitions that need be [19:36] done inside of the unit agent. [19:36] """ [19:36] hazmat: That's not the case, last we spoke [19:37] hazmat: resolved means "I have resolved the problem." (hence the 'd') [19:37] hazmat: The problem as in "the reason why this hook failed" [19:39] Argh.. [19:39] What was the last message I sent? === niemeyer_ is now known as niemeyer [19:46] niemeyer, failed." [19:46] hazmat: Ok, so it did went through, thanks [19:46] hazmat: I ended up just providing better organized comments in the merge proposal [19:46] hazmat: So feel free to ignore the IRC stuff [19:46] niemeyer, resolved is a request to make something happen.. [19:47] hazmat: Yes, to allow the relation/unit to go ahead [19:47] its marking the unit as resolved, but the actual resolution happens on the agent [19:47] hazmat: It's a statement made by the user that the problem has been resolved [19:47] hazmat: No.. [19:47] hazmat: The resolution has already happened [19:47] hazmat: We're simply telling the agent "go on" [19:47] niemeyer, it hasn't till the agent processes the resolved request [19:47] hazmat: I hope one day we can make agents which resolve problems, though ;-) [19:48] hazmat: It has happened [19:48] hazmat: That's why the command is "resolved" and not "resolve" [19:48] niemeyer, the request has been made, but not acted upon [19:48] hazmat: The request to go on, yes [19:48] yes [19:48] hazmat: Not the request to resolve the problem which caused the script/hook to fail [19:48] but till its processed by the agent, the goal is not accomplished. [19:49] hazmat: I'm banging on that because that's the idea we have to provide to users [19:49] hazmat: Agreed.. it's just terminology [19:49] hazmat: The goal of letting the agent continue normal operation [19:49] hazmat: Not the goal of resolving the problem.. that one must have been done already [19:50] hazmat: (out of band) [19:50] right [19:56] Is it just me or Launchpad fonts seem to be a bit less readable lately? [19:57] Setting the font color to black and increasing them a bit does wonders [19:58] * niemeyer <= getting old [19:59] <_mup_> ensemble/expose-provisioning r240 committed by jim.baker@canonical.com [19:59] <_mup_> Tests for watch_service_states [20:02] <_mup_> ensemble/unit-agent-resolved r269 committed by kapil.thangavelu@canonical.com [20:02] <_mup_> update unit tests to use relation workflow accessor on lifecycle. [20:03] niemeyer, the resolved branch is ready for review (fixed the accessor issue) [20:03] hazmat: Sweet, thanks [20:05] Y.all("body") [20:05] EWINDOW [20:15] <_mup_> ensemble/debug-hook-scope-guard r210 committed by kapil.thangavelu@canonical.com [20:15] <_mup_> merge trunk and resolve conflict. [20:20] niemeyer, regarding the relation resolved stuff, i realized that the retry flag argument to the unit relations doesn't have any real meaning, as it just places them back into an operational state, it doesn't rexecute a failed hook. [20:20] hazmat: Hmmm, good point [20:21] hazmat: Although, shouldn't it reexecute? [20:22] niemeyer, it doesn't at the moment, we don't save enough contextual information regarding the hook to do so, nor do we have a means to populate it artificailly in the context.. we could if we want to enable it but i think its probably future work for such.. for now i'd just raise an error if a unit rel with retry specified [20:23] hazmat: Sounds good [20:23] niemeyer, also regarding the debug-hook stuff, i've looked it and looked over the review, i tried out the screen user multiplexing but it would require suid, the sudo implementation works well [20:24] niemeyer, not sure that there's anything else to do for it.. [20:24] niemeyer, https://code.launchpad.net/~hazmat/ensemble/debug-hook-scope-guard/+merge/54433 [20:26] it will be nice to incorporate that functionality into other subcommands (deploy, add-unit, resolved --retry) with a --debug-hook option, also makes debugging install/start much easier. [20:28] hazmat: I'll check it out [20:32] bcsaller: I'm wondering a bit about the caching behavior of that get_config() method [20:32] bcsaller: Is this really what we want to do? [20:32] bcsaller: It means get_config() never updates the state read [20:32] bcsaller: I wonder if we should simply return a new YAMLState on every get_config() [20:32] bcsaller: How would that affect existing code? [20:33] hazmat: ^ You may have some input on this as well? [20:33] that was to provide a consistent view throughout hook execution [20:33] bcsaller: The consistent view is more than throughout hook execution.. it never updates [20:33] bcsaller: (remember, this is the ServiceState type) [20:33] but the lifetime of the service manager is scoped to the hook, if thats not the case then it should change [20:34] just stepping back in [20:34] my understanding is these managers are created as needed [20:34] bcsaller: So ServiceState.get_config() effectively returns the configuration of some arbitrary point in the past, when whoever first called it [20:35] so we want it to update, but be cached in the hook context for hook execution [20:35] if the service manager always returns a new one thats fine, then the hook needs to cache it [20:35] bcsaller: Creating a whole new manager to update the state of the config feels a bit like things are reversed [20:35] hazmat: Yeah, that's what it feels to me too [20:35] bcsaller: +1 [20:35] bcsaller: Otherwise it really becomes random.. [20:35] ServiceState.get_config should do no caching [20:35] simple change [20:35] cool [20:35] bcsaller: The hook itself can't know when it was first read [20:36] we don't have any caching in our state apis [20:36] bcsaller: Maybe someone else called get_config() for whatever reason, and the "snapshot" becomes ancient even before it should [20:36] minus the hook context [20:36] bcsaller: Please test that non-caching behavior [20:37] * hazmat wonders what to work on [20:37] bcsaller: +1, besides that [20:37] hazmat: Oh ho ho.. I has ideas.. [20:37] hazmat: ;-) [20:38] niemeyer, i thought you might, just looking over the milestone bugs atm [20:38] bcsaller: It feels like this branch is almost getting to a one-liner :-) [20:38] niemeyer, i'm probably going to have some more to do, after the reviews land... but i think i might manage another small task [20:39] hazmat: Yeah, a small task might require some further thinking [20:40] i've got a couple from the top of the milestone, cli beautification and bootstrap/shutdown with environmental awareness [20:40] niemeyer, did you have one in particular in mind? [20:40] hazmat: Nah.. we already talked about them.. they're too large for this week [20:41] hazmat: Well.. hmmm [20:41] yeah.. those require some thinking, i could at least tackle it for the simple case [20:41] hazmat: Here is an idea: implement an auto-dependency-resolution hack. [20:41] and leave the lifecycle.start nested case for post budapest [20:41] niemeyer, i get to HACK?! :-) [20:41] cool [20:42] hazmat: Yeah, like, totally simple and dirty [20:42] i'll see what i can do there [20:42] hazmat: Something specific for UDS, which we remove afterwards [20:42] niemeyer, sounds good [20:47] <_mup_> ensemble/config-state-manager r209 committed by bcsaller@gmail.com [20:47] <_mup_> get_config shouldn't cache inside service layer [21:14] <_mup_> ensemble/config-get r212 committed by bcsaller@gmail.com [21:14] <_mup_> added test verify cached copy is returned on get_config in hook [21:18] Hmmm [21:19] I have to run an errand.. will do that now that there's still sunlight and will be back to reviewing in 1.5h or so. [21:21] niemeyer, cheers [21:21] i'm gonna roll out in a few as well for 1.5hr [21:29] <_mup_> ensemble/expose-provisioning r241 committed by jim.baker@canonical.com [21:29] <_mup_> Tests for ServiceState.watch_service_units [21:42] hasta luego [22:02] That was faster than expected [23:19] <_mup_> ensemble/trunk r215 committed by bcsaller@gmail.com [23:19] <_mup_> Service state changes supporting configuration options [r=niemeyer] [f=746695] [23:19] <_mup_> Introduces get_config to ServiceState returning a YAMLState object. [23:23] [23:23] <_mup_> ensemble/expose-provisioning r242 committed by jim.baker@canonical.com [23:23] <_mup_> Test get_opened_ports, watch_opened_ports [23:25] backs [23:30] this auto dep stuff has one unforseen requirement.. i need to track the deps pairwise to do the relation work