[09:57] * hazmat yawns [10:09] <_mup_> Bug #842488 was filed: Enable cloud-init debug output to better support problem analysis < https://launchpad.net/bugs/842488 > [10:19] <_mup_> ensemble/stack-crack r332 committed by kapil.thangavelu@canonical.com [10:19] <_mup_> clean up bad merge from formula-provider-storage url integration [10:21] hmm.. one more openstack ec2 incompatiblity found.. only affects shutdown though [10:22] <_mup_> Bug #842497 was filed: Error on openstack env shutdown < https://launchpad.net/bugs/842497 > [10:25] hazmat: what's the distinction between a machine_id and a machine_state_id? [10:27] hazmat: because it seems to me that they're the same, except that the requirement that it be an int is never expressed with reference to machine_id === Hussain is now known as Guest46158 [14:02] SpamapS, i thiknm you're planning on getting a new ensemble into ubuntu, right? when you do i would really like to have bug 842488 in. [14:02] <_mup_> Bug #842488: Enable cloud-init debug output to better support problem analysis < https://launchpad.net/bugs/842488 > [14:02] if people are going to use those packages, then this will really help debug. [14:19] smoser: its trivial enough, I'll drop it in as a patch until it gets merged [14:28] hazmat: since the txaws MP's were approved, but I'm not on that particular team, can you merge both branches into txaws? [14:28] hazmat: I'll handle putting them in as a patch to the package [15:05] fwereade, re machine_state_id - i only see one reference to that in trunk, in ensemble.agents.provision [15:05] fwereade, in that case, it's certainly the same as machine_id [15:06] jimbaker`: heh, I didn't actually look for other references, I thought I'd already seen it elsewhere ;) [15:06] jimbaker`, cheers [15:08] fwereade, re being constrained to an int, the one place that appears to be the case is in terminate_machine, in the corresponding argparse. normally it's something like 0 or 42, but we often test it as a "machine-0" [15:09] we have tried not to overly constrain what these IDs are [15:09] jimbaker`: it seems to me that it's tighter than that, just a sec [15:10] jimbaker`, ensemble.state.machine, get_machine_state [15:10] jimbaker`, if it's not an int (or a str that can be inted) it will fail [15:11] fwereade, indeed that's the case [15:11] jimbaker`, so while many things can still work if it's not an int, it feels kinda wrong to throw random data around elsewhere [15:12] if, say, bootstrap were to create a machine called "bootstrap-0", it would indeed start a machine, but the rest of the system wouldn't be able to use it properly [15:12] fwereade, yeah, i'm not certain what distinction is intended here [15:13] jimbaker`, it's the sort of thing that makes me want to just tighten things up a little by asserting that the machine-id field in machine_data actually is an int when we call start_machine [15:13] the tests imply you can pass anything :) [15:14] fwereade, even if you were to remove this constraint? i have never asked hazmat about it, but the tests as we have discussed do use strings preferentially [15:15] jimbaker`, I'm not sure I want to remove any constraints, just make the ones we already have (but I only just discovered) explicit [15:15] fwereade, the external machine id generation is just a (convenient) artifact of the internal ZK representation [15:15] jimbaker`, your point being that the constraint is itself unnecessary? [15:16] fwereade, it's quite possible this constraint is rather old [15:16] and we have never exercised it [15:16] it's certainly the intent expressed in a lot of tests that the machine id could in fact be something besides an int [15:17] jimbaker`, hmm, makes sense [15:25] fwereade, fwiw, hazmat is credited with that code (or at least touching those lines) in bzr blame. but my bzr fu is currently too weak to know how to map it to the rev # on trunk [15:26] jimbaker`, I'm just going to leave an XXX there for now, I don't want to make any logic changes while I'm documenting [15:26] fwereade, sounds like a good plan [15:27] fwereade, btw, are you doing the doc changes to use autodoc in one big branch? [15:28] jimbaker`, I'm stacking them [15:28] jimbaker`, I'm finding the documenting quite hard going tbh [15:28] jimbaker`, but it's teaching me a lot about the bits ofthe code I don't know [15:28] jimbaker`, but if I tried to do everything at once I think I'd go insane [15:30] fwereade, good to split up. it would make reviewers insane too. just don't know if you should be the one doing all of these fixes as well [15:34] fwereade, also, is it necessary to fully specify the full path in the rst? my understanding is that the full path is only necessary for resolving ambiguities [15:34] eg :class:`ensemble.machine.ProviderMachine` vs :class:`ProviderMachine`. i guess it's worth experimenting with [15:43] fwereade, regrettably it doesn't seem able to do the xref unless the path is explicit [15:44] jimbaker`, indeed, I resisted the temptation to start trying to hack up sphinx to make it resolve non-ambiguous references ;) [15:45] jimbaker`, and, well, it's teaching me a lot, but to be fair I'm not that likely to finish everything [15:45] fwereade, well it does seem some sort of hacking up is required, as seen in generate_modules.py [15:45] jimbaker` that's distinct from hacking sphinx itself ;) [15:46] fwereade, indeed, there is that distinction [15:46] jimbaker`, anyway, I have something else to do that's oddly uninspiring, and docs feel like a good thing to work on while I try to figure out what's bothering me in the other stuff [15:47] jimbaker`, I'm not committing to documenting everything, but I can make things a little better and teach myself stuff in the process :) [15:48] fwereade, indeed, it's a very good way to learn the codebase by having to do some sort of explicit/formal pass over it, vs just reading code [15:54] Hey folks [15:54] niemeyer, hi [15:54] I'm heading out to lunch.. will be biab [15:55] niemeyer, enjoy! [15:55] lynxman: any word on macports? [16:01] jcastro: nope :( https://www.macports.org/ports.php?by=name&substr=ensemble [16:01] so is there like someone we can ping? [16:04] jcastro, lynxman: what abpout homebrew? [16:04] m_3: if you get the port it works fine :) [16:05] jcastro: I'll try to ping them tomorrow in the irc channel, I'm supposedly off today (why am I on irc? I don't know) [16:05] heh, ok [16:12] jcastro: thanks for the syndication! [16:12] we're missing some posts though [16:13] yeah the plugin got upgraded [16:13] Juan's got a couple in that same timeperiod that've gone missing [16:13] membase [16:13] and hpcc [16:13] ok [16:15] lemme know if we should do something different with the formatting to make it easier [16:15] thanks man! [16:15] ok I think I see the problem [16:15] you need a "planet" tag AND the "featured" tag [16:15] so using both those tags should autosyndicate you guys [16:15] negronjl: ^ [16:16] ok, I'll add them on future posts [16:16] jcastro: should I retroactively do it to old ones? [16:16] I've been adding them by hand in the syndication thing, I guess that would make more sense [16:17] jcastro: I'll add them to all of mine ... let's see what happens then [16:17] k [16:17] hey so m_3 [16:17] me2 [16:18] about that node.js "easy" article I linked you to [16:18] the one on HN. [16:18] I'm working on a post now [16:18] I was thinking a nice response on how we're making all that easier [16:18] and then when we pingback the guy I can send him a note [16:18] and then he can mention it on the original article, that would be win [16:19] yup! [16:19] hazmat: pong [16:19] one of the comments is by a friend [16:19] small world [16:19] oh awesome [16:25] jcastro: posts tagged [17:22] * niemeyer waves [17:23] fwereade: ping [17:23] niemeyer: pong [17:23] fwereade: Heya! [17:23] niemeyer: hey :) how's it going? [17:23] fwereade: Nicer today :) [17:24] niemeyer: jolly good [17:24] niemeyer: as you say, yay, modern medicine :) [17:24] fwereade: How's the Orchestra stuff going? [17:25] niemeyer: I've been finding it oddly tricky to redo the put-auth stuff [17:26] fwereade: Hmm.. how so? [17:26] niemeyer: I ended up doing docs and poking at go today, I'm hoping whatever's bothering me will sort itself out by tomorrow [17:26] niemeyer: whenever I start, it feels like I'm reinventing the wheel [17:27] fwereade: Ok [17:27] niemeyer: HTTPClient doesn't seem to be quite right -- I need at least a couple of distinct classes that differ slightly from the ones they've provided [17:27] niemeyer: really I just need to force myself to finish it off, and it'll probably turn out to be fine [17:28] fwereade: Cool.. I'm hoping to have a hand from you on the store stuff [17:28] niemeyer: cool, that sounds like fun :) [17:28] fwereade: But for that we need to have the Orchestra fully closed [17:29] niemeyer: well, I'll polish that auth stuff off as soon as I can and hassle RoAkSoAx for any more bugs he can think of ;) [17:30] fwereade: Awesome [17:30] fwereade: Once you've finished with that, please ping me so we can synchronize on the store [17:31] niemeyer: cool, will do :D [17:31] niemeyer: hopefully tomorrow :) [17:33] Awesome! [17:47] fwereade, hazmat introduced that change in get_machine_state in jan 13 as part of the misc-demo-fixes branch. but it seems more to be able to parse internal ids [17:48] SpamapS, yes re merge, post plumbers [17:49] bus time.. bbiab [17:53] fwereade, jimbaker` the machine i corresponds to the zk sequence value of the machine state node [17:53] s/i/id [17:53] hazmat, correct [17:53] jimbaker`, trying to back track through the conversation, not sure what the question was [17:53] hazmat, fwereade was simply bringing up this restriction as part of his doc work [17:54] ah [17:54] hazmat, in many tests, machine id is often something like "machine-0" [17:54] but in a few places, like get_machine_state, we explicitly require it to be an integer, or something that has an integer portion [17:55] so "machine-0" would in fact qualify in that case ;) [17:55] yeah.. in retrospect, it would hav ebeen nice to have those namespaced to something more blantaly obvious.. ssh 0 .. vs ssh machine/0 [17:55] hotspot on a bus .. rocks [17:55] hazmat, i always liked how aws does it, i-abcdef [17:56] and of course we do that in the topology tests, s-0 or m-0 [17:57] jimbaker`, at this point are namespace precedent is "qualifier/id" for service units, which is pretty nice overall i think... definitely true that tests violate those blatantly where they can [17:57] hazmat, all vehicles should be a hotspot. i definitely enjoy my verizon router for that aspect [17:58] i was able to get the local dev provider bootstrapped on a plane [17:58] hazmat, and maybe the tests should do just that [17:59] SpamapS, is there a way to prime a debcache dir with additional packages.. i really want to have the ability to do completely offline demos, just trying to understand how to go about it.. afaics, i would also need to setup a file apt repo accessible the containers for formula pkg installs [18:05] <_mup_> ensemble/local-ubuntu-provider r348 committed by kapil.thangavelu@canonical.com [18:05] <_mup_> work around path issues when launching machine agent, better multi bootstrap/shutdown handling, incorporate unpack formula into start unit, fix up admin identity into hierarchy initialization, make unitdeploy.is-running return a deferred [18:07] bcsaller, so i got about as far as creating the lxc container in response to ensemble deploy with the latest on the above branch, still lots of work, but i'm optimistic.. [18:07] hazmat: I am as well [18:07] hazmat: where are you now? [18:08] bcsaller, i'm wondering if we can do away with the ensemble-branch/origin stuff and try to always deploy into the container the version running the cli, via copying the source tree into the container.. maybe too aggressively implicit. [18:08] bcsaller, on a bus just left sfo airport about 10m ago.. heading north on the 1 [18:08] hazmat: if the right .deb is in /var/cache/apt/archives it will not be downloaded [18:08] hazmat: for local dev that might work, but the origin stuff would still be needed on EC2 I'd think [18:09] /var/cache/apt/archive actually [18:09] bus w/wifi, nice [18:09] bcsaller, personal t-mobile hotspot.. works just as well :-) [18:09] hazmat: oh debootstrap.. hm [18:10] bcsaller, its already outside the container on the host... its just a bit of an ugly issue when using multiple branches against the same env.. different deploys with different unit code.. and bad from an upgrade perspective.. probably makes a debian maintainer cry somewhere ;-) [18:10] SpamapS, excellent thanks [18:11] SpamapS, i think the /var/cache/apt/archives priming should be able to do it.. the debcache thing isn't going to help per se, if we can't pass the additional packages to be installed when doing the debootstrap [18:11] hazmat: you can also just build your own mirror w/ reprepro [18:22] SpamapS, that looks great, i was figuring we could do something like a shared local package cache onto a block dev, we overlayfs into the containers [18:23] hazmat: lxc-clone perhaps? ;) [18:23] SpamapS, well i don't really want to clone the entire container, i just want a primed package cache for any given container [18:24] hazmat: the default container builder only builds the mini-buntu once. [18:24] SpamapS, i know.. but it would be nice to extend that to common packages we might install.. i mean if i have 20 units of wordpress... i really don't need to download apache/php/wordpress times [18:25] IMO there's a missing hook command which is the "I'm ready to be turned into an image" which is executed whenever any common operations (like package installs) are done. [18:31] SpamapS, perhaps.. its a little tricky... an image implies a static base point, but from the moment of deploy a formula may be launched with different configs.. it might work within a service, for a quick new unit, on the same machine... [18:32] hazmat: it should work within every service if the formula author identifies the point at which they have done no more unique configuration. [18:32] so during 'install' you say 'ready-for-imaging' right before you start services or record hostnames or anything like that. [18:32] bcsaller, we totally have to hit avatar's again... life changing [18:32] Then the provisioning agent can snapshot your EBS, make an image of it, and every add-unit to that service is super fast. [18:34] SpamapS, good point.. assuming volume management.. which we'll probably need to tackle next cycle [18:34] if there's no volume management, then the 'ready-for-imaging' command could simply do the bundling itself. [18:35] Thinking in terms of real use cases vs. playing w/ ensemble.. the faster add-unit is, the more useful it is to people... they won't mind the first deploy being 10 minutes if every add-unit after that is 40 seconds. [18:35] SpamapS, so there are faster tricks we can do to get there [18:36] er.. not faster in terms of clock time, but in terms of dev time [18:37] i figured out the easiest way to preallocate machines just using the existing infrastructure, is to have our bootstrap setup the zk tree with however many nodes we want, and they'll be created as standby nodes for new deploys after the first deploy [18:37] anyway, to your point about containers.. I think lxc-clone is what you want [18:37] hazmat: yeah that would be huge.. to be able to decouple machine creation from deploy/add-unit [18:38] SpamapS, its basically in the vision of a min/max environment config [18:38] where you spec min/max machines you want to pay for [18:38] SpamapS, its starts to fall down because we're doing dynamic port management [18:39] so we can't ever do placement to avoid conflicts, since we have no static analysis capabilities upfront at formula deploy time [18:39] i don't really see the need for dynamic port management.. but otoh.. getting true density is going to need some network trickery anyways [18:42] okay folks....gotta fire in some nearby woods...railroad tracks are a good barrier between the woods and my neighborhood, but I need to go afk for a bit until I know it's safe [18:43] robbiew, good luck and be safe [18:43] robbiew: Ugh :( [18:44] Austin dealing with the same type of fires we had 3 years ago. Nasty stuff. [18:44] SpamapS, re containers, i'm pretty sure lxc-clone isn't what i want... i really just want a local deb repo... for offline ensemble env deploys, the reprepo looks spot on for it, thanks [18:44] robbiew: be safe [18:45] hazmat: ok, cool. :) [18:45] * hazmat goes back to working on presentation [19:20] It surprises me a bit that we have no required/optional flag in the config [19:21] agreed [19:22] there will be plenty of services with no sane default config.. they should error at deploy. [19:23] alright rev 336 uploaded to ubuntu [19:24] SpamapS: Hmm [19:24] SpamapS: That was probably the rationale for the choice [19:25] fire contained....all is good [19:25] SpamapS: The service should always have a sane default [19:25] :) [19:25] robbiew: Phew [19:25] niemeyer: there's no sane default title for a blog. :) [19:25] "Abandoned Server" maybe [19:25] SpamapS: Indeed, but it's fine to define it afterwards [19:25] "TELL MY OWNER TO CHANGE ME" [19:25] SpamapS: Exactly! ;-) [19:26] niemeyer: indeed...was bit dangerous for awhile, but the fire chief lives in our neighborhood...so I'm thinking that helped get resources here quickly :D [19:27] niemeyer: we run into this all the time in packaging.. some services need deep configuration before being started. [19:27] robbiew: As some say, it's important to know everything, but just to have the phone number of someone who does ;) [19:27] s/it's im/it's not im/ [19:27] mysql you can really shoot yourself in the foot if you don't tune it first [19:27] postgres too [19:27] because the initial database creation can change the way the thing works. [19:28] SpamapS: Sure, and that's where Ensemble gets in :) [19:28] I guess my point is, deploying it w/o configuration would result in something that can't be related to things [19:28] SpamapS: Hmm.. I don't get it.. [19:29] SpamapS: Why would we build a formula that can't be related to things? [19:29] There are defaults, but they get you in trouble. [19:29] SpamapS: Sounds like we should change them? [19:29] niemeyer: if you don't turn on binary logging in mysql.. you can't relate to slaves. [19:29] but if you turn it on, you double your write load [19:29] which you may not want to do if you have chosen drbd as your HA method [19:30] SpamapS: Sounds like an easy choice to make.. enable by default.. we don't support drbd [19:30] SpamapS: and offer a flag for people to tune [19:30] SpamapS: Those who want something else, can have it [19:30] What if you do your replication w/ rabbitmq [19:30] meh [19:30] Its not critical [19:30] SpamapS: :-) [19:31] I'm just saying that often times this is what annoys sysadmins about packaging.. that the packagers make too many decisions for the user. [19:31] In fact [19:31] what we need isn't optional/required [19:31] but priority [19:32] But thats later, when we have configurators and not just config.yaml .. n/m [19:48] <_mup_> ensemble/rename-shutdown-command r340 committed by jim.baker@canonical.com [19:48] <_mup_> Merged trunk [20:01] niemeyer, re config required/optional.. how about runtime relation declarations [20:01] robbiew, awesome, looks quite serious from a distance [20:02] <_mup_> ensemble/rename-shutdown-command r341 committed by jim.baker@canonical.com [20:02] <_mup_> Fixed review items [20:02] we just mod the formula state serialization directly to add the additonal rel endpoints [20:03] hazmat: How's one related to the other? [20:04] good question :-) [20:04] :) [20:04] there both relating to runtime configuration of a service [20:05] hazmat: Good try ;-) [20:05] say i have a generic wsgi/rails/node deploy formula.. pulls from vc... etc. based on configuration.. figures out what the particular service dependencies are for this app and then updates the relation declarations [20:06] atm, i have to declare the entire world of possibilties as optional relations [20:06] hazmat: Yeah, but there are some important reasons for that [20:07] hazmat: I know what you mean, but we'll have to dig deeper to see how to model that kind of relationship [20:07] hazmat: It's not sensible to simply dynamically show up with new relationships [20:07] hazmat: The admin would be surprised, the formula itself would become weird (how can a new relation be introduced if there are no files expecting for it), etc [20:08] <_mup_> ensemble/trunk r339 committed by jim.baker@canonical.com [20:08] <_mup_> Merged rename-shutdown-command branch [r=niemeyer,fwereade][f=838215] [20:08] <_mup_> Renames shutdown command to destroy-environment. [20:09] niemeyer, dynamic relations seem more 'sensible' to me then dynamic ports... everything is a tradeoff.. dynamic over static looses analysis, i mean the admin is configuring the formula to point to a particular platform app.. [20:10] hazmat: Awesome.. we've managed to talk about dynamic ports and required/optional config settings in the context of dynamic relations :-) [20:10] :-) [20:10] i should eat my lunch while its warm [20:10] hazmat: Enjoy! [20:46] bcsaller: Is it the case that all of the inputs to config.validate are string=>string? [20:46] bcsaller: If you remember that from the top of your head at all.. I can look otherwise [20:53] niemeyer: I thought conversions came out of the validate process [20:53] bcsaller: Yeah, that's what I think as well.. cool [21:39] <_mup_> ensemble/go-formulas r16 committed by gustavo@niemeyer.net [21:39] <_mup_> Added config validation support. [21:43] <_mup_> Bug #843299 was filed: Formula config in Go port needs validation support < https://launchpad.net/bugs/843299 >