[00:23] <Lord_Set2> Hello
[00:30] <bigjools> hi
[01:03] <Lord_Set> How are things going bihjools?
[01:20] <bigjools> busy busy busy
[01:22] <Lord_Set> Any update on the MAAS issue with enlistment? Or do you have an eta?
[01:22] <bigjools> I'm in the middle of a fix
[01:22] <bigjools> just testing something out
[01:22] <Lord_Set> Awesome you rock
[01:22] <Lord_Set> If you want me to do some testing as well let me know
[01:30] <bigjools> yup
[01:55] <bigjools> Lord_Set: try this package https://launchpad.net/~maas-maintainers/+archive/experimental/+build/5800532
[01:55] <bigjools> it's got a change that's not landed in trunk yet so is highly experimental but seems to work for me locally
[02:09] <mwhudson> if i was to try to set up a MAAS that had a bunch of real arm server nodes and virtual machines enlisted, could i make that work?
[02:09] <mwhudson> or would i just go insane and fall into the sea?
[02:10] <bigjools> mwhudson: no reason why it won't work
[02:10] <mwhudson> cool
[02:10] <bigjools> there's a section in the docs for setting up VMs
[02:10] <mwhudson> it was more the highly heterogeneous nature that worried me
[02:11] <bigjools> maas doesn't care what it is
[02:11] <bigjools> it's an equal opportunities provisioner
[02:11] <mwhudson> :)
[02:11] <Lord_Set> Bigjools: How much testing have you done with ARM machines such as Raspberry Pi? Or some of it's more powerful cousins out there?
[02:12] <mwhudson> i've run it on highbank before now
[02:12] <bigjools> well there just happens to be the right man here :)
[02:12] <mwhudson> Lord_Set: you need remote, well, management of some kind
[02:13] <mwhudson> i guess you could have rpi and a pdu
[02:13] <Lord_Set> The team here came up with a great concept we want to build using arm machines for super high density... like 30+ machines per 2U of space.
[02:13] <mwhudson> Lord_Set: where is "here"?
[02:13] <Lord_Set> I am the lead network and systems engineer of a few "connected" startups.
[02:14] <Lord_Set> Same owner and CEO/CIO started a bunch of companies that each complement each other infrastructure wise.
[02:15] <Lord_Set> We are using Ubuntu Server, MAAS, and Juju as the core of our infrastructure for deployment over geographically disjoint locations.
[02:16] <Lord_Set> We are pushing a handful of our services and platform live this week.
[02:18] <mwhudson> cool :)
[02:19] <mwhudson> have you seen hp's moonshot?
[05:37] <jtv> bigjools: I didn't notice earlier because my connection to the board was broken, but you're doing "Hook power_types registry to rpc call" while I'm doing "Registry for power types."
[05:37] <bigjools> jtv: argh
[05:38] <jtv> I expect quite a bit of overlap.
[05:38] <bigjools> I am doing that too
[05:38] <bigjools> currently fighting tests
[05:38] <bigjools> registry bit is easy
[05:38] <bigjools> but I am having trouble stopping the test data leaking out of the test in the singletons
[05:38] <jtv> The way I did it, JSON_POWER_TYPE_PARAMETERS contained only the fields.
[05:39] <jtv> One thing we can always do is start out by patching each registry to contain a copy of its original contents.
[05:40] <bigjools> that's what I am doing
[05:40] <bigjools> hang on let me show you a diff
[05:40] <bigjools> jtv: https://code.launchpad.net/~julian-edwards/maas/power-registry/+merge/210343
[05:42] <bigjools> jtv: the Mocks added in the tests leak out so I am not restoring state properly
[05:42] <bigjools> I can see why
[05:42] <bigjools> just need to fix it and then it's done.  how far did you get?
[05:44] <jtv> Wrote and initialised the registry, pared down JSON_POWER_TYPE_PARAMETERS to list just the fields, made the RPC function compose its answer based on the registry, added a module of functions-to-patch-out-RPC-calls, and was fixing up tests because the global data structure that the MAASServerTestCase patch depended on no longer existed.
[05:45] <bigjools> mine is done
[05:45] <bigjools> ummm
[05:45] <bigjools> I didn't remove or change JSON_POWER_TYPE_PARAMETERS
[05:45] <bigjools> it can stay as it is, surely?
[05:46] <jtv> Can, but probably shouldn't.
[05:46] <bigjools> why?
[05:47] <jtv> I didn't find any good place to put the whole thing — it was too much for the driver's __init__.py, even though it's really just another "built-in" list; and power_schema.py should be about the JSON schema, not the core definition of built-in power types.
[05:48] <jtv> So I figured power_schema.py should know the fields, but the names/descriptions belonged in a "builtin_power_types" list in driver.
[05:48] <bigjools> let's move it in a separate branch - since mine is done and working shall we use it and then yours can be a bout moving the struct around?
[05:48] <jtv> Sure.  I'm already doing part of it in a separate branch.
[05:48] <jtv> By the way, can't you just use self.patch to override the registry during testing?
[05:48] <bigjools> ok feel free to review that MP
[05:48] <bigjools> no,  because it's a global
[05:49] <bigjools> you have to modify its content
[05:49] <bigjools> not its reference
[05:49] <jtv> Yes, but patching it would let you do either, right?
[05:49] <jtv> "Replace with a copy, and restore after the test."
[05:50] <bigjools> it didn't work with patch
[05:50] <bigjools> that's all I know
[05:50] <bigjools> and I thnk it's because the registry classes keep a reference to the old object
[05:51] <bigjools> and patch just replaces the reference with a new one but doesn't help with things that already took a reference
[05:51] <jtv> Right, I was thinking more along the lines of patching the classes.
[05:51] <bigjools> you can't do that
[05:51] <bigjools> they don't have anything to patch
[05:51] <jtv> Yes you can.  Python.  :)
[05:51] <bigjools> you have to patch the global
[05:52] <jtv> Yes, but what if you replaced the double "global _registry" with a single class variable?
[05:53] <bigjools> just so we can use patch?
[05:53] <bigjools> not really worth it IMO
[05:53] <jtv> OK
[05:53] <bigjools> I mean, feel free to disagree :)
[05:53] <jtv> Don't want to slow this down with bikeshedding.
[05:54] <bigjools> good man
[05:54] <jtv> Lies.
[05:54] <jtv> One thing that bothered me a bit about the Registry class is that entries are not keyed on names.
[05:57] <jtv> It's probably nothing, but I do wonder if anything could go wrong with poorly written drivers, e.g. ones that might re-register new versions of what was meant to be the same object on every call or something.
[05:58] <bigjools> hopefully that will be weeded out in tests
[06:00] <jtv> It's the sort of thing that's hard to test well though.  And on the other end, I guess one thing we'll be doing a lot is look up items by name.
[06:00] <jtv> So a dict keyed on name would seem to make sense.
[06:02] <bigjools> well the registries are just lists
[06:02] <bigjools> I don;t mind making them dicts
[06:03] <jtv> I think we'd find it more natural in the end.
[06:03] <bigjools> ok let's do a separate change for that
[06:03] <jtv> OK.
[06:03] <jtv> I'm reviewing your branch.
[06:03] <bigjools> cheers
[06:04] <jtv> There's something in TestRegistry.setUp that I don't understand: how does _registry.update(...) delete items that have been added during tests?
[06:04] <jtv> Ah, I see now.
[06:05] <bigjools> it turns out using Mock as a dummy resource in the registries was a great thing - it weeded out leakage
[06:05] <jtv> It's because the tests never actually add any keys to that dict.
[06:05] <bigjools> Mock can't be serialised in the rpc
[06:05] <jtv> Ah good.
[06:05] <bigjools> argh let me fix that
[06:05] <bigjools> just needs a clear() first
[06:05] <jtv> That'd do it.
[06:05] <jtv> Also, may I recommend dict.copy() or perhaps even deepcopy()?
[06:06] <jtv> Because items() no longer returns a list in python 3.
[06:06]  * jtv watches diff update
[06:06] <bigjools> pushed
[06:06] <bigjools> oh
[06:06] <bigjools> ok I'll change it again...
[06:06] <bigjools> deepcopy is needed I think
[06:06] <bigjools> mmm or is it
[06:07] <jtv> Would be more thorough.
[06:07] <jtv> And this is not the test-case base class or anything, so no need to worry about the added cost.
[06:07] <bigjools> pushed again with a copy(), it will be enough just to keep a reference
[06:08] <jtv> Come on, Launchpad, it's been seconds!  Where is my update?
[06:08] <jtv> Ah, there it is.
[06:08] <bigjools> shame not many people get to see the longpoll
[06:08] <jtv> With the added clear() in tearDown, it's finally worth having a tearDown in the first place.  :)
[06:09] <bigjools> aaye
[06:09] <bigjools> tests all pass anyway :)
[06:09] <jtv> I mean, compared to just adding a cleanup in setUp.
[06:10] <jtv> Approved with comments.  :-P
[06:10] <bigjools> yes
[06:10] <bigjools> naturally
[06:10] <bigjools> jtv: what is wrong with my commit message!
[06:11] <jtv> !?  Just now I saw the green link with "Set a commit message."
[06:11] <jtv> Shut up, bot.
[06:11] <bigjools> I prob set it after you loaded the page
[06:11] <jtv> Ah, I guess that part doesn't refresh.
[06:11] <jtv> More longpoll!
[06:12] <bigjools> you missed two bird of paradise flowers coming out by hours...
[06:15] <Lord_Set> So question. How far out would a feature of MAAS to be able set static dhcp leases?
[06:15] <bigjools> Lord_Set: it already has that
[06:16] <Lord_Set> Really? It is a CLI function or only possible by manually adding servers?
[06:17] <bigjools> as soon as maas sees the lease, it writes a host{} map to the config
[06:17] <jtv> You can't control the leases though.
[06:17] <Lord_Set> Hmm interesting. Ive had leases drop more than once and IPs change on me.
[06:17] <jtv> Except perhaps by editing the config...  I don't recall whether we have proper support for that.
[06:17] <jtv> IPs should normally only change if either:
[06:17] <jtv> 1. You run out of addresses in the DHCP range, or
[06:17] <Lord_Set> What is the default lease time?
[06:18] <jtv> 2. The second DHCP request looks like it comes from a different machine.
[06:18] <jtv> The leases last forever.
[06:18] <Lord_Set> Interesting. Once I redeploy MAAS after the current issues have been fixed I will do some in depth testing with it.
[06:27] <bigjools> jtv: can you remember where the simplestreams index lives?
[06:29] <jtv> bigjools: not off the top of my head, no.
[06:29]  * jtv digs in scripts
[06:30] <bigjools> I see it
[06:30] <jtv> OK
[06:31] <bigjools> thanks, you reminded me it's in mie
[06:33] <bigjools> jtv: BootImage has a have_image() which is only called from tests AFAICS
[06:34] <jtv> Yup, looks like.
[06:35] <bigjools> I'm adding label to the BootImage model
[06:41] <bigjools> also I would love it if we could fix the "ctrl-c only pretends to stop tests" thing :(
[06:56] <lifeless> jtv: addCleanup is way better than teardown, if your test classes have it :>
[06:56] <jtv> Hi lifeless.  Yes, but it gets a bit uglier with multiple consecutive statements.
[06:58] <lifeless> jtv: how so?
[07:00] <jtv> If you need to execute multiple consecutive statements in the cleanup, you must either define a function anyway (reducing the advantage over writing a tearDown) or add them separately and worry about the ordering.
[07:02] <lifeless> jtv: by ordering, you mean the LIFO nature ?
[07:03] <jtv> Yes.
[07:06] <lifeless> interesting feedback
[07:07] <lifeless> most folk I know just addCleanup adjacent or before the thing that needs cleanup
[07:07] <lifeless> so its very contextual and needs little thought
[07:08] <bigjools> lifeless: : it's normally fine indeed, but gets unwieldy if you start needing more than a basic lambda
[07:09] <lifeless> bigjools: can you link me an example of such?
[07:09] <bigjools> I can't
[07:10] <lifeless> heh, np - I'm just interested in seeing places it falls down (so I can consider how to make it better)
[07:10] <bigjools> but one sec
[07:11] <bigjools> lifeless: https://code.launchpad.net/~julian-edwards/maas/power-registry/+merge/210343 line 61 of the diff
[07:12] <jtv> I was going to say "use addCleanup instead," and then we found that it needed an additional statement.
[07:13] <bigjools> exachary
[07:15] <lifeless> bigjools: isn't that just
[07:16] <lifeless> self.addCleanup(_registry.update, self.saved_registry)
[07:16] <lifeless> self.addCleanup(_registry.clear)
[07:16] <lifeless> ?
[07:16] <lifeless> given that tearDown may fail to run at all...
[07:16] <bigjools> yes but as I said it's unwieldy
[07:16] <bigjools> and depends on pop ordering, so reads weirder!
[07:17] <bigjools> if I could add everything in a single call it would look nicer
[07:17] <lifeless> a list in FILO order it would reverse for you ?
[07:18] <bigjools> mebbe, yeah.  a list of calls
[07:18] <bigjools> when does tearDown fail to run?
[07:19] <lifeless> if setUp fails for any reason
[07:20] <lifeless> (and also tearDown doesn't run the remaineder if part of tearDown fails)
[07:29] <lifeless> so, def addCleanups(self, cleanups_list): for cleanup, in reversed(cleanups_list): self.addCleanup(cleanup etc
[07:30] <jtv> Would the list really be easier to read than a function though?
[07:30] <jtv> Simple enough if the calls take no arguments, but otherwise...
[07:30] <lifeless> jtv: it would have the property of running every element in it
[07:30] <lifeless> jtv: which is equivalent to deeply nested try:finally: in a function
[07:31] <lifeless> jtv: which you need for correctness in most end of test code, but is often skipped
[07:32] <bigjools> lifeless: I would need to think about what reads the clearest I think
[07:33] <bigjools> jtv: dammit, will need to implement tftp path changes for the boot images already because of tests, those damn tests....!
[07:33]  * bigjools eats
[08:14] <jtv> Does it make sense to hook the boot-images registry to RPC at this stage?  Given that we already have an API for this, I mean.
[08:23] <jtv> Wouldn't it make more sense to slip the driver API in underneath the existing metadataserver API?
[08:37] <Lord_Set> What is the status of the current MAAS build?
[08:39] <bigjools> it ought to work
[08:39] <Lord_Set> Awesome
[08:40] <Lord_Set> Will give it a try in my test racks here
[08:40] <bigjools> jtv: I am only putting the "label" part in the path
[08:42] <Lord_Set> Have you guys seen an issue where trusty will sometimes freeze during boot while mounting the swap?
[08:42] <Lord_Set> It randomly happens on all my servers
[09:13] <bigjools> no, you need to ask in #ubuntu-server about that
[09:52] <melmoth> hola maas people ! can someone confirm that /var/log/maas/rsyslog/* logs fils are only filled when the nodes are being installed. And untouched after installation ?
[09:52] <melmoth> (well, confirm or correct :-) )
[11:53] <rbasak> melmoth: AIUI, after installation you're in control via whatever stack you have deployed. Eg. use the rsyslog charm.
[11:53] <rbasak> melmoth: at that point you're in juju territory really, rather than maas.
[11:53] <melmoth> ok, the customer is using rsyslog with rsyslog-forxarder charm
[11:53] <rbasak> (same with your question the other day about relations)
[11:54] <melmoth> he just asked confirmation those file are only generated during installation, now i can confirm.Thanks
[11:54] <melmoth> rbasak the relation issue is escalated, there s is something obviously broken and i dont undertand what
[11:54] <rbasak> Well, that's to the best of my knowledge. I'm not really an expert in this area.
[18:52] <tych0> hi bigjools, did you get a chance to look at that lxc power thing yesterday?
[18:56] <kentb> is /etc/maas/preseeds/enlist the place to override and enlisting / commissioning node from using archive.ubuntu.com during the enlist/commission phase?  I'd like to use my local mirror instead.
[22:24] <bigjools> tych0: sorry no I got sidetracked
[22:24] <tych0> bigjools: no worries
[22:25] <bigjools> and why on earth is kentb trying to hack preseeds to set the archive proxy
[22:25] <bigjools> tych0: looking now
[22:27] <tych0> because he wants to use his local mirror instaed :-)
[22:27] <bigjools> well there is a setting for this in the config page
[22:28] <bigjools> tych0: ok so this is your change? http://bazaar.launchpad.net/~tycho-s/maas/lxc-power/revision/2040#etc/maas/templates/power/lxc.template
[22:28] <tych0> bigjools: yeah
[22:29] <tych0> well, there are a few
[22:29] <tych0> that whole branch's diff
[22:29] <bigjools> well the template
[22:29] <bigjools> what was getting quoted?
[22:29] <tych0> {{preseed}}
[22:29] <tych0> ends up as
[22:29] <tych0> '#cloud-config ...'
[22:29] <tych0> instead of just #cloud-config
[22:29] <bigjools> it's probably the code
[22:30] <bigjools> tempita won;t do that
[22:30] <tych0> oh
[22:30] <bigjools> it might be using a repr somewhere perhaps
[22:30] <tych0> so you think whatever get_preseed returns is bad?
[22:30] <tych0> ah
[22:30] <tych0> or that
[22:30] <bigjools> it's possible
[22:30] <bigjools> stick a breakpoint in
[22:30] <tych0> i'm not calling repr anywhere
[22:31] <bigjools> or hack some code to write its contents to a /tmp/ file
[22:31] <tych0> well, the other problem is that the maas dev environment didn't work
[22:31] <tych0> i got some big logs
[22:31] <bigjools> yay
[22:31] <bigjools> it works fine here, what are you doing? :)
[22:31] <tych0> http://paste.ubuntu.com/7050909/
[22:31] <tych0> and stuff when i ran it
[22:31] <tych0> just via 'make run'
[22:32] <bigjools> did you make syncdb?
[22:32] <tych0> i can't remember, so probably not
[22:32] <bigjools> also, write a test to check your preseed
[22:32] <bigjools> well, template output
[22:32] <tych0> yeah, i just wanted to get it to work first :-)
[22:32] <bigjools> tsk
[22:32] <bigjools> tests first
[22:33] <bigjools> honestly, it's quicker if you write tests as you write the code
[22:33] <tych0> fair enough
[22:33] <bigjools> TDD :)
[22:34] <tych0> yeah, i mostly just have used languages with compilers in the past
[22:34] <tych0> i know that makes me a heretic
[22:34]  * tych0 ducks
[22:34] <tych0> anyway, i'm off for a quick break before sprint dinner
[22:35] <tych0> i promise i will write some tests before i make an MP :-)
[22:35] <bigjools> compilers can deal with tests too :)
[22:36]  * bigjools did C++ for more years than I care to recall
[22:42] <roaksoax_> bigjools: howdy! It is not possible to specify user data to the MAASDataSource to execute custome stuff post install..is it?
[22:42] <bigjools> no idea off hand
[22:43] <bigjools> that's a cloud-init function isn;t it?
[22:43] <roaksoax_> bigjools: yes and now. Cloud-init gets user-data from the MAAS data source and executes. But if I wanted to execute custom stuff, we really can't
[22:43] <bigjools> oh so you want to put more stuff in?
[22:44] <roaksoax_> bigjools: yep
[22:44] <bigjools> ok I don;t know without looking
[22:44] <bigjools> let me check quickly
[22:44] <roaksoax_> cool, thjanks!
[22:45] <bigjools> I *thought* that this was all passed through on the API request to start a machine
[22:45] <bigjools> it's what juju does
[22:46] <bigjools> yeah it's a parametert user_data for Node.start()
[22:46] <roaksoax_> bigjools: cool, so I can technically tell maas to do it via the api
[22:47] <bigjools> yes just not in the UI
[22:47] <bigjools> web UI
[22:49] <roaksoax_> bigjools: yeah, and has to be base64
[22:49] <roaksoax_> ok cool thanks!
[22:49] <bigjools> correct