| jam | morning all | 05:24 |
|---|---|---|
| jam | glad you made it back safe bigjools | 05:24 |
| bigjools | morning jam | 05:24 |
| bigjools | me too ;) | 05:24 |
| === 18VAACLAG is now known as wallyworld | ||
| === wallyworld is now known as Guest12715 | ||
| jtv | Hi jam, hi bigjools | 07:34 |
| === mcclurmc_away is now known as mcclurmc | ||
| mgz | did anyone catch smoser yesterday? | 10:33 |
| mgz | he posted a why-what? comment in bug 1044503 which is a bit suprising as I thought we'd asked him about it last week | 10:34 |
| ubot5 | Launchpad bug 1044503 in maas (Ubuntu Raring) "kernel command line is not easily customizable" [High,Triaged] https://launchpad.net/bugs/1044503 | 10:34 |
| mgz | jam, dimitern: have put up a very simple migration branch, will commit some smarts around model and fallback now | 11:50 |
| Daviey | mgz: i did, i did! | 11:54 |
| mgz | Daviey: and the upshot was? :) | 11:55 |
| mgz | I'm still not really sure how this is going to be used, beyond maybe some simple customisations for specific hardware (like the highbank console thing currently hardcoded in the arch method) | 11:56 |
| Daviey | mgz: No upshot.. i just happend to catch him. | 11:57 |
| Daviey | now i think about it, it's not that helpful. | 11:57 |
| mgz | well, if you catch him again, don't let him go! | 11:58 |
| Daviey | because he bit your finger so? | 12:06 |
| mgz | hm, jenkins is unhappy, "Failed to create a temp file on /home/ubuntu/launchpad-jobs/workspace/maas-merger-quantal" | 12:10 |
| jam | mgz: it has been unhappy fora few days, it is either a perms issue, or a out-of-disk space, IMO | 12:10 |
| jam | rvba: ^^ you have direct access to jenkins, right? | 12:11 |
| jam | mgz: commented on https://code.launchpad.net/~gz/maas/1.2_add_db_kernel_params/+merge/133044 | 12:25 |
| mgz | thanks jam | 12:27 |
| rvba | jam: yes, the started yesterday, the whole filesystem of the vm went read only. matsubara is aware of the problem and worked on it yesterday; apparently he did not manage to fix the problem yet. I'm waiting for him or Larry to come online to see what they think. | 12:59 |
| rvba | s/yes, the/yes, that problem/ | 12:59 |
| matsubara | rvba, hey, I'm around now. I waiting on Larry as well since I couldn't get thta read-only problem yesterday | 13:14 |
| rvba | matsubara: all right. Please keep us posted when you'll have news, this problem is really blocking us now. | 13:16 |
| matsubara | rvba, will do. sorry about that | 13:16 |
| rvba | ta | 13:16 |
| roaksoax | rvba: hey man! how's everything? how was your flight back | 14:03 |
| rvba | roaksoax: Hey Andres, everything all right. How are you? | 14:03 |
| roaksoax | rvba: long :) | 14:04 |
| roaksoax | rvba: so anyways, where are we standing towards an SRU | 14:04 |
| rvba | Indeed. | 14:05 |
| roaksoax | rvba: are there things ready for me to start preparing for SRU? | 14:06 |
| rvba | roaksoax: you're talking about the precise SRU right? | 14:07 |
| gwd-laptop | I've got a question: Does MaaS require IPMI to function, or can it work with just DHCP / PXE / &c? | 14:07 |
| roaksoax | rvba: quantal | 14:08 |
| roaksoax | rvba: i need to catch up with the security team on the precise SRU | 14:08 |
| rvba | roaksoax: ok | 14:09 |
| roaksoax | rvba: but for 12.10 we has half of the bugs in the list already fixed, so I think we need to start SRU'ing by chunks | 14:09 |
| roaksoax | rvba: i'm also gonna be trying to reproduce the upgrade bug again and see if it shows up | 14:09 |
| rvba | roaksoax: we've committed most of the fix which will go in the SRU already https://launchpad.net/maas/+milestone/12.10-stabilization | 14:09 |
| rvba | fixes* even | 14:10 |
| rvba | roaksoax: 2 bugs are assigned to you in that list. Not sure what's the status for these two…? | 14:10 |
| roaksoax | rvba: well 1 is pretty hard to fix as there's no real way to find out whether it is a VM or real HW. and the second one is mostly IPMI issue itself, but will handle it by waiting more time before optaining an IP address, and dealing with the 0.0..0.0 as a failure to obtain IPMI address | 14:11 |
| mgz | huh. we haven't backported our other changes to 1.2 yet so basing on 1.2 was maybe not helpful. | 14:18 |
| * mgz adjusts the test | 14:18 | |
| mgz | jam, dimitern: how goes? | 14:32 |
| dimitern | mgz: I saw your change and I'm adapting my tests and code to match what you added | 14:32 |
| dimitern | I'll be ready later today with my changes, but still need a ui review | 14:33 |
| gwd-laptop | Ping? Anyone have an answer to my question? | 14:46 |
| mgz | gwd-laptop: what ever it was, it's not in my scrollback | 14:48 |
| gwd-laptop | mgz: Thanks -- the question was: "Does MaaS require IPMI to function, or can it work with just DHCP / PXE / &c?" | 14:48 |
| roaksoax | gwd-laptop: no it doesn't require IPMI to function, but then you would need to manually turn on machines | 14:50 |
| gwd-laptop | mgz: I think that's probably OK -- in this case they'll actually be VMs. :-) | 14:50 |
| * mgz takes all the credit | 14:50 | |
| gwd-laptop | Oops, sorry, didn't look closely. :-) | 14:50 |
| roaksoax | gwd-laptop: yeah, there's an option to use virsh as a power method | 14:51 |
| gwd-laptop | I work for Citrix on the Xen open-source team. I want to write a charm to install Xen / XCP on MaaS, so that they can be tested with the OpenStack testing that the Server team does. But we don't use MaaS for our test infrastructure at Citrix, so I was thinking of trying to use a virtualized environment. | 14:51 |
| mgz | that sounds like a good plan. not sure what docs we have currently on testing like that. | 14:52 |
| gwd-laptop | roaksoax: Would doing it manually work as well? I'm a lot more familiar with setting up XenServer than with setting up libvirt -- although, it's probably not bad to get a bit more practice with the latter... | 14:53 |
| smoser | mgz, the screen shots didn't make a lot of sense to me. | 14:54 |
| roaksoax | gwd-laptop: hold on, you want to deploy Xen based Vm's with MAAS? | 14:54 |
| roaksoax | gwd-laptop: or you simply want ot deploy Xen/XCP on machines using a charm? | 14:54 |
| gwd-laptop | roaksoax: What I actually want is a charm to install Ubuntu + Xen in a MaaS environment. | 14:55 |
| gwd-laptop | roaksoax: But I don't actually have a real MaaS environment, so I'm thinking of trying to set up a virtualized environment, just to test the charm. | 14:55 |
| gwd-laptop | roaksoax: If it's not really feasible, I can probably get a phsyical MaaS system set up here, but it's just more work. | 14:57 |
| roaksoax | gwd-laptop: right, so yeah a virtualized environment should be enough. The power management in maas is simply to start/stop machines/vm's when juju deploying something | 14:57 |
| gwd-laptop | roaksoax: So if I'm willing to babysit the installation and flip the virtual switch myself, I should be able to test the charm. Great -- I'll give that a try. | 14:58 |
| mgz | smoser: sorry, screenshots? customising the kernel params is pretty much just and admin level config thing to avoid some hardcoding of stuff | 14:58 |
| smoser | screenshots on the merge proposal | 14:58 |
| dimitern | smoser: that ones I added? | 14:58 |
| smoser | and i was never clear on what "tags" meant. i just wante dto see some example. | 14:58 |
| roaksoax | gwd-laptop: yeah, power management is not really required :) | 14:58 |
| gwd-laptop | roaksoax: Cool -- thanks for your help! | 14:59 |
| roaksoax | gwd-laptop: anytime! | 14:59 |
| mgz | dimitern: you seem to have proposed a 1.2 based branch against lp:maas rather than lp:maas/1.2 maybe? | 15:00 |
| dimitern | mgz: it's agains 1.2, the MP is there now just for ui review | 15:01 |
| dimitern | so it's on trunk (by mistake) | 15:02 |
| smoser | dimitern, yeah, i suppose. | 15:03 |
| smoser | i jus wanted an example description of how tags will be rendered. | 15:03 |
| mgz | smoser, okay, so the concrete stuff I can see this doing is changing stuff like the compose_arch_opts function in provisioning.kernel_opts into an admin level command like 'add-tag --kernel-params="console=ttAMA0" --definition="contains(/node/product, 'Highbank')"' | 15:03 |
| dimitern | smoser: so the tags are things you can attach to nodes, and the kernel opts can be part of the tag info, an in addition there are the global params in settings | 15:04 |
| mgz | so, rather than needing a provisioning server code change to alter the params for a specific set of hardware, you define something that matches against the lshw output, then the final stage boot uses the params attached to that tag | 15:04 |
| roaksoax | rvba: alright, so we are SRU'ing quantal MAAS into precise | 15:05 |
| mgz | roaksoax suggested this would be enough for what you guys want to do, hopefully that's the case? | 15:05 |
| smoser | hm.. | 15:05 |
| roaksoax | well yes, I suggested based on the fact that this was decided to be resolved using tags | 15:06 |
| smoser | mgz, so --definition= says 'attach tag named kernel-params with value "console=...." to all machines that are Highbank' | 15:07 |
| rvba | roaksoax: yeah, but that is going to require some extra work because of the dependencies (Django 1.4, etc.) | 15:07 |
| roaksoax | rvba: yeah, i guess we could simply SRU the minimal change as well? | 15:07 |
| smoser | and 'kernel-param' is some special tag that would even be identified as such in the maas-cli ? | 15:07 |
| roaksoax | rvba: that'[s the only blocker really | 15:07 |
| mgz | definition just specifies which nodes have that tag, based on their reported lshw output | 15:08 |
| mgz | by attaching the kernel params to tags, we hope to make the boot after that admin customisable based on the hardware | 15:08 |
| rvba | roaksoax: not sure it's the only blocker, I need to check the new dependencies that we use in the quantal version. | 15:08 |
| roaksoax | rvba: ok cool that would be helpful | 15:09 |
| roaksoax | mgz: right, so i think that regardless of the definition, it's still necessary to be able to assign kernel params to any given system | 15:09 |
| roaksoax | mgz: and might be required to assigned based on the name of the system | 15:10 |
| mgz | but what I'm still not clear on is which boots matter | 15:10 |
| smoser | well, any boot in which you want to do something matters | 15:10 |
| smoser | :) | 15:10 |
| mgz | with the ability to manually add a tag to any name, you can give one (or more) boxes completely custom params | 15:10 |
| smoser | so what happens if multiple tags named 'kernel-params' match a node? | 15:11 |
| mgz | what are the names for the three steps again? enlistment, commissioning, something else? | 15:11 |
| smoser | and are admin specified appended after MAAS required (ie, iscsi_target=....) | 15:11 |
| mgz | smoser: the easiest thing for us is to just take the params from one matching tag, in a deterministic fashion | 15:12 |
| jtv | mgz: you mean deployment? | 15:12 |
| smoser | that is the easiest thing, yes. | 15:12 |
| mgz | basically, tag-level customisation can't kick in till after we have the lshw output, which happens on the comissioning step, right? | 15:12 |
| roaksoax | right | 15:13 |
| smoser | so if i did like you suggested above with but with a different value with different criteria | 15:13 |
| smoser | what would happen? | 15:13 |
| smoser | which match would "win" | 15:13 |
| mgz | the globally configured ones could maybe be used before that, but there still needs to be a way for the nodes to talk to the maasserver to get that out | 15:13 |
| smoser | is that determinable? | 15:13 |
| mgz | smoser: one idea is to just take the first tag lexicographically, but we've not coded it yet, so what ever you like :) | 15:14 |
| mgz | the UI dimitern is working on aims to make this clear | 15:14 |
| smoser | that word is way to big for me. | 15:15 |
| mgz | by displaying the params used and where they came from | 15:15 |
| mgz | smoser: A comes before Z and so on | 15:15 |
| smoser | i'm confused. | 15:16 |
| smoser | which is why i just wanted some example that showed what would happen | 15:16 |
| nveitch | i think thats not a great idea | 15:17 |
| dimitern | smoser: you have tags, named "big-gpu", "large-mem", "0-special-tag", etc., so when sorted the 3rd one will be applied first, even if more than one of them match a single node | 15:17 |
| nveitch | chronologically would be better | 15:17 |
| mgz | the problem with chronologically is that would either need tracking seperately or would be unstable in the face of minor tweaks to tag definitions | 15:18 |
| smoser | right. | 15:18 |
| dimitern | smoser: and the other ones matching will be ignored, if none matches - use the global params from settings | 15:18 |
| smoser | so tag has name and value | 15:18 |
| smoser | right? | 15:18 |
| smoser | this is confusing me | 15:18 |
| dimitern | nveitch: ordering them chronologically poses problem with how to control the order and fine-tune it | 15:19 |
| mgz | they have a name, and a definition, which specifies what they match | 15:19 |
| dimitern | smoser: name, description (comment) and optionally, kernel_params | 15:19 |
| smoser | because it seems like your implying that tag_name="0-special-tag" and value="console=ttyS0" | 15:19 |
| mgz | (and a comment, which is just fluff, and shortly also an optional list of kernel params) | 15:19 |
| smoser | but then i dont know where the string 'kernel_params' would fit. | 15:19 |
| smoser | oh | 15:19 |
| smoser | that is odd | 15:19 |
| dimitern | smoser: the kernel_params field of the tag is the complete boot command line for the kernel, as I understand it - all params as a string | 15:20 |
| smoser | so you're suggesting that names have, importantly, a name and a 'kernel_params' | 15:20 |
| mgz | no, tag (name='0-special-tag', definition='true', kernel_params='console=ttyS0) | 15:20 |
| smoser | dimitern, well, it can't have the entire set of parameters, because that is not knowable outside of maas. | 15:20 |
| smoser | that is wierd. | 15:20 |
| dimitern | tag names are used just for sorting | 15:20 |
| smoser | dont you think this is weird? | 15:21 |
| smoser | we're promoting "kernel_params" to some sort of super item that is even then part of a tag? | 15:21 |
| dimitern | smoser: maybe I don't get it well enough to tell why it's weird? | 15:21 |
| mgz | maas will still supply some internally (needs to for various things that take arguments), but will allow you one extra set of params to stick on the end | 15:21 |
| smoser | if you told me "you can apply tags to nodes or groups of nodes" | 15:21 |
| mgz | smoser: yes. but I don't think it really matters. | 15:21 |
| smoser | i would think that a "tag" is either a key:value pair, or (less usefully) a 'name' | 15:22 |
| smoser | but i would probably not expect that it is a key:value:kernel_params triplet | 15:22 |
| dimitern | smoser: a tag is a few extra properties, which can be attached to a node, one of them can be the kernel params | 15:22 |
| mgz | it complicates the tag is name+definition thing, but sticking tag associated kernel params elsewhere wouldn't really change much | 15:22 |
| dimitern | smoser: it's not just key/value | 15:22 |
| mgz | I would prefer if the apis were kept clean of kernel stuff, but jam disagreed on the basis that this is the simplest thing | 15:23 |
| mgz | it would be perfectly possible to stick the actual data whereever, the main question is if getting that stuff based on one tag would be okay for what you'll want to do with it | 15:26 |
| mgz | the main issue with fancier things seems to be that kernel params are pretty opaque to us, we can't easilly tell what overrides what, what combines with what, so merging multiple params is non-trivial | 15:28 |
| smoser | right. | 15:29 |
| smoser | so i think its not at all unreasonable to just throw up your hands. and not attempt to merge or join in any way, but just let some definition "win" | 15:30 |
| smoser | but with tags, that seems somewhat arbitrary. | 15:30 |
| smoser | so i suspect thats why the lexical ordering based on 'name'. | 15:31 |
| mgz | right, and then we document that you name your tags specially for kernel params with 001- prefixes if you want clarity | 15:32 |
| smoser | but if we're playing the funny name game ... | 15:33 |
| smoser | then why not | 15:33 |
| smoser | 001-kernel-params="console=ttyS0" | 15:33 |
| mgz | because how do you then associate 001-kernel-params with a particular set of machines? | 15:34 |
| smoser | however you do that anyway. | 15:34 |
| smoser | i had assumed the --definition thing was not specific | 15:34 |
| smoser | to kernel params | 15:34 |
| mgz | or you mean pickling the params into the tag name? | 15:34 |
| smoser | if i have both key and value for a "tag" (which 'im still confused by) | 15:35 |
| smoser | then no pickling needed. | 15:35 |
| mgz | tags are just names, with some fancy logic attached | 15:35 |
| mgz | the fancy logic is optional | 15:35 |
| mgz | rather than needing to manually look at each machine and say whether a tag applies to it or not (node triage...) | 15:36 |
| mgz | you can give a definition that we then use to automatically apply the tag if appropriatte to any new machines as they show up | 15:36 |
| smoser | ok. that at least maks sense. | 15:37 |
| mgz | the kernel params are again just a bit of extra logic bundled with the tag, to say when booting a machine that matches this tag, add in these params | 15:37 |
| smoser | then yeah, my solution would require pickling name/value | 15:37 |
| smoser | which i dont think is beautiful | 15:37 |
| smoser | but i dont think the extra special attribute is either. | 15:37 |
| smoser | but then my suggestion was that you could just have a name of "001-kernel-params=console=ttyS0" | 15:38 |
| smoser | which would probably be improved to move kernel-params to the beginning. | 15:38 |
| smoser | but either way. | 15:39 |
| smoser | http://pad.daviey.com/maas-kernel-params | 15:39 |
| smoser | i put 3 points there that i'd like to at least have considered. | 15:39 |
| smoser | and know how i would address them | 15:40 |
| smoser | i think it'll work, but i just iddn't wan tto create somethign that wouldn't solve anyone's actual problems | 15:40 |
| mgz | thanks, extra details are useful | 15:42 |
| mgz | the -- is interesting... I wonder if we can leave that just to the user supplied params? | 15:45 |
| mgz | or will some of the ones that maas needs to add in itself have to be mixed before and after? | 15:45 |
| mgz | ...or can we assume user supplied ones should always be carried over to the installed system? | 15:46 |
| === mcclurmc is now known as mcclurmc_away | ||
| smoser | mgz, well, the -- is annoyhing. | 15:57 |
| smoser | and i ran into this personall a bit ago. | 15:57 |
| smoser | the BOOTIF= is the most annoying bit | 15:58 |
| smoser | as it gets copied over :) | 15:58 |
| smoser | as IPAPPEDN is "append" | 15:58 |
| === mcclurmc_away is now known as mcclurmc | ||
| rvba | jam: jtv mgz: matsubara fixed the Jenkins problem (apparently, rebooting the vm many times fixed it). | 17:11 |
| jtv | rvba: \o/ thanks! | 17:12 |
| rvba | Diogo will investigate further but in the meantime so that it does not happen again but we can land our branches. | 17:12 |
| jtv | And thanks to matsubara for the work... what fixes it for me is a manual “fsck -f -y /dev/<partition>” from a boot shell. | 17:13 |
| mgz | go matsubara | 17:14 |
| jtv | rvba: you landed my lint branch already? | 17:14 |
| rvba | jtv: yes, I used that branch as a test :) | 17:14 |
| jtv | I was so looking forward to landing that one. :) | 17:14 |
| roaksoax | rvba: jtv so when do you guys have a catchup call | 17:19 |
| jtv | roaksoax: normally around 08:00 UTC | 17:19 |
| rvba | roaksoax: 08:30 UTC | 17:20 |
| jtv | (I'm being vague because I believe it was just being changed due to DST) | 17:20 |
| jtv | Ah yes, the half hour too. | 17:20 |
| roaksoax | uh, that's early for me | 17:20 |
| roaksoax | :) | 17:20 |
| roaksoax | rvba: ok so we need to have a catch up call to talk about the upgrade path | 17:20 |
| roaksoax | and so on | 17:20 |
| rvba | roaksoax: ok, we can do that tomorrow. | 17:21 |
| roaksoax | alright cool | 17:23 |
| jtv | roaksoax: thanks for the review... I'm out too. :) | 17:34 |
| mgz | matsubara: tried merging again, but it failed once more alas... | 17:36 |
| matsubara | mgz, which merge proposal? | 17:37 |
| mgz | jenkins job 252, merge for lp:~gz/maas/1.2_fix_duplicated_039_migration | 17:39 |
| matsubara | job #253 passed | 17:41 |
| mgz | was that against trunk or against 1.2? | 17:42 |
| matsubara | 1.2 | 17:43 |
| matsubara | there are separate jobs for 1.2 and trunk | 17:43 |
| mgz | shall I try again and hope? | 17:44 |
| matsubara | I just did | 17:46 |
| matsubara | let's see | 17:46 |
| matsubara | mgz, the eagle has landed | 18:12 |
| mgz | woho! | 18:15 |
| mgz | clearly jenkins likes you better :) | 18:15 |
| === mcclurmc is now known as mcclurmc_away | ||
| === mcclurmc_away is now known as mcclurmc | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!