| lifeless | bigjools: yo | 01:58 |
|---|---|---|
| bigjools | g'day | 01:59 |
| negronjl | hello all | 01:59 |
| lifeless | bigjools: meet Juan, negronjl meet Julian, dev lead for maas. | 01:59 |
| negronjl | hello bigjools | 01:59 |
| bigjools | hi | 02:00 |
| negronjl | bigjools: I am getting ready to do a MaaS demo @ Structure and I am having issues with TFTP | 02:00 |
| negronjl | bigjools: TFTP daemon ( I believe tftpd-hpa ) is not running on boot so I started it and now I get TFTP unable to locate configuration file | 02:01 |
| bigjools | is manage_tftpd set to 1 in /etc/cobbler/settings? | 02:03 |
| negronjl | bigjools: checking ... | 02:03 |
| negronjl | bigjools: yes | 02:04 |
| bigjools | ok run "cobbler sync" | 02:04 |
| bigjools | and it should write a config out | 02:05 |
| negronjl | bigjools: ok ... brb ( setup is in the other room ) | 02:05 |
| negronjl | bigjools, lifeless: well.... this is embarrassing, I had "cobller sync" in my restart-everything script .... | 02:07 |
| negronjl | bigjools, lifeless: that did it by the way | 02:07 |
| bigjools | nice :) | 02:07 |
| bigjools | great | 02:07 |
| * negronjl bangs his head against the wall a couple of times :/ | 02:07 | |
| lifeless | you'll be glad to know cobbler is being nuked atm | 02:09 |
| lifeless | so next version should have new and interesting failure modes. | 02:09 |
| bigjools | with extreme pleasure | 02:09 |
| negronjl | lifeless, bigjools: NICE!!!!!!!! | 02:09 |
| negronjl | less banging on head | 02:10 |
| negronjl | bigjools: ping | 04:03 |
| negronjl | has there been a resolution to but 992075 | 04:47 |
| negronjl | sorry bug 992075 | 04:47 |
| ubot5 | Launchpad bug 992075 in maas (Ubuntu) "Commissioning status persists with cloud-init 0.6.3-0ubuntu1" [Undecided,Confirmed] https://launchpad.net/bugs/992075 | 04:47 |
| negronjl | cloud-init-nonet killed by TERM signal | 04:47 |
| negronjl | now I'm stuck there. | 04:47 |
| bigjools | negronjl: sorry no idea, we need smoser to look at that | 04:50 |
| negronjl | bigjools: thx ... I'll ping smoser tomorrow | 05:24 |
| lifeless | negronjl: get some sleep :P | 05:49 |
| negronjl | lifeless: yup. I think I will ... later all .. and thanks for the help | 05:50 |
| rick1 | I always get "httpd does not appear to be running and proxying cobbler" when I was running maas-import-isos, is there any hints? | 10:32 |
| === matsubara is now known as matsubara-lunch | ||
| rvba | roaksoax: hi there, I've got 2 questions for you about the preseed files. | 12:12 |
| rvba | roaksoax: first question: where exactly is the maas-enlist.preseed used? It's used to create an "enlist" profile in scripts/maas-import-isos but then I don't see where this profile is used. | 12:14 |
| === matsubara-lunch is now known as matsubara | ||
| rvba | roaksoax: second question. I see you've divided the pressed files in templates and snippets and the snippets are included by the templates. I see that most of the snippets (actually, all but one) are used only once and most of them are only one lines (or, at most, a few lines). I was wondering if you would be against some reorganisation for these files. The first reason, as described previously is that I | 12:17 |
| rvba | don't really see the point in having a snippet when it's used only once. The second reason is that the templating language we're planning to use (tempita) support inheritance (a la Django) but not inclusion. What do you say? | 12:17 |
| rvba | s/one lines/one line/ | 12:18 |
| roaksoax | rvba: 1. /var/lib/tftpboot/pxelinux.cfg/default | 13:39 |
| roaksoax | rvba: when a system PXE boots, and it doesn't find itself in the PXE server, then it shows a PXE menu which the various options (cobbler profiles). The default to be run is maas-enlist | 13:40 |
| roaksoax | rvba: 2. uhmmm what snippets are you referrering too. I think that ideally we would like to keep the concept of snippets because each prpovide a different capability that we might not want in the preseed | 13:42 |
| roaksoax | rvba: smoser ^^ | 13:42 |
| roaksoax | err | 13:42 |
| roaksoax | smoser: ^^ what are your thouhgts | 13:42 |
| rvba | roaksoax: (1.) All right, I'll need to coordinate with jtv then because he's working on tftp right now. Thanks! | 13:44 |
| rvba | roaksoax: (2.) I'm not suggesting dropping the notion of splitting up the templates into small bits. I'd like to know if you prefer to keep the mechanism as is or move to a inheritance mechanism (a la Django templates). I'm asking because only is supported by default by the template language we use :). | 13:45 |
| rvba | s/because only/because only the latter/ | 13:46 |
| rvba | roaksoax: I'm referring to all of the snippets in contrib/snippets/. | 13:48 |
| roaksoax | rvba: so the thing is that the concepts of snippets are good because they keep small pieces apart to do different things that can be reused when necessary | 13:48 |
| smoser | i dont think anyone is specifically tied to the word "snippets" | 13:49 |
| rvba | roaksoax: indeed. What I'm proposing is somewhat similar. It allows you to do the exact same thing but uses inheritance instead of inclusion. | 13:49 |
| rvba | It's sort of done the other way around. | 13:49 |
| roaksoax | rvba: right, but would the preseed be able to say | 13:50 |
| smoser | without much information, i suspect that you'll still be rednering the preseed files with an object's ".preseed()" or something. | 13:50 |
| roaksoax | rvba: right, but would the preseed be able to say "i want to inherit this feature for XYZ?" | 13:50 |
| smoser | so in the end you'll have the same, dynamic result in that you can customize that stuff. | 13:50 |
| roaksoax | smoser: right but a snippet can not only contain a dynamic result, but a static one | 13:50 |
| smoser | well, foo.packages() can just return "some static package list" | 13:51 |
| rvba | roaksoax: definitely, it would be done in a different way though. Let me give you an example. | 13:51 |
| smoser | (and then foo.packages() is used by foo.preseed()) | 13:51 |
| smoser | an example would be good. but rvba the key element is that you need preseed files, and even kernel command line to be customizable | 13:52 |
| roaksoax | and "snippets" if they are kept | 13:52 |
| smoser | snippets are a mechanism to get runtime evaluation | 13:53 |
| smoser | and we need "pluggable runtime evaluation" in some mechanism. | 13:53 |
| smoser | i'm not willing to argue over how that is implemented. | 13:53 |
| smoser | s/willing/interested/ | 13:53 |
| rvba | smoser: preseed files will be customizable. We've added a simple way for now and we're ready to make it more complex if we need to. | 13:54 |
| rvba | smoser: 1. one will be able to add a new directory where templates will be looked up first before resorting to the default (maas provided) ones. | 13:55 |
| rvba | roaksoax: 2. We've added a (simple) look-up mechanism: for instance if MAAS is looking up the template named 'enlist' it will first see if 'enlist_$nodearch_$nodesubarch_$hostname' exists. Then 'enlist_$nodearch_$nodesubarch"… etc… the 'enlist' and finally 'generic'. | 13:56 |
| rvba | smoser: ^ | 13:57 |
| rvba | roaksoax: so, we would have a 'master' template: http://paste.ubuntu.com/1047323/ and then two child templates inheriting from it: http://paste.ubuntu.com/1047325/ http://paste.ubuntu.com/1047334/ | 13:58 |
| smoser | rvba, my concern with that is that it does not appear "dynamic" | 13:59 |
| rvba | roaksoax: the results would be (respectively) http://paste.ubuntu.com/1047337/, http://paste.ubuntu.com/1047338/ | 13:59 |
| rvba | smoser: what do you mean by that? | 13:59 |
| roaksoax | rvba: and would you be able to evaluate, within the tempaltes, whether a particular system is running precise, oneiric, or quantal? or things like that? | 14:00 |
| smoser | by what you show there, ti see no way for me to extend 'top' to do a lookup into a hardware database and output specific information (such as kernel command lines) for that entity. | 14:01 |
| smoser | (or actually do any sort of "dyanmic" nature) | 14:01 |
| roaksoax | we also would lose the ability to be able to remove stuff from the preseed (or master template), or momentarily comment out. We would have to create another master template and then inherit | 14:01 |
| roaksoax | TBH, I personally like the way how cobbler handles templating | 14:02 |
| rvba | roaksoax: smoser these template will have access to as much context as you want: the node itself (node.architecture will give you the architecture), the release, etc. | 14:02 |
| smoser | if what you have shown us there is the extent of the dynamic nature, then i think i can pretty definitively tell you it will end up not being good enough. | 14:02 |
| rvba | smoser: this template system supports if/loops etc. | 14:02 |
| rvba | And we can extend it to include python code directly even if it's rather ugly. | 14:03 |
| roaksoax | rvba: we currently need importing of poython code | 14:03 |
| smoser | i really could care less about "ugly" | 14:03 |
| smoser | but thats fine. | 14:03 |
| smoser | if we can execute loops, if, and python code, i'm fine with that. | 14:04 |
| rvba | smoser: by ugly I mean less easy to test. We'd rather keep the default templates clean and testable. But for one-off customization, fine. | 14:04 |
| smoser | fair | 14:05 |
| rvba | roaksoax: are you referring to the base64 module? | 14:05 |
| roaksoax | ya | 14:05 |
| roaksoax | rvba: in orchestra we used to have our own custom python module to do stuff | 14:06 |
| roaksoax | so I personally think it might be nice to have that ability in case is needed | 14:06 |
| rvba | No problem, that's built-in. | 14:06 |
| rvba | You can do that kind of stuff (http://paste.ubuntu.com/1047347/) so I'm sure there is all the flexibility we need. | 14:08 |
| roaksoax | rvba: so basically, we could do something like: http://pastebin.ubuntu.com/1047349/ | 14:08 |
| roaksoax | ? | 14:08 |
| rvba | roaksoax: well, no, that's the point (or the problem ;)). | 14:08 |
| rvba | roaksoax: you would do something like that:http://paste.ubuntu.com/1047351/ | 14:10 |
| rvba | The master would look like that: http://paste.ubuntu.com/1047352/ | 14:10 |
| rvba | Basically, it defines 'slots' which are filled in by the child templates inheriting from it. | 14:11 |
| roaksoax | rvba: so this is what happens: http://pastebin.ubuntu.com/1047353/ | 14:11 |
| roaksoax | right? | 14:11 |
| rvba | roaksoax: exactly | 14:11 |
| roaksoax | rvba: so in order for us to add/remove stuff, we have to write a lot of "code" rather than simply importing a snippet right? | 14:12 |
| rvba | roaksoax: which (I think – but what do I know ;)) makes sense because the most specific stuff is then written in the child template. | 14:12 |
| roaksoax | rvba: doesn't that make it non-easy to administer instead of having snippets that are easily plugable to the preseed? | 14:12 |
| rvba | roaksoax: I don't agree… hold on one sec. | 14:13 |
| roaksoax | rvba: because in the current approach we have. 1. a preseed. 2. we can easily plug features to the preseed. 3. don't worry about anything | 14:13 |
| rvba | roaksoax: We have the exact same feature here, hang on :) | 14:13 |
| rvba | roaksoax: so I suppose your point is that you'll ahve to write the 'preseed' bit over and over again right? | 14:14 |
| rvba | roaksoax: that's not the case, you'll put it in the master template. And child templates will still be able to override it (or not). | 14:14 |
| roaksoax | here we are kinda like 1. inherit the preseed. 2. manually add stuff 3. create another child used 1 and 2. 4. in order to add more stuff, we either go to 2, or 4, and create 5 | 14:14 |
| roaksoax | rvba: and that's the thing, can't we simply make all the snippet inheritable? | 14:16 |
| rvba | roaksoax: what do you mean by 'make all the snippet inheritable'? | 14:16 |
| roaksoax | rvba: that would be much easier. If we can inherit master, can't we just also inherit any other in the same file/temaplte? | 14:16 |
| rvba | roaksoax: 'master' is the file name. You can inherit from whatever file you want as long as it's available on the filesystem. | 14:17 |
| rvba | roaksoax: in your example, the bit that is in {{def preseed}} would be put in the master because basically all child template will use this. | 14:18 |
| roaksoax | rvba: right, but can't we have various masters? | 14:19 |
| rvba | roaksoax: absolutely. | 14:19 |
| rvba | We can. | 14:20 |
| roaksoax | rvba: so we can have this then: http://pastebin.ubuntu.com/1047370/ | 14:20 |
| roaksoax | rvba: simply keep the snippet code as "snippet" and in the child file we just "include" the snippet as if it was a master | 14:20 |
| roaksoax | rvba: so including 3 masters in a child means, 3 different parts together, right? | 14:21 |
| roaksoax | that way, you kind keep the same functionality | 14:21 |
| rvba | roaksoax: there is the problem, there is no 'include' tag. 'Inclusion' is done the other way around: by using inheritance. | 14:21 |
| rvba | roaksoax: you last paste describe something that is not possible. | 14:22 |
| roaksoax | rvba: right, but what I mean, maas.preseed --> inherit maas_proxy snippet code, inherit maas_disable_pxe snippet code | 14:22 |
| roaksoax | etc | 14:22 |
| roaksoax | rvba: what's the tag used for inheritance? | 14:23 |
| roaksoax | rvba: how would you write a template that will inherit | 14:23 |
| rvba | roaksoax: inherit | 14:23 |
| rvba | roaksoax: you're right, in my examples, s/include/inherit/ ;) | 14:23 |
| roaksoax | rvba: http://pastebin.ubuntu.com/1047380/ | 14:24 |
| roaksoax | would that be possible? | 14:24 |
| rvba | roaksoax: yes, but snippets/maas_proxy would have to simply define something. And that something would then have to be used in that file. | 14:25 |
| rvba | roaksoax: I think it would be quicker if we could have a quick chat about this. Note that I'm not again adding the beloved 'include' tag. | 14:26 |
| rvba | roaksoax: can we have a quick chat in a g hangout? | 14:26 |
| roaksoax | rvba: sure, but can you give me say 15 mins? | 14:27 |
| rvba | roaksoax: sure. | 14:27 |
| roaksoax | rvba: and would we be able to do this? http://pastebin.ubuntu.com/1047388/ | 14:27 |
| roaksoax | rvba: specially note the last part where we are adding stuff to a late_command line, rather than adding various late_commands. and we need to be able to that | 14:27 |
| roaksoax | rvba: that's one of the issues that I see | 14:28 |
| rvba | roaksoax: let me try something and I'll be able to reply to your question. | 14:28 |
| roaksoax | brb | 14:28 |
| roaksoax | rvba: sorry, i'm back | 15:08 |
| rvba | roaksoax: all right, no worries.* | 15:09 |
| roaksoax | rvba: ok so were you able to see whether the above works? | 15:10 |
| rvba | roaksoax: it does not. | 15:10 |
| roaksoax | smoser: you wnat it? | 15:10 |
| * smoser reads backscroll | 15:12 | |
| negronjl | smoser: I modified the image as you recommended ... I am now at the point where I get the error: cloud-init-nonet main process (344) killed by TERM signal. Should I wait it out and then try to log in or is there something else I should be doing? | 15:24 |
| negronjl | smoser: I'm in the node ... anything in particular I should be looking at? While you get back to me I'll be looking around in that node | 15:28 |
| smoser | negronjl, log into the system now. | 15:35 |
| smoser | and you want to run the cloud-init code that would read the data from the server | 15:36 |
| smoser | so... baically | 15:36 |
| smoser | /proc/cmdline has a url on it | 15:36 |
| smoser | cloud-init reads that, saves it into /etc/cloud/cloud.cfg.d/91-something | 15:36 |
| negronjl | smoser: ok ... waiting on the system ( as I was waiting I tried it again ) | 15:37 |
| smoser | negronjl, ok. sorry. | 15:39 |
| negronjl | smoser: no worries ... I appreciate you taking the time to help | 15:39 |
| negronjl | smoser: I see the /proc/cmdline output as well as /etc/cloud/cloud.cfg.d/91_kernel_cmdline_url.cfg | 15:43 |
| negronjl | smoser: the only thing i see out of all that that seems out of place is http://ubuntu-maas/MAAS/metadata for the metadata_url ( ubuntu-maas is resolvable to 127.0.1.1 ) | 15:45 |
| negronjl | smoser: not sure if that's correct or not | 15:45 |
| smoser | negronjl, yeah, thats wrong. | 15:48 |
| smoser | :) | 15:48 |
| smoser | cloud-init should be more annoying in that case. | 15:48 |
| smoser | can i 'look at that system at all? | 15:48 |
| negronjl | smoser: it's in an isolated network. Can you give me an example of what it should look like so I can figure it out? | 15:50 |
| negronjl | smoser: I will also try to put this set up online somewhere where we can see it | 15:51 |
| smoser | h.. | 15:51 |
| smoser | can you pastebin /var/log/cloud-init.log ? | 15:51 |
| smoser | and /var/log/cloud-init-output.log (if that is present) | 15:51 |
| negronjl | I'll give that a shot ... | 15:51 |
| negronjl | smoser: it's worth mentioning that this worked fine the first two times that I tried it. | 15:52 |
| negronjl | smoser: that happened yesterday ..... I then lost my cobbler configuration ( i had to run cobbler sync ) ... it has never worked after that | 15:53 |
| smoser | negronjl, so. | 16:01 |
| smoser | i think what is wrong is that you need to run dpkg-reconfigure maas | 16:02 |
| smoser | (which will re-set the value of the "maas-server") | 16:02 |
| smoser | but how is ubuntu-maas resolvable? | 16:02 |
| smoser | who is feeding that data, because that is what is wrong. | 16:02 |
| smoser | maas told cloud-init to get some data from the url http://ubuntu-maas/MAAS/metadata . | 16:02 |
| smoser | clearly, thats bad. | 16:02 |
| smoser | :) | 16:02 |
| negronjl | smoser: let me try that ... | 16:02 |
| smoser | on the node, though? | 16:03 |
| smoser | you said "resolvable" | 16:03 |
| smoser | resolvable how? | 16:03 |
| smoser | 'host ubuntu-maas' ? | 16:03 |
| negronjl | ping ubuntu-maas | 16:03 |
| smoser | what about 'host ubuntu-maas' ? | 16:03 |
| negronjl | host ubuntu-maas ( 127.0.1.1 ) | 16:03 |
| negronjl | just ran dpkg-reconfigure maas ... trying again | 16:04 |
| smoser | k. so who is serving 'ubuntu-maas == 127.0.1.1' ? | 16:04 |
| smoser | yo uhave that in local dns somewhere? | 16:04 |
| smoser | is maas serving dns ? | 16:04 |
| smoser | where is htat coming from. because that is bad. | 16:05 |
| negronjl | checking | 16:06 |
| negronjl | apparently the maas-server is running a dns server that is serving that address .. | 16:07 |
| negronjl | I believe it's via dnsmasq | 16:07 |
| smoser | roaksoax, ^ | 16:08 |
| smoser | you know how that happened ? | 16:09 |
| smoser | negronjl's maas-server is telling nodes that 'ubuntu-maas' (itself) can be reached at 127.0.1.1 | 16:09 |
| smoser | surprisingly that doesn't work for systems other than itself :) | 16:09 |
| negronjl | I am grepping around to see if I find out where that is being configured | 16:09 |
| roaksoax | negronjl: is this a clean install? what does /etc/hosts show? | 16:11 |
| negronjl | roaksoax, smoser: i found it on /etc/hosts on the maas server | 16:12 |
| negronjl | roaksoax, smoser: i just don't see how that affects the other nodes | 16:12 |
| smoser | negronjl, roaksoax this would seem to be a significant issue | 16:13 |
| roaksoax | negronjl: so when you dpkg-reconfigure maas you are using ubuntu-maas right? | 16:13 |
| smoser | and quite possible the root of a lot of people's issues | 16:13 |
| smoser | as the installer (i think) is going to write a 127.0.1.1 entry for 'hostname' | 16:13 |
| smoser | and then maas is going to | 16:14 |
| smoser | a.) tell other systems "talk to me by my new hostname, $HOSTNAME" | 16:14 |
| roaksoax | smoser: so my thinking is that negronjl used as 'network address' ubuntu-maas instead of the IP address. | 16:14 |
| smoser | *and* | 16:14 |
| smoser | b.) (via dns) my new hostname $HOSTNAME == 127.0.1.1 | 16:14 |
| smoser | what do you mean, roaksoax ? | 16:14 |
| roaksoax | smoser: since dnsmasq is serving on the localnode, it reads /etc/hosts and founds that ubuntu-maas is 127.0.1.1, and that's why it gets serveed that way | 16:14 |
| negronjl | smoser, roaksoax: I commented out the offending /etc/hosts entries | 16:15 |
| smoser | what do you mean " negronjl used as 'network address'" | 16:15 |
| roaksoax | negronjl: what did you input when you did 'sudo dpkg-reconfigure maas' | 16:15 |
| smoser | ok.. | 16:15 |
| smoser | so i dont know if that is going to fix your problem | 16:15 |
| roaksoax | smoser: on sudo dpkg-reconfigure maas, it asks for an IP address which will be the address that it iwll hand out to the clients | 16:15 |
| negronjl | smoser, roaksoax: dpkg-reconfigure maas => I used 192.168.1.2 ( my network address ) | 16:15 |
| roaksoax | smoser: that address is set on /etc/maas/maas_local_settings.py | 16:16 |
| roaksoax | smoser: as DEFAULT_MAAS_URL | 16:16 |
| negronjl | checking /etc/maas/maas_local_settings.py | 16:16 |
| smoser | negronjl, so after you make that change | 16:16 |
| negronjl | that file still says http://ubuntu-maas/ | 16:17 |
| roaksoax | smoser: if, by default, you do not set that in DEFAULT_MAAS_URL, it will try to guess the hostname/ip of the node | 16:17 |
| roaksoax | negronjl: that's weird | 16:17 |
| smoser | then the 'url=' parameter that you saw on the kernel command line should serve you data with a new-and-iproved maas_url | 16:17 |
| roaksoax | negronjl: can you add -x to the postinst script to see what's going on please? | 16:17 |
| negronjl | I can manually change the DEFAULT_MAAS_URL to the correct value but, It's worth finding out how it got that way to begin with | 16:17 |
| roaksoax | negronjl: ahhh I see | 16:18 |
| roaksoax | negronjl: is this a clena install? | 16:18 |
| negronjl | no idea .. i just got the hardware from the Montreal office | 16:20 |
| negronjl | so I don't think so | 16:20 |
| negronjl | smoser, roaksoax: proc_cmdline: http://paste.ubuntu.com/1047565/ 91_kernel_cmdline_url.cfg: http://paste.ubuntu.com/1047566/ | 16:21 |
| roaksoax | negronjl: yeah | 16:21 |
| roaksoax | negronjl: i know that the issue is | 16:21 |
| roaksoax | negronjl: so they *manually* edit /etc/maas/maas_local_settings.py to use ubuntu-maas instead of the IP address because I'm guessing they also had some kind of external DNS | 16:22 |
| negronjl | roaksoax: what should i set that value to then ? | 16:22 |
| roaksoax | negronjl: the IP address | 16:22 |
| negronjl | roaksoax: doing it now ... | 16:22 |
| roaksoax | negronjl: then restart apache | 16:22 |
| roaksoax | negronjl: and just in case a sudo cobbler sync | 16:22 |
| smoser | roaksoax, well, one other option. | 16:22 |
| smoser | is to actually set 'ubuntu-maas' to resolve correcctly | 16:23 |
| smoser | which should be acheivable via editing /etc/hosts and restating dnsmasq | 16:23 |
| negronjl | that's what I was thinking | 16:23 |
| roaksoax | yes | 16:23 |
| roaksoax | that too | 16:23 |
| roaksoax | but dpkg-reconfigure maas only accepts IP addresses | 16:23 |
| smoser | ah | 16:23 |
| negronjl | roaksoax: I put the right address in dpkg-configure maas | 16:23 |
| smoser | (well, that is kind of silly, but roaksoax and i have argued about that before) | 16:23 |
| negronjl | roaksoax: It just did _not_ update the values on maas_local something or other .py | 16:24 |
| roaksoax | negronjl: yes, but the checking mechasnims checks for a regex that matches an IP address | 16:24 |
| negronjl | roaksoax: ahh .. I get it now | 16:24 |
| roaksoax | negronjl: and since DEFAULT_MAAS_URL = ubuntu-maas instead of DEFAULT_MAAS_URL = X.Y.Z.A , then it didn't replace it | 16:24 |
| negronjl | we need to change it to an IP on maas_local_???.py | 16:24 |
| roaksoax | negronjl: no if you set the IP in /etc/hosts | 16:25 |
| roaksoax | (the correct IP) | 16:25 |
| negronjl | roaksoax: maybe i missed something but, i thought that, when you dpkg-reconfigure maas, there was a regex that would look for an IP ( presumably in /etc/maas/maas_local_settings.py ) and change it ... that would still be broken if I were to leave "ubunt-maas" in /etc/maas/maas_local_settings.py and just have it resolve properly in /etc/hosts | 16:27 |
| negronjl | that would solve _my_ issue but, it wouldn't solve the root issue ... the regex is not cathing the value to change | 16:28 |
| negronjl | brb | 16:29 |
| negronjl | back | 16:33 |
| roaksoax | negronjl: right, becuase in the first place, there shouldn't be a hostname instead of an IP address. that hostname mas changed manually, not by packaging | 16:34 |
| roaksoax | negronjl: hence breaking it | 16:34 |
| negronjl | trying again | 16:35 |
| roaksoax | negronjl: packaging on a clean install sets an IP address. If we reconfigure, it will be changed correctly. However, since someone manually changed from a IP to a hostname by directly editing the config file, then dpkg-reconfigure does update the value correctly as it checks that an IP address is there | 16:36 |
| negronjl | roaksoax: I see ... a suggestion would be that, on reconfigure of any kind, ignore what is there and just replace it ... my two cents. | 16:37 |
| negronjl | roaksoax: but I understand the issue. | 16:37 |
| negronjl | roaksoax: and it is working correctly now | 16:37 |
| negronjl | roaksoax, smoser: Thanks for all of the help. I'll keep poking at this and hopefully won't need to bother you guys anymore | 16:37 |
| smoser | roaksoax, if someone edits the config file, then maas should respect that on dpkg-reconfigure | 16:40 |
| smoser | and if they used a dnsname, it should accept that too | 16:40 |
| roaksoax | negronjl: yet. could please file a bug? :) | 16:45 |
| roaksoax | smoser: I agree it should not specifically check for an IP to be set, but IMHO I think we shouldn't rely on DNS | 16:46 |
| roaksoax | smoser:becuase of the issue we just saw | 16:46 |
| smoser | well, it did the rigth thing. | 16:47 |
| smoser | (which was to do what the human told it to) | 16:47 |
| smoser | its fine to not use dns by default | 16:47 |
| roaksoax | ack | 16:48 |
| kurt_ | Can you guys tell me if PXE boot from a kvm client to another bridged VM maas server on a separate host is supported? | 16:50 |
| kurt_ | ie. should the kvm client PXE boot from the VM maas on another host? | 16:50 |
| kurt_ | with the current version | 16:50 |
| kurt_ | or is that a bigjools question? | 16:51 |
| smoser | kurt_, it really just depends on the networking i think. | 16:56 |
| smoser | i'm not an expert on pxe, but the pxe broadcast must get from the booting-system (or kvm instance, or whatever) to the pxe server, and then the traffic also get back. | 16:57 |
| kurt_ | I have both interfaces bridged and connected via a dedicated switch | 16:57 |
| kurt_ | so they are on the same network | 16:57 |
| smoser | then i would suspect it to work. | 16:57 |
| kurt_ | but, you think that functionality for PXE boot for kvm is there then? | 16:58 |
| smoser | well, kvm pxe boot works, yes. | 16:59 |
| smoser | and if the networking is all set up correctly, then it has no way to *not* work. | 16:59 |
| smoser | :) | 16:59 |
| negronjl | smoser, roaksoax: how do I get oath token for juju to work on maas | 17:00 |
| smoser | i think in th emaas ui somewhere? | 17:00 |
| smoser | (you cna get it from the DB for your user-id) | 17:01 |
| negronjl | smoser: thx | 17:01 |
| smoser | kurt_, did i answer your question? | 17:02 |
| kurt_ | yes, mostly…but I wasn't able to get it to work before... | 17:02 |
| kurt_ | bigjools was having me look at vender? | 17:02 |
| kurt_ | vendev | 17:03 |
| kurt_ | or something like this? | 17:03 |
| kurt_ | I was wondering why he had me going down this path - what advantage it had over straight PXE boot | 17:03 |
| negronjl | I have each node ( the HP microservers ) enabled for WoL, I also have them as WoL at the maas-server but they don't start up .... am I missing anything here | 18:40 |
| negronjl | ? | 18:40 |
| negronjl | smoser, Daviey, roaksoax ^^ | 18:42 |
| smoser | you are missing something | 18:43 |
| smoser | specifically https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/981461 | 18:44 |
| ubot5 | Ubuntu bug 981461 in ifupdown (Ubuntu Precise) "wol broken on HP ProLiant N40L (Broadcom tg3 driver)" [High,Triaged] | 18:44 |
| smoser | wake on lan on those is not reliable unless you're really nice to the driver. | 18:44 |
| negronjl | smoser: this demo will not be very impressive at all | 18:46 |
| smoser | you just have to be really nice, negronjl | 18:46 |
| negronjl | smoser: I will have to "juju bootstrap" then, find out which machine has been allocated to it and power that machine on ... | 18:46 |
| smoser | in the happy path, everything is fine. Daviey or roaksoax might remember more details | 18:47 |
| negronjl | smoser: thx | 18:47 |
| * smoser wonders if negronjl was being sincere in his 'thx' | 18:48 | |
| smoser | in the purely happy path, though, i think it is fine. but if you ever are in the point where you're hard power-offing a mchine, it ends up foobarred. | 18:48 |
| negronjl | smoser: of course ... it's all data that's needed ... not very happy data but, data nonetheless | 18:49 |
| smoser | its not actually terribly odd for wakeonlan to not really be reliable. | 18:49 |
| roaksoax | negronjl: they didn't get you the pdu? | 18:51 |
| roaksoax | i thought they were getting one | 18:51 |
| negronjl | i do have a pdu | 18:51 |
| negronjl | actually ... I have two ... none of them arrived in working condition | 18:52 |
| roaksoax | really? I'm thinking that 1 might be the one that didn't work at all | 18:53 |
| roaksoax | the other one is the new one that should be working | 18:53 |
| roaksoax | negronjl: you will have to configure it though | 18:53 |
| roaksoax | negronjl: and can't you just do this? juju bootstrap --constraints maas-name=natty | 18:53 |
| negronjl | i have tried with them all ... WoL is not happening | 18:54 |
| roaksoax | negronjl: there's a process for wol to work. 1. manually start machine and let it enlist. 2. Accept & Commission and manually start the machine and let it commission | 18:55 |
| negronjl | Have you guys tried the workaround on bug 981461 | 18:55 |
| ubot5 | Launchpad bug 981461 in ifupdown (Ubuntu Precise) "wol broken on HP ProLiant N40L (Broadcom tg3 driver)" [High,Triaged] https://launchpad.net/bugs/981461 | 18:55 |
| roaksoax | negronjl: 3. juju deploy will now work | 18:55 |
| roaksoax | err wol will now work | 18:55 |
| negronjl | roaksoax: i did 1 and 2 ... the machines now show up as ready | 18:56 |
| negronjl | but, when I try juju bootstrap ... nothing happens | 18:56 |
| roaksoax | negronjl: pastebin /etc/cobbler/power/power_ether_wake.template | 18:57 |
| negronjl | k .... give me just a sec | 18:57 |
| negronjl | roaksoax: http://paste.ubuntu.com/1047848/ | 18:59 |
| roaksoax | negronjl: is that the network you are using? /usr/bin/powerwake -b 192.168.1.255 "$power_address" | 19:00 |
| negronjl | roaksoax: yes .. that is the broadcast address ( 192.168.1.0/24 ) | 19:01 |
| roaksoax | negronjl: what if you manually try to WoL a server? | 19:02 |
| negronjl | roaksoax: give me a command and I'll run it. | 19:02 |
| roaksoax | negronjl: powerwake -B 192.168.1.255 <MAC> of one of the ready servers | 19:02 |
| negronjl | roaksoax: trying | 19:02 |
| roaksoax | negronjl: melmoth and alexmoldovan had the same issue last week and the "solutions" above fixed the issue. (allowing them to shutdown cleanly without disconnecting them from the power outlet at all) | 19:03 |
| negronjl | roaksoax: I'll try that too ... I'll report back in a few minutes ... I also have the Ubuntu Contributing Developer meeting now. | 19:04 |
| roaksoax | k | 19:04 |
| burnbrighter | negronjl: out of curiosity, why are you sending to the broadcast address - won't that broadcast that command to all nodes? I don't understand the purpose. | 19:40 |
| negronjl | burnbrighter: not an expert on this but, the magic packet goes out on the wire and the mac address ( included in the command ) will the pick it up and _turn itself on_ | 19:42 |
| burnbrighter | right, but don't you want to direct it to the right IP addr? | 19:43 |
| negronjl | burnbrighter: i _think_ it works by putting the packet on broadcast ... haven't seen it done by direct ip | 19:45 |
| lifeless | burnbrighter: http://en.wikipedia.org/wiki/Wake-on-LAN#Magic_packet | 19:50 |
| burnbrighter | kk, thnx | 19:50 |
| lifeless | can't use IP | 19:50 |
| lifeless | IP isn't available for a powered off machine that uses DHCP, because it isn't holding a lease. | 19:51 |
| burnbrighter | gotcha, thnx for the clarification | 19:51 |
| negronjl | we need a better solution for the WoL thing ... is this an HP issue ? | 19:56 |
| lifeless | negronjl: have you spoken to daviey or the folk that helped mark with this setup ? | 19:57 |
| lifeless | negronjl: I'm worried that we're getting close to the wire, when the setup *was* working :( | 19:58 |
| negronjl | lifeless: I have ... I have random and intermittent issues ... it may work now and it may not .... frustrating | 20:00 |
| negronjl | lifeless: at this point, I am trying to get the WoL part done so I can move on to the juju part of this. | 20:00 |
| lifeless | negronjl: cool; let me know if you need more help from $wherever. | 20:03 |
| lifeless | negronjl: I presume it needed a rebuild / reconfiguration or some such ? | 20:04 |
| roaksoax | burnbrighter: the reasing why the -B BCAST.address was added is because if the server has multiple network interfaces, it won't send the WoL packets through the right interface | 20:05 |
| negronjl | lifeless: I'm not even sure what it needs at this point. | 20:05 |
| * negronjl feels like a good kick is in order :) | 20:05 | |
| roaksoax | negronjl: your best bet would try to make one of those PDU's work | 20:06 |
| negronjl | roaksoax: forgive me if this is a stupid question but, how would a pdu help me whith WoL | 20:08 |
| negronjl | s/within/with | 20:08 |
| roaksoax | negronjl: i meant your best bet would try to make use of the PDU so you don't worry about WoL | 20:08 |
| roaksoax | as WoL is unreliable | 20:09 |
| negronjl | roaksoax: will MaaS be able to control the PDU ? | 20:09 |
| roaksoax | negronjl: MAAS does not control anything in power, cobbler does | 20:09 |
| roaksoax | negronjl: but yes, is it an APC PDU? | 20:10 |
| roaksoax | it is just matter of doing a couple minor changes | 20:10 |
| roaksoax | negronjl: making use of a fence-agent to contorl the PDU in cobbler would be relatively easy | 20:10 |
| negronjl | roaksoax: it is APC PDU | 20:13 |
| roaksoax | negronjl: so yeah, should be fairly easy to use with a fence-agent (from fence-agents package) | 20:14 |
| roaksoax | negronjl: it will just telnet the APC PDU telling it "turn on outleat where machine 'oneiric' is" | 20:15 |
| burnbrighter | roaksoax: is not cobbler going away. Bigjools is making that happen now? | 20:16 |
| roaksoax | burnbrighter: it is | 20:16 |
| roaksoax | burnbrighter: the current version still uses a subset of cobbler | 20:16 |
| roaksoax | err, a few pieces of cobbler | 20:16 |
| burnbrighter | kk | 20:16 |
| negronjl | roaksoax: Let me try to set up the PDU first ... if the process is simple enough, would you help me out with it | 20:17 |
| roaksoax | negronjl: yes, of course | 20:19 |
| negronjl | gotta go get some food ... I'll be back in a minute | 20:23 |
| roaksoax | k | 20:28 |
| * negronjl is back | 21:18 | |
| negronjl | trying one last time the whole WoL thing. | 21:18 |
| negronjl | roaksoax: if you're around .... I'll take you up on the pdu thing for maas | 21:38 |
| negronjl | lifeless: you around ? | 21:40 |
| roaksoax | negronjl: is it configured? | 21:40 |
| negronjl | roaksoax: all nodes are ready | 21:40 |
| negronjl | roaksoax: is what configured ? | 21:40 |
| roaksoax | negronjl: pdu | 21:40 |
| negronjl | roaksoax: no | 21:41 |
| negronjl | roaksoax: didn't want to do anything without talking with you first | 21:41 |
| negronjl | roaksoax: should I flush the entire environment or, is it ok to leave the nodes as they are ( Ready ) | 21:41 |
| lifeless | negronjl: hi, yes, OTP just now, whats up ? | 21:42 |
| roaksoax | negronjl: leave them as they are for now. Get the PDU configured first and make sure it is in working condition | 21:42 |
| negronjl | lifeless: nm ... roaksoax is helping me. | 21:42 |
| negronjl | roaksoax: ok ... I'll ping you when done ... | 21:42 |
| roaksoax | negronjl: and that you can assign the outlets to specific machines | 21:42 |
| roaksoax | negronjl: so just for now, assign 1 outlet and call it the same name what a node in maas would be called | 21:43 |
| roaksoax | i.e. oneiric | 21:43 |
| negronjl | roaksoax: got it | 21:44 |
| roaksoax | negronjl: and in the meantime, sudo apt-get install fence-agents | 21:44 |
| roaksoax | and the command to turn on the oulet would be something like: fence_apc -a <IP of APC> -l <user> -p <password> -n <name of outlet, or outlet ID> -o on | 21:47 |
| negronjl | lborda: are both of the PDUs working ? | 21:54 |
| lborda | negronjl, no | 21:54 |
| negronjl | lborda: want to make sure that I don't spin my wheels on a PDU that may not be working | 21:54 |
| lborda | negronjl, one is broken | 21:54 |
| negronjl | lborda: hmmm... why ship it then | 21:55 |
| negronjl | lborda: ok | 21:55 |
| lborda | negronjl, no idea, I told them not to ship... :P | 21:55 |
| negronjl | lborda: the one that needs the adapter ... is that the one that works ? | 21:55 |
| lborda | negronjl, yes | 21:55 |
| negronjl | lborda: ok .. let's see if i can get the adapter on it ... it'b being difficult | 21:56 |
| lborda | negronjl, yeah... good luck | 21:57 |
| lborda | negronjl, we got the pdu with no adapter... do you have one? | 21:58 |
| negronjl | lborda: now I'm confused ... | 21:58 |
| negronjl | lborda: There is a PDU with a label that say "Broken PDU" ... that one ( oddly enough ) requires no adapter for me to use | 21:59 |
| negronjl | lborda: the other one requires a power adapter ( it's a locking plug ). There is an adapter here but, it doesn't fit. | 21:59 |
| negronjl | lborda: Do you have any idea which one of these PDUs is the "good" one ? | 22:00 |
| lborda | lborda, the "broken pdu" is broken | 22:00 |
| negronjl | lborda: ok | 22:00 |
| negronjl | lborda: ... and I don't have an adapter for the "good" one. | 22:00 |
| negronjl | so ... no PDU then . | 22:00 |
| negronjl | ... and WoL is not working either ... this is just GREAT :( | 22:01 |
| negronjl | roaksoax: You had mentioned that there is a process by which this works. Here is what I have done so far ( in case I am just doing this wrong ): ... | 22:02 |
| negronjl | - Turn machines on. | 22:02 |
| negronjl | - They show up on Maas Web UI as Declared | 22:02 |
| negronjl | - Edit the node ( change power to Wake on LAN ) | 22:02 |
| negronjl | - Accept and Commit | 22:02 |
| negronjl | ---- machines do their thing and shut down | 22:02 |
| negronjl | - They show up as "Ready" on the MaaS Web UI | 22:02 |
| negronjl | roaksoax: Is that the right thing to do so far ? | 22:03 |
| roaksoax | negronjl: they have WoL working by defaulkt, you do not have to manually tell them so | 22:04 |
| negronjl | roaksoax: ok ... noted .... | 22:08 |
| negronjl | roaksoax: now what should I be doing ... | 22:08 |
| negronjl | roaksoax: I tried going to the maas ui, getting the token, putting it in envirnonments.yaml and juju bootstrap | 22:09 |
| negronjl | roaksoax: juju bootstrap --constraints maas-name=quantal ( my node 1 ) | 22:09 |
| negronjl | roaksoax: according to maas, the unit get allocated to admin ( my username ) but, WoL doesn't do anything. | 22:09 |
| negronjl | roaksoax: if I turn the machine on manually that doesn't get me any further ... juju status still hangs. | 22:10 |
| negronjl | roaksoax: atm I am trying again without telling maas that the nodes are WoL ... unless you have any suggestions | 22:10 |
| roaksoax | negronjl: ok so isn't quantal deploying? | 22:11 |
| roaksoax | negronjl: it needs to install ubuntu, install juju, etc etc | 22:11 |
| negronjl | roaksoax: let me try this again. | 22:12 |
| negronjl | roaksoax: once the machines are in "Ready" status ... should I be doing anything else ? | 22:12 |
| roaksoax | negronjl: when a machine is in ready status it is ready to use, they do not have ubuntu on them | 22:13 |
| roaksoax | negronjl: so if youjuju deploy, it will take time for it to install ubuntu on it | 22:13 |
| negronjl | roaksoax: ok ... | 22:13 |
| roaksoax | after it installs, it will reboot | 22:13 |
| roaksoax | and continue the process as if it was a cloud image for juju purposes | 22:13 |
| negronjl | roaksoax: when i juju deploy --constraints maas-name=quantal the node doesn't WoL ... I have to start it up manually | 22:14 |
| roaksoax | that is, install juju and appropriate deps, use a charm (in case of deployment) etc | 22:14 |
| negronjl | roaksoax: I will see if it deploys ( even without WoL ) | 22:14 |
| roaksoax | negronjl: ok, so that means that the machine probably wasn't shutdown cleanly | 22:14 |
| negronjl | roaksoax: k... how do we shut these down cleanly | 22:15 |
| negronjl | roaksoax: I can't log into them | 22:15 |
| roaksoax | negronjl: add an SSH key in juju | 22:16 |
| roaksoax | and you'll have to deploy again | 22:16 |
| roaksoax | err | 22:16 |
| roaksoax | SSH key in MAAS | 22:16 |
| negronjl | trying that ... | 22:17 |
| negronjl | roaksoax: am I supposed to add the SSH key _before_ I accept & commission the machines or can I do it now that the machines are "Ready" | 22:17 |
| roaksoax | negronjl: i can't recall TBH, as there was a bug on it, but for you to not do that, edit "/var/lib/cobbler/kicstarts/maas.preseed" | 22:19 |
| roaksoax | negronjl: and set a password there | 22:19 |
| roaksoax | for the ubuntu user | 22:19 |
| negronjl | roaksoax: ok ... doing that | 22:20 |
| negronjl | roaksoax: done ... should I cobbler sync ?? | 22:21 |
| roaksoax | nope, not necessary | 22:22 |
| negronjl | roaksoax: ok ... trying this thing again ... | 22:22 |
| roaksoax | it will take time to deploy though | 22:23 |
| negronjl | roaksoax: k. I have a monitor/keyboard attached to the node so I can see if it fails anywhere | 22:23 |
| roaksoax | negronjl: ok cool | 22:36 |
| roaksoax | brb | 22:37 |
| negronjl | roaksoax: the PDU is out of the question now ... I'll keep working on this and will ping you back when I get stuck again ( which I'm sure will be soon ). | 22:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!