| bigjools | hi roaksoax and smoser | 00:11 |
|---|---|---|
| smoser | bigjools, hey | 00:11 |
| bigjools | this morning would have been better had my wife not woken me up from the deepest sleep ever | 00:12 |
| smoser | bigjools, :-( | 00:17 |
| smoser | lp:~smoser/+junk/backdoor-image/ | 00:17 |
| smoser | has "backdoor" | 00:17 |
| bigjools | cool | 00:17 |
| * bigjools grabs | 00:17 | |
| smoser | just point it at an image and it will backdoor it | 00:17 |
| smoser | user 'backdoor' | 00:18 |
| bigjools | I don't normally do backdoor | 00:18 |
| smoser | --user bigjools | 00:20 |
| bigjools | :) | 00:20 |
| bigjools | it's the ephemeral image I want in this case, right? | 00:21 |
| smoser | right. | 00:22 |
| smoser | /var/lib/ephemeral/.....disk.img | 00:22 |
| smoser | its probably most proper to 'restart tgt' afterwards | 00:22 |
| smoser | but i've never actually had to do that. | 00:22 |
| smoser | but clearly if it had open filehandles, it could be confusing. | 00:23 |
| bigjools | yeah | 00:23 |
| bigjools | I was going to ask - is it possible to proxy tgt via the clusters? at the moment ephemerals get pulled from the region | 00:23 |
| smoser | i dont know. i think the cluster would just want to run tgt also. | 00:24 |
| smoser | you want that to be as close as possible | 00:24 |
| smoser | its block level io | 00:24 |
| smoser | if i understand you correctly | 00:24 |
| smoser | bigjools, oh. | 00:24 |
| smoser | i thought about one thign that might be screwing you. | 00:25 |
| smoser | i dont think proxy settings or mirrors get sent down to commissioning | 00:25 |
| bigjools | it's just that it's a bottleneck right now - we might need to start copying ephemerals to clusters | 00:25 |
| smoser | right. thats what i was saying. i think you should plan on copying ephemerals to clusters. | 00:25 |
| * bigjools tries commissioning again | 00:25 | |
| smoser | i have to run | 00:25 |
| bigjools | thanks for the script | 00:26 |
| smoser | i've tested it on / | 00:26 |
| smoser | but not actually on an ephemeral image. | 00:26 |
| bigjools | ok :) | 00:26 |
| smoser | i've looked at it though (when pointed at an image) | 00:26 |
| smoser | but on / it added a user that i could ssh in as | 00:26 |
| smoser | so it seemed ok. | 00:26 |
| smoser | later. | 00:26 |
| bigjools | cheers | 00:26 |
| bigjools | oooo | 00:27 |
| bigjools | I saw an error flash by | 00:27 |
| bigjools | something about tty error | 00:27 |
| * bigjools tries to log in | 00:27 | |
| smoser | tty error. | 00:28 |
| smoser | that is strange. | 00:28 |
| bigjools | I think it mentioned stderr too | 00:29 |
| bigjools | Oct 5 00:27:16 10-0-0-100 kernel: [ 25.467047] init: cloud-config main process (899) terminated with status 1 | 00:29 |
| bigjools | Oct 5 00:27:16 10-0-0-100 kernel: [ 25.885676] init: cloud-final main process (1049) terminated with status 1 | 00:29 |
| smoser | copy off /var/log/cloud-init.log | 00:30 |
| smoser | and /var/log/cloud-init-output.log | 00:30 |
| smoser | (you cna proably sudo apt-get install pastebinit) | 00:30 |
| smoser | and do it that way | 00:30 |
| bigjools | ProcessExecutionError: Unexpected error while running command. | 00:30 |
| bigjools | Command: ['locale-gen', 'en_US.UTF-8'] | 00:30 |
| bigjools | Exit code: 1 | 00:30 |
| bigjools | there | 00:30 |
| bigjools | I can scp it then paste, one sec | 00:30 |
| bigjools | smoser: no cloud-init-output file, but here's cloud-init.log: http://pastebin.ubuntu.com/1261098/ | 00:32 |
| bigjools | look around line 246 onwards | 00:33 |
| smoser | bigjools, /var/lib/cloud/instance/cloud-config.txt and /var/lib/cloud/instance/user-data.txt.i | 00:36 |
| smoser | i really have no idea why 'local-gen' would fail like that. but its not deadly. | 00:37 |
| bigjools | cloud-config.txt is empty | 00:37 |
| bigjools | user data has plenty | 00:38 |
| smoser | ah. yeah, it would be user-data. | 00:39 |
| smoser | there is no cloud-config in that. | 00:39 |
| smoser | but user-data has that script. and that *shoudl* get run. | 00:39 |
| bigjools | http://paste.ubuntu.com/1261109/ is its contents | 00:41 |
| smoser | bigjools, something is stopping you from reaching runlevel 3 i think. | 00:44 |
| smoser | so the cloud-final job is not running | 00:44 |
| smoser | bigjools, what does | 00:45 |
| smoser | 'runlevel' | 00:45 |
| smoser | and sudo status rc | 00:45 |
| smoser | show? | 00:45 |
| bigjools | N 2 | 00:47 |
| bigjools | and | 00:47 |
| bigjools | rc stop/waiting | 00:47 |
| bigjools | it must be that tty error | 00:48 |
| bigjools | smoser: ^ | 00:48 |
| smoser | oh wait. that is wierd. | 00:49 |
| smoser | cloud-final ran | 00:49 |
| smoser | try running it again | 00:50 |
| smoser | sudo start cloud-final | 00:50 |
| bigjools | start: Job failed to start | 00:50 |
| bigjools | I can't find any log for it | 00:56 |
| smoser | ok. try it more manually. i guess. | 00:56 |
| smoser | sudo cloud-init modules --mode=final --debug | 00:56 |
| smoser | need --debug before modules | 00:57 |
| bigjools | boom | 00:58 |
| bigjools | well it has a lot of tracebacks but it did complete | 00:59 |
| bigjools | machine powered off | 00:59 |
| bigjools | smoser: so something is wrong with the upstart conf perhaps? | 00:59 |
| bigjools | log: http://paste.ubuntu.com/1261129 | 01:00 |
| smoser | how much memory on the system? | 01:02 |
| smoser | the FALLBACK stff is not really a big deal. its operating as mostly designed. | 01:02 |
| bigjools | 2Gb | 01:02 |
| smoser | the issue is that it logs to rsyslog, but then from within it, the userdata its running calls /sbin/poweroff | 01:03 |
| bigjools | it's an HP cube | 01:03 |
| smoser | and rsyslog gets killed | 01:03 |
| smoser | so logging breaks. | 01:03 |
| smoser | but id ont understand why it wouldnt have run | 01:03 |
| smoser | i have no idea really. | 01:03 |
| bigjools | why does it only happen when console=ttyS0 is specified? | 01:03 |
| smoser | something failed. | 01:03 |
| smoser | it doesn't make any sense to me. | 01:04 |
| bigjools | jeez, 29C at 11am, gonna be hot today | 01:04 |
| bigjools | I'll retry and see if I can see that error on the console, now I know when to look for it | 01:06 |
| bigjools | ah it did it on enlistment too | 01:08 |
| bigjools | stty: standard-input: input/output error | 01:08 |
| bigjools | I am going to file a critical bug | 01:09 |
| smoser | the stty error is maybe not related. | 01:10 |
| bigjools | I am thinking that | 01:10 |
| smoser | can you get dmesg | 01:10 |
| smoser | and basically just tar up | 01:10 |
| smoser | /var/log/ | 01:10 |
| === matsubara is now known as matsubara-afk | ||
| smoser | it just doesn't make sense. | 01:11 |
| smoser | bigjools, now that i think about it the tee /dev/tty2 is probably a bad idea. | 01:11 |
| smoser | as maybe the tee is in this situation too. | 01:11 |
| smoser | since we're /sbin/halting | 01:12 |
| smoser | the tee might get killed and cause unnecessary angst. | 01:12 |
| smoser | i like the log though. | 01:12 |
| smoser | maybe we should have the script do | 01:12 |
| smoser | sh -c 'sleep 10 && /bin/poweroff' & | 01:12 |
| smoser | i can come up with something better too, but basically just amke sure that cloud-init is done. | 01:13 |
| bigjools | smoser: why does "start cloud-final" fail? | 01:14 |
| bigjools | isn't that a hint? | 01:14 |
| bigjools | https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1061977 | 01:14 |
| ubot5 | Ubuntu bug 1061977 in MAAS "Machine fails to commission when console=ttyS0 is present on kernel opts" [Critical,Triaged] | 01:14 |
| smoser | can youg get dmesg | 01:15 |
| smoser | and the /var/log attachd there. | 01:15 |
| bigjools | yeah | 01:15 |
| smoser | it is /dev/console | 01:18 |
| smoser | it completely is that | 01:18 |
| smoser | the fact that running it trhoug upstart died | 01:18 |
| smoser | and then manually did not | 01:18 |
| smoser | because its output will go to /dev/console when you do it via upstart | 01:18 |
| bigjools | what from /var/log do you want? | 01:19 |
| smoser | maybe the kernel has a buffer and just gets fed up at some point. | 01:19 |
| smoser | jsut get it all | 01:19 |
| bigjools | k | 01:19 |
| smoser | if you dont mind. | 01:19 |
| smoser | you didn't have any private data there :) | 01:19 |
| bigjools | haha :) | 01:19 |
| smoser | oh. yeah. | 01:21 |
| smoser | while you're on that machine | 01:21 |
| smoser | can you just try | 01:21 |
| smoser | echo HI MOM | sudo tee /dev/console | 01:21 |
| smoser | and see what it does | 01:21 |
| smoser | i think we're gonna get an input output error basically. | 01:22 |
| bigjools | tee: /dev/console: Input/output error | 01:22 |
| smoser | yeah. so it makes sense. | 01:22 |
| smoser | but i thought the kernel was supposed to be smarter. | 01:22 |
| smoser | you do have 'console=tty1' on the cmdline also, right? | 01:22 |
| bigjools | yes | 01:22 |
| smoser | so there is no physical serial port, right? | 01:26 |
| smoser | just to be sure. | 01:26 |
| bigjools | there isn't - unless the BMC is providing one surruptitiosly | 01:26 |
| smoser | hm.. | 01:28 |
| smoser | interesting. | 01:28 |
| smoser | bigjools, you see kernel output on the monitor, right? | 01:28 |
| bigjools | I do | 01:28 |
| smoser | bigjools, the localegen died because it tried to write to its stdout | 01:30 |
| smoser | bigjools, so i've got to go afk. | 01:39 |
| smoser | but i'd like it if you reviewd my patch for https://code.launchpad.net/~smoser/maas/preserve-sources-list/+merge/127825 | 01:39 |
| bigjools | smoser: ok | 01:39 |
| smoser | particularly, i'm not happy about the test, it seems long winded, but didn't know how to make better. | 01:40 |
| smoser | i dont really know what to do for your ttyS0 issue. | 01:40 |
| smoser | other than not specifying console= at all | 01:40 |
| smoser | which imo seems busted. | 01:40 |
| bigjools | smoser: we have to remove it | 01:40 |
| smoser | as a default. | 01:40 |
| bigjools | busted, but less busted than not booting at all | 01:40 |
| smoser | well, to be fair, you are the first person to see this. | 01:40 |
| bigjools | how much testing has it had though? | 01:41 |
| bigjools | not on VMs | 01:41 |
| smoser | more than a few systems. | 01:41 |
| bigjools | I bet the others all have serial console | 01:41 |
| smoser | well, yes. | 01:41 |
| smoser | and so will the real target audience. | 01:41 |
| smoser | so changing the default (which is hard coded in source code) to accomodate a little toy system | 01:42 |
| smoser | doesn't make a lot of sense to me. | 01:42 |
| bigjools | well hang on | 01:43 |
| bigjools | you don't know that every system will have a serial console | 01:43 |
| smoser | you're right. | 01:45 |
| smoser | but i don't that it generally fails so badly if that is the case. | 01:45 |
| bigjools | kernel bug? | 01:46 |
| smoser | http://www.mjmwired.net/kernel/Documentation/networking/netconsole.txt | 01:48 |
| smoser | i've always wanted to play with that. | 01:48 |
| bigjools | smoser: I'll improve your test and land your branch | 01:54 |
| smoser | thanks. i'll poke the kernel guys tomorrow am. | 01:57 |
| smoser | but i dont have any ideas. | 01:57 |
| === jtv1 is now known as jtv | ||
| smoser | http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg246433.html | 02:11 |
| roaksoax | howdy | 02:39 |
| roaksoax | bigjools: howdy | 02:50 |
| bigjools | roaksoax: hey | 02:54 |
| roaksoax | bigjools: so how do you feel about the packaging? | 02:55 |
| bigjools | roaksoax: positive | 02:55 |
| roaksoax | bigjools: alright, i'm gonna try to upload to archive tomorrow so we can test upgrades from precise | 02:55 |
| bigjools | roaksoax: \o/ | 02:56 |
| roaksoax | and get community to find more bugs if any | 02:56 |
| roaksoax | that's would help us too | 02:56 |
| bigjools | I did a fix the other day to prevent questions when ugprading, fingers crossed :) | 02:56 |
| roaksoax | bigjools: which one is it? | 02:57 |
| bigjools | roaksoax: config for cluster controller | 02:58 |
| bigjools | it looks at DEFAULT_MAAS_URL if it can find it | 02:59 |
| roaksoax | bigjools: the autodetection of maas-url, yeah I saw | 02:59 |
| roaksoax | bigjools: it works | 02:59 |
| bigjools | I did test it :) | 02:59 |
| roaksoax | bigjools: yeah me too, it works :) | 02:59 |
| bigjools | are you still thinking about changing the sed stuff? | 02:59 |
| roaksoax | bigjools: i don't know TBH.. doing so will require .d support for the py's | 03:00 |
| roaksoax | bigjools: and would require a way to provide .d for the yaml conf's too | 03:01 |
| bigjools | I fear that is too much work at this late stage :( | 03:01 |
| roaksoax | bigjools: Indeed!! the easier way right now is simply installing to /usr/share/maas and then symlink everything to /etc/maas/ | 03:01 |
| bigjools | yeah | 03:01 |
| roaksoax | bigjools: however, the problme with that is that user settings won't be preserved on upgrade | 03:02 |
| roaksoax | bigjools: i also found another bug in upgrading but with dbconfig-common, for some reason it was not preserving the password info and was re-writing the config file. | 03:02 |
| roaksoax | the latter is expected, but should have presrved the password, but anyways, i work arounded it by simply sourcing that file and grabbing the password from there instead of letting dbconfig-common to rewrite the config | 03:03 |
| roaksoax | bigjools: now to continue on the config, I think it would be the best for now... will have to check with Daviey to see what's his appreciation on this | 03:04 |
| roaksoax | bigjools: maybe we can simply send the configs in /usr/share/maas and add some code in the configs itself that source whatever is in /etc/maas | 03:04 |
| roaksoax | smoser: still around? | 03:05 |
| bigjools | roaksoax: are you going to land your packaging branch? | 03:33 |
| roaksoax | bigjools: yeah, i'm just testing one more thing | 03:33 |
| bigjools | ok I put the bug back to in-progress :) | 03:34 |
| bigjools | tarmac will set it when the branch lands | 03:34 |
| roaksoax | cool | 03:34 |
| bigjools | roaksoax: I shall force a daily build now | 04:35 |
| roaksoax | bigjools: sure | 04:36 |
| bigjools | being a build admin has its bonuses | 04:37 |
| roaksoax | bigjools: indeed it is | 04:38 |
| roaksoax | i envy you | 04:38 |
| roaksoax | hahah | 04:38 |
| roaksoax | i used to use PPA's extensively | 04:38 |
| * roaksoax bed | 04:40 | |
| bigjools | we bumped the prio permanently on the daily build ppa :) | 04:40 |
| bigjools | nn roaksoax | 04:40 |
| roaksoax | night | 04:40 |
| jam | allenap, rvba: So it turns out that MAASClient doesn't return an object like the Django Client return. It has a 'object.code' vs 'object.status_code' and 'object.read()' rather than 'object.content' | 06:23 |
| jam | so... mocking/stubbing/ bad for the real world :) | 06:23 |
| === lifeless_ is now known as lifeless | ||
| jam | allenap: you lied to me... :), list data is not supported by MAASClient. When it gets down into 'make_payload' you support bytes/unicode/IOBase and callable. | 06:50 |
| bigjools | hey lifeless, any idea wtf is going on here? http://paste.ubuntu.com/1261414 | 07:03 |
| bigjools | I think we have a nasty bug with omshell here :/ | 07:21 |
| bigjools | good grief, I thought I'd seen bad code but this is hideous. http://ftp.fr.netbsd.org/cvsroot/src/usr.sbin/dhcp/common/Attic/parse.c,v | 07:24 |
| lifeless | bigjools: looks like a url to me | 07:30 |
| lifeless | bigjools: maybe it needs quotes ? | 07:30 |
| bigjools | lifeless: I think the base64 parser in omshell blows | 07:30 |
| bigjools | quotes don't help | 07:30 |
| lifeless | bigjools: that is very odd | 07:30 |
| bigjools | looking at the code it treats + and / specially | 07:30 |
| lifeless | oh joy. | 07:30 |
| bigjools | but the code is sooooo bad that it's taking me a while to work out what | 07:31 |
| allenap | jam: Sorry, I didn't realise you were talking about MAASClient. The cli supports multiple values per key, and so does the server side; it's just MAASClient that needs a little love to get it there. | 08:22 |
| jam | allenap: yeah, I got it to work with https://code.launchpad.net/~jameinel/maas/maasclient-multipart/+merge/128176 | 08:31 |
| jam | though it only fixes POST, GET uses urlencode() which just strs the list | 08:31 |
| jam | allenap: is there a good way to generate a huge number of nodes for testing? Like I want to populate the DB with 1000, 10000, 100,000 nodes. | 08:32 |
| allenap | jam: There's a way. I'm not sure about a good way. | 08:33 |
| jam | allenap: well, pressing 'add node' in the web ui is pretty bad | 08:33 |
| jam | I can do it in SQL surgery, but I need to generate mac addresses, etc. for each. | 08:33 |
| jam | So it makes me think I should do it in python, but via the API or just direct on the db | 08:33 |
| jam | but how hard is it to grab maasserver.model objects and play with them | 08:34 |
| jam | I imagine 'settings' needs to be set correctly for me to mutate the db | 08:34 |
| allenap | jam: How about: make syncdb && make harness; from maasserver.testing.factory import factory; factory.make_node() | 08:42 |
| jam | allenap: if make harness will do it, that works for me | 08:42 |
| jam | That was the sort of command I was looking for | 08:43 |
| jam | w7z: mumble? | 09:08 |
| jam | allenap: so what can I do to land the multipart list stuff? I don't actually have a 'mapping' I have an Iterable | 10:40 |
| jam | but string is an Iterable | 10:40 |
| jam | though apparently getattr(s, '__iter__') fails. | 10:40 |
| mgz | there's no sane way to do it without isinstance | 10:42 |
| allenap | jam: If it's not a string type, assume it's iterable of string types, and let things blow up if it's not? | 10:42 |
| jam | allenap: well there are layering issues. The part that checks if it is a string type only returns 1 payload to attach, so you need to loop at a higher level. | 10:42 |
| jam | or change the lower thing to always return a list of payloads | 10:42 |
| jam | or.. | 10:42 |
| allenap | Blast. | 10:43 |
| jam | (It also accepts files) | 10:43 |
| jam | which are iterable, and you want to upload the whole file as one payload | 10:43 |
| allenap | jam: A file isn't a collections.Sequence... but I guess stick with list and put it in the docstring. | 10:44 |
| jam | allenap: I could do "if isinstance(x, collections.Sequence) and not isinstance(x, basestring)" | 10:45 |
| allenap | jam: Yeah. Or change make_payload to gen_payloads, so allowing it to yield multiple things, then all the isSomething checks can be done there. | 10:47 |
| rvba | jam: as I said on the MP, I don't understand ~jameinel/maas/ignore_results, we have a global celery settings for that. | 10:49 |
| jam | rvba: where? (my system at least was improved by nuking the mnesia schema) | 10:49 |
| jam | rvba: if you can point me to it, I'll happily reject/revert my patch. | 10:49 |
| rvba | jam: CELERY_IGNORE_RESULT is set to True in etc/celeryconfig_common.py | 10:49 |
| jam | as it could be me killing things repeatedly. | 10:49 |
| jam | rvba: so right now, I have to install maas-dns in order to do 'make run' is that true? | 10:50 |
| jam | if I don't the service fails to startup properly | 10:50 |
| allenap | jam: Something like http://paste.ubuntu.com/1261655/ | 10:50 |
| jam | and the only log I see is the 'you should install maas-dns' | 10:50 |
| * allenap looks forward to "yield from" | 10:51 | |
| rvba | jam: no, a local named instance is started when you run 'make run. | 10:51 |
| rvba | ' | 10:51 |
| jam | rvba: well, I can't get 'make run' to talk to me, the last error in the terminal is the 'you should install maas-dns on the region controller' | 10:51 |
| jam | I had this problem for quite a while this week. | 10:52 |
| jam | until it magically fixed itself | 10:52 |
| rvba | jam: that message is for the packaged version, I did not bother changing that message in the dev instance. | 10:52 |
| jam | (my best guess at this point, is I got enough queues laying around that rabbit would say 'come back later' when asked to update dns, so it wasn't crashing the startup process.) | 10:52 |
| jam | allenap: I switched to the 'yield' form, care to approve? | 11:13 |
| allenap | jam: Sure. | 11:13 |
| rvba | jam: I'm seeing the dns error too, I'll investigate. But this is only a task failing, it should not cause trouble for the other services. | 11:19 |
| jam | rvba: so I installed maas-dns, which installed maas, I then uninstalled it, but I now have a twistd process runinng. | 11:21 |
| allenap | jam: +1 :) | 11:21 |
| jam | everytime I kill it, it comes back with a new pid | 11:21 |
| jam | I'm guessing upstart is keeping it alive? | 11:21 |
| jam | ah, maas-txlongpoll | 11:22 |
| jam | which didn't get uninstalle | 11:22 |
| jam | d | 11:22 |
| rvba | jam: as I said, you don't have to install maas-dns on a dev instance… also, you'll get into trouble if you install the maas package and try to play with a dev instance at the same time :/ | 11:22 |
| allenap | Gah, so buildout says it's installing testresources 0.2.5, but then uses the system one, 0.2.4. How completely useless. | 11:22 |
| jam | allenap: buildout config as it stands takes system packages over packages it installs. | 11:23 |
| jam | I brought it up earlier | 11:23 |
| jam | as being useless, too. | 11:23 |
| jam | (vs using system packages if there isn't a custom package installed) | 11:24 |
| jam | allenap: so you have to uninstall the system package to have buildout get the right one. | 11:24 |
| jam | and then re-install it in the future when you want to use the system package for some other project. | 11:24 |
| allenap | jam: Ah, right, thanks. At least there's a workaround. | 11:24 |
| jam | rvba: uninstalling package 'maas' is triggering maas-txlongpoll to *start* | 11:26 |
| rvba | jam: ! | 11:26 |
| jam | (which naturally prevents uninstalling maas) | 11:26 |
| rvba | jam: I've spotted one problem: looks like the wrong celeryconfig is loaded on a dev instance (by the region worker at least). | 11:28 |
| rvba | That's why its trying to write to /etc/bind/maas and failing. | 11:28 |
| jam | rvba: I'm also getting failures in "_write_temp_file" in provisioningserver/utils.py | 11:42 |
| jam | (when I create a node, it triggers writing the dns config again, and I get a failure trying to write the temp file) | 11:43 |
| jam | rvba: so it looks like that is the N^2 behavior I was seeing. Every node added is regenerating a full DNS list. | 11:44 |
| jam | (it is failing, but it still is pulling together the data which is O(N) and when you allocate N nodes == O(N^2) by the end) | 11:44 |
| rvba | jam: the DNS writing task should only be triggered with the node gets an IP address. Only then do we need to update the DNS config. | 11:48 |
| rvba | s/with the node/when the node/ | 11:48 |
| jam | rvba: well I'm doing factory.make_node() in a loop | 11:48 |
| jam | which probably gives addresses? | 11:48 |
| rvba | jam: no, there is no lease creation in there as far as I can tell. | 11:49 |
| jam | rvba: maybe it knows the dns config didn't get written properly yet? | 11:49 |
| jam | it is definitely looping on failing to create it. | 11:50 |
| jam | but it may be unrelated to me creating 1k nodes. | 11:50 |
| jam | "Sending due task 'upload_dhcp_leases'" is in the log file an awful lot. | 11:51 |
| rvba | That's a celerybeat task that is run every minute. | 11:51 |
| jam | rvba: I'm seeing it roughly every few seconds | 11:58 |
| jam | however, 'make_node' also creates a new nodegroup if you don't pass one in. | 11:58 |
| rvba | jam: ah! right, that's what triggering all the DNS stuff. | 11:59 |
| rvba | jam: because each nodegroup gets created with a configured interface IIRC. | 12:00 |
| rvba | jam: I've just fixed the "wrong celeryconfig" problem. A stupid mistake in the startup script. | 12:13 |
| jam | rvba: \o/ | 12:13 |
| jam | so creating 1000 nodegroups takes about 15min, but creating 1000 nodes is <1min. | 12:13 |
| jam | that seems reasonable. | 12:13 |
| rvba | jam: The plan is to remove the DNS config pre-population. But post 12.10 release. | 12:14 |
| jam | rvba: sure, but you won't be creating 100,000 nodegroups in the immediate term, so 15min for 1000 seems fine. | 12:15 |
| rvba | jam: right. | 12:16 |
| allenap | rvba: Do you know if the [/+] needs to be on both sides for the [/+]no[/+] bug to manifest, or either side? | 13:20 |
| rvba | allenap: from Julian's investigation (and the mailing list message we found), I think it's on both sides. | 13:21 |
| allenap | rvba: That's one of the weirdest bugs I've ever heard of. | 13:22 |
| rvba | allenap: yeah :/. | 13:22 |
| allenap | rvba: Do you know anything about the code that submits commissioning results (op=signal)? | 13:24 |
| rvba | allenap: not really, maybe I help you with something still? | 13:25 |
| rvba | I can* | 13:25 |
| allenap | rvba: I'm fixing the code that sets the power parameters. Now, it was broken: it was checking for power_type.upper() in map_enum(POWER_TYPES), i.e. checking that the value of power_type given over the wire matched the Python name of the enum item. | 13:31 |
| rvba | allenap: what's wrong exactly? | 13:33 |
| allenap | rvba: Well, that works for IPMI, because the Python name == uppercase(enum value). | 13:34 |
| allenap | rvba: But not for the others. | 13:34 |
| allenap | rvba: I mean, we should be expecting the enum *values* over the wire, right? | 13:35 |
| rvba | allenap: right! | 13:36 |
| allenap | rvba: Okay, I'm not mad then. The reason I want to ask someone about it is to avoid breaking scripts somewhere else. tl;dr this is what tdd looks like without t. | 13:48 |
| rvba | allenap: haha :). I think Julian mentioned IDD recently. | 13:49 |
| === dpb_ is now known as Guest72485 | ||
| rvba | allenap: hm,in his comment in the code, Julian said "if '/' or '+' appear either side of the word 'no'" | 14:02 |
| rvba | allenap: rarg, I've tested it and it's either side indeed, not both sides. | 14:04 |
| allenap | rvba: Gah. Top marks for you though, for confirming. | 14:05 |
| === matsubara-afk is now known as matsubara | ||
| jam | mgz: any progress on search? | 14:10 |
| mgz | jam: getting there | 14:11 |
| allenap | rvba: Do you have any time this afternoon to review the extra changes I had to make to https://code.launchpad.net/~allenap/maas/anon-power-setting/+merge/128127? | 14:13 |
| rvba | allenap: sure, I'll do it right now. | 14:13 |
| allenap | rvba: Thanks. | 14:13 |
| roaksoax | rvba howdy¡¡ i pushed the branch and selected u as reviewer | 14:15 |
| roaksoax | did you see it¿ | 14:15 |
| roaksoax | ? | 14:15 |
| rvba | roaksoax: I did :) | 14:15 |
| roaksoax | rvba cool then :) | 14:15 |
| roaksoax | thanks | 14:15 |
| rvba | roaksoax: I'll get to it in ~30 minutes. | 14:16 |
| roaksoax | rvba awesome thanks | 14:16 |
| roaksoax | allenap thanks for taking care of the power settings for enlistment | 14:17 |
| jam | mgz: so I just did a quick 'wrap process_node_tags in lsprof' and the results are: out of 107s under profiling, 7.8s is parsing and processing the XML, 67s is downloading the hardware details, 32s is uploading the system_ids results, and 2.3s is getting the system_ids list. | 14:20 |
| jam | I think the MAASClient handling of repeated tags is a bit slow (I think it encodes each value as a separate multipart message section. | 14:20 |
| jam | uploading 12,000 values is a lot, but not 32s a lot, given that we can read that many tags in 2.3s. | 14:20 |
| allenap | roaksoax: No worries. It hasn't landed yet... when I picked up that rock I found some bugs :) | 14:21 |
| mgz | so, it is the passing data around overhead... still seems very high | 14:22 |
| jam | And then getting the hardware details is *way* too expensive as well, given if it was a raw DB query, we could get the results in... checking. | 14:22 |
| jam | I get the first result here in about 10s. | 14:23 |
| jam | 32s to get everything. | 14:23 |
| jam | mgz: 32s just to get the content out seems a bit too expensive as well. | 14:23 |
| jam | time xpath_exists is still 6.8s, time in lxml on the current code is only 7.8s. | 14:24 |
| jam | but "select hardware_details from maasserver_node" > /dev/null is 32s. | 14:25 |
| jam | mgz: Is there a lot of quoting that Postgres is having to do? | 14:25 |
| mgz | ...no, but maybe it's re-serialising? it shouldn't need to, but the 9.1 support does seem a little unpolished. | 14:26 |
| jam | mgz: time to select from an xml column into a new text table is 4.1s. | 14:28 |
| mgz | there might be a magic cast that will help? | 14:28 |
| mgz | what does adding ::text do if anything? | 14:28 |
| jam | mgz: time select content from alt_hardware_details >/dev/null => 32s. | 14:29 |
| mgz | gr. | 14:29 |
| mgz | might be sane to just ditch some of the db changes and pull from the original location :) | 14:29 |
| jam | mgz: interestingly, using 'bytea' instead of text is *slower* | 14:31 |
| jam | 1m24s | 14:31 |
| mgz | heh, I think that's probably django | 14:33 |
| jam | mgz: this is in psql | 14:36 |
| jam | no django involved | 14:36 |
| jam | psql -c "select ..." | 14:36 |
| mgz | ...that's very suprising then | 14:36 |
| jam | mgz: so something very strange when 'select DATA from...' is slower than 'select xpath_exists(DATA)' on the same content. | 14:36 |
| jam | mgz: if I alter column set storage plain, it fails because it can't fit the 24kB documents in an 8kb page. | 14:37 |
| mgz | jam: it would make sense were it storing some custom data structure optimised for querying | 14:37 |
| mgz | that's not actually the case though as I understand it, but it's probably doing more work than it needs to for serialisation | 14:37 |
| jam | mgz: maybe, but we've confirmed that lxml can take raw text and parse it with an XPath object in ~the same time. | 14:37 |
| roaksoax | allenap: so the metadata server now imports the method from the maas API? | 14:38 |
| jam | I think the big cost is postgres turning the data into an SQL result. | 14:38 |
| allenap | roaksoax: Yeah, and the implementation has changed quite a lot. | 14:39 |
| jam | mgz: select substring(content from 24000 for 100); is 3.7s | 14:40 |
| jam | so it is all about the number of bytes coming out of psql | 14:40 |
| jam | mgz: -A flag | 14:42 |
| jam | mgz: time psql -A -c "select content" >/dev/null is 1.5s | 14:42 |
| jam | mgz: --no-align :) | 14:42 |
| mgz | :) | 14:42 |
| jam | so psql is iterating all the data, caching it, figuring out how to align it, before outputting it. | 14:42 |
| Daviey | roaksoax: can we remove maas-provision | 14:43 |
| rvba | allenap: I'd like to check my sanity… can you run 'make lint' on trunk? | 14:44 |
| jam | mgz: and potentially psql is operating in 'fixed max mem' mode, and so is spooling to disk to do that work. | 14:44 |
| rvba | allenap: I'm getting: src/apiclient/multipart.py:74: undefined name 'make_payload' | 14:45 |
| jam | but yeah, 1.6s if not aligning, and piping to 'wc -l -c' (if you leave in -w then wc slows things down to 7s looking for word chunks) | 14:45 |
| allenap | rvba: Quite a bit of lint. | 14:45 |
| allenap | rvba: And I see that too. | 14:46 |
| allenap | rvba: I'm free to clean that up. | 14:46 |
| rvba | allenap: cool. Make you can clean the regexp in omshell while you're at it. I know you like regular expressions :) | 14:47 |
| allenap | rvba: As in, the before-or-after thing, or just pretty it up? | 14:47 |
| jam | mgz: ts = time(); [node.hardware_details for node in Node.objects.all()]; td = time() | 14:47 |
| jam | is 6.8s. | 14:47 |
| rvba | allenap: the before-or-after thing :) | 14:47 |
| jam | Which is a fair amount slower than 1.6, but nothing like the 60s we see | 14:47 |
| rvba | allenap: I think it just needs a conditional expression based on a backref… but you know that better than me :) | 14:48 |
| allenap | rvba: Can we check for ([/+]no|no[/+)? | 14:49 |
| jam | mgz: get_hardware_details is spending 67s total, 18s is reading the response and json.loads it. However, 48.7s is 'post' | 14:49 |
| jam | mgz: so I think... MAASClient.post needs some serious poking. | 14:49 |
| roaksoax | Daviey: from the archives? | 14:49 |
| roaksoax | Daviey: i'd say we can | 14:49 |
| roaksoax | and we should | 14:49 |
| rbasak | smoser: are you planning to SRU bug 978127? | 14:50 |
| ubot5 | Launchpad bug 978127 in cloud-init (Ubuntu) "incorrect time on node causes failed oauth" [High,Fix released] https://launchpad.net/bugs/978127 | 14:50 |
| jam | (it is possible that some of the slowdown is the server processing our request, but 24s in dispatch to upload, 58s in 'encode_multipart' | 14:50 |
| jam | allenap: ^^ I'm guessing encoding wasn't tuned for handling 100,000 records, right? | 14:51 |
| smoser | rbasak, i suppose you need it after ephemeral also. | 14:52 |
| smoser | why can't you just get an architecture that doesn't suck, rbasak ? | 14:52 |
| rbasak | smoser: that's the reason I ask, yes. Right now we're still defaulting to a precise ephemeral, and I keep needing to fix the RTC | 14:52 |
| smoser | precise ephemeral should be fixed | 14:52 |
| rbasak | smoser: can you add a bug task for precise, please? I don't ahve permission | 14:53 |
| smoser | if you're still seeing an issue, then the problem is not solved correctly. | 14:53 |
| rbasak | smoser: seeing an issue out of today's maas daily | 14:55 |
| smoser | rbasak, can you show me? | 14:55 |
| smoser | i basically tested this. | 14:55 |
| smoser | by having cloud-init's upstart job first break the clock | 14:55 |
| rbasak | smoser: half an hour please? I just worked around it for a juju test and d-i just started | 14:55 |
| smoser | sure. | 14:56 |
| allenap | jam: Ha, wow. | 14:56 |
| Daviey | roaksoax: right, i will remove it from the archive... but i want to make sure you have an upgrade path | 14:58 |
| Daviey | roaksoax: can you raise a bug requesting removal as it is deprecated etc.. and i'll process it | 14:59 |
| mgz | hm, I should make fail an alias for tail -f, that was a funny tyop | 14:59 |
| roaksoax | Daviey: right, so maas Conflicts/Replaces on maas-provision, which obviously causes it to be removed. However, it not being in the archives would make the transition smoother, wouldn't it? because the packages would simply be completely removed | 15:00 |
| rvba | allenap: I think so yeah. All I know about this problem is summarized right here ;) : http://paste.ubuntu.com/1262047/ | 15:00 |
| jam | allenap: apparently 30s of the 80s is in 'set_type' | 15:01 |
| allenap | jam: That's... weird. Maybe it's recoding things at that point. | 15:02 |
| jam | allenap: well, it is under lsprof, so some things are more expensive than they are in 'real life'. I'm doing a quick test of casting the strings to bytes rather than unicode | 15:02 |
| jam | see if that has a big difference | 15:02 |
| jam | since bytes payloads don't have set_type called | 15:03 |
| rvba | allenap: I've got a tiny branch up for review if you have time: https://code.launchpad.net/~rvb/maas/big-networks/+merge/128269 | 15:03 |
| jtv | Trunk seems to be failing tests and have lint in it. :( | 15:03 |
| allenap | jtv: I'm fixing those now. | 15:03 |
| jtv | Ah | 15:03 |
| rvba | roaksoax: did you put your branch up for review? I don't see it in the "active code reviews" list. | 15:04 |
| jam | allenap: interestingly, rabbit can take ~1 minute from the time I put something in the queue, before it actually triggers on the provisioning worker | 15:04 |
| roaksoax | rvba: I change the status to work in progress | 15:04 |
| rvba | roaksoax: ok, got it. | 15:05 |
| allenap | jam: That's not good. | 15:05 |
| jam | allenap: (the background is, we can run xpath_exists() in the database taking ~7s to run, or we can dump the raw xml out in about 1.5s, but it takes 42s to read the data via APIS and upload the results back) | 15:06 |
| jam | so a 6x overhead | 15:06 |
| jam | and we only spend 7.8s in etree.XPATH() | 15:06 |
| jam | allenap: as for what causes the rabbit slowdown, I'm not sure. It looks a little like it is recoverying from trying to write the dns config or something. | 15:08 |
| jam | (waiting for celery-beat?) | 15:08 |
| smoser | rbasak, is there anything you're aware of that you'd like in SRU not on https://bugs.launchpad.net/ubuntu/precise/+source/cloud-init | 15:11 |
| mgz | okay, sorted apart from what to do with InvalidConstraint... does the view somehow need to catch it and do something clever... atm you just get a "we broke" error page which is not acceptable for a typo | 15:13 |
| flacoste | roaksoax: do you have everything you need for IPMI in enlistment? | 15:15 |
| roaksoax | flacoste: yes, just waiting for it to land | 15:15 |
| flacoste | roaksoax: right, ok | 15:16 |
| roaksoax | thanks :) | 15:17 |
| Daviey | roaksoax: state of bug 1052056 ? | 15:17 |
| ubot5 | Launchpad bug 1052056 in freeipmi (Ubuntu) "[FFe] [MIR] freeipmi" [Undecided,In progress] https://launchpad.net/bugs/1052056 | 15:17 |
| roaksoax | Daviey: need to address the unused variable compiler warnings, but haven't really have the time to investigate how to do it since I have been pretty much swamped with the other stuff | 15:19 |
| Daviey | ok, thanks | 15:19 |
| Daviey | roaksoax: do you have your latest debdiff? | 15:19 |
| roaksoax | Daviey: yes, hold on | 15:20 |
| Daviey | thanks | 15:20 |
| roaksoax | Daviey: http://paste.ubuntu.com/1262079/ | 15:21 |
| Daviey | thanks | 15:21 |
| jam | allenap: well, lsprof says that set_type() spends its time calling get_params set_param, __delitem__ | 15:22 |
| jam | which appears to have to encode/decode the params | 15:22 |
| jam | however, it is an lsprof-ism, real-world time is 42s => 41.5s using 'bytes' instead of 'unicode', but lsprof time is 109 vs 85s | 15:27 |
| jam | so... ignore that one. | 15:27 |
| rvba | roaksoax: http://paste.ubuntu.com/1262102/. One question: why use the absolute path '/usr/sbin/ipmi-chassis-config' instead of 'ipmi-chassis-config'? I thought Scott said it was a bad thing to do… | 15:34 |
| roaksoax | rvba: following what was done before me | 15:34 |
| rvba | Fair enough :) | 15:34 |
| roaksoax | rvba: however, i think usr/sbin is not in the path, and hence there was a problem executing the scripts, I don't know if yourecall having us discuss something like that | 15:35 |
| rvba | roaksoax: yeah, it rings a bell. | 15:35 |
| rbasak | smoser: I'm not so bothered about bug 1028501 any more, since MAAS doesn't need it to work any more | 15:36 |
| ubot5 | Launchpad bug 1028501 in cloud-init (Ubuntu Precise) "cloud-init selects wrong mirrors for arm" [High,Triaged] https://launchpad.net/bugs/1028501 | 15:36 |
| rbasak | smoser: I think that means that bug 978127 is the only one I care about in cloud-init for precise. Though I'm worried that I'm missing something else. | 15:37 |
| ubot5 | Launchpad bug 978127 in cloud-init (Ubuntu Precise) "incorrect time on node causes failed oauth" [High,Triaged] https://launchpad.net/bugs/978127 | 15:37 |
| roaksoax | rvba: btw.. celeryconfig.py and celeryconfig_cluster.py are not meant to be modified by the user right? | 15:37 |
| rvba | roaksoax: no. Only maas_local_celeryconfig_cluster.py and maas_local_celeryconfig.py should be modified by the user. | 15:38 |
| roaksoax | rvba: not even celeryconfig_common right? | 15:38 |
| rvba | roaksoax: no. | 15:38 |
| smoser | rbasak, did your install go? can i see failed oauth now? | 15:45 |
| rbasak | smoser: having trouble getting to the point where I can reproduce. I set the clock back to 1970, and now it can't add my PPA for maas-enlist to work around the SRU not being in yet | 16:05 |
| rbasak | Changing the hardware clock is a little bit tedious | 16:05 |
| rbasak | I have to install some usable OS first, since there's no "recovery disk" | 16:06 |
| smoser | boot the ephemeral image. | 16:06 |
| smoser | silly. | 16:06 |
| rbasak | ...which I can't log in to | 16:06 |
| smoser | http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image | 16:06 |
| rbasak | and yes, there are ways round it | 16:06 |
| smoser | use that to add a user 'backdoor'. | 16:06 |
| rbasak | The easiest way round it is to install a usable OS | 16:06 |
| smoser | i think that script is easier. | 16:07 |
| smoser | personally | 16:07 |
| smoser | ./backdoor-image /var/lib/ephemeral/......./disk.img | 16:07 |
| smoser | then ssh in | 16:07 |
| smoser | or login on console. | 16:07 |
| rbasak | What about disabling poweroff? | 16:08 |
| rbasak | Another step to do | 16:08 |
| smoser | this is true. | 16:08 |
| smoser | maas needs a "rescue" environment. | 16:09 |
| rbasak | I'm going to make an armhf recovery initrd when I get round to it | 16:09 |
| smoser | cirros is probably really close | 16:09 |
| smoser | to what you ened there. | 16:09 |
| smoser | and if you ever do "get round to it" i'd rather you fix cirros to work for you. | 16:09 |
| rbasak | I was going to base it on ubuntu core | 16:11 |
| rbasak | Which I think might Just Work out of an initrd | 16:11 |
| rbasak | Need to test it though | 16:11 |
| smoser | well, fo ryou particular use case here, you can probaly just boot the existing initramfs and kernel | 16:13 |
| smoser | and pass 'break' on the cmdline | 16:13 |
| smoser | if you have console access. | 16:13 |
| smoser | if not then you need ssh or the like. | 16:13 |
| rbasak | roaksoax: are you planning on landing a fix for the ipmi_si module parameters today? If not I need to file a bug. | 16:19 |
| roaksoax | rbasak: that's actually what i'm doing now | 16:19 |
| rbasak | roaksoax: OK no problem. I'll leave it then. | 16:19 |
| rbasak | smoser: OK, reproduced finally | 16:30 |
| rbasak | smoser: now to get you in | 16:30 |
| rbasak | smoser: which I presume will involve running your backdoor script ;-) | 16:30 |
| roaksoax | rvba: updated the branch, ready for final review. Thanks a lot for working on it | 16:40 |
| rvba | roaksoax: np. Branch approved. | 16:46 |
| roaksoax | allenap: are you gonna make changes to the anonymous power settings for enlistment or will you land it? | 17:07 |
| roaksoax | rvba: we don't want users to modify the maas-http.conf either right? | 17:11 |
| rvba | roaksoax: well, if they want to serve the site over HTTPS they will have to change maas-http.conf. | 17:12 |
| roaksoax | rvba: ok, so i'll leave it as is | 17:13 |
| roaksoax | rvba: btw.. https://jenkins.qa.ubuntu.com/job/maas-merger-quantal/127/console | 17:13 |
| rvba | Yeah, I've seen. It seems to be related to the changes jam checked in. | 17:13 |
| roaksoax | boomer | 17:14 |
| rvba | But it's really just a problem in the tests I think. | 17:14 |
| roaksoax | yeah but that will be holding off all MP's | 17:14 |
| rvba | Like… the tests were simply not updated. | 17:14 |
| rvba | Good point. | 17:15 |
| rvba | hm, I wonder how I could even land a fix then :) | 17:15 |
| rvba | I guess I just merge manually. | 17:15 |
| roaksoax | rvba: a fix won't make the jenkings job fail, so it would land the branch | 17:17 |
| rvba | ah right. | 17:17 |
| roaksoax | rbasak: | 17:25 |
| roaksoax | https://code.launchpad.net/~andreserl/maas/maas_commissioning_modprobe_params/+merge/128294 | 17:26 |
| rbasak | roaksoax: approved, thanks! | 17:28 |
| roaksoax | allenap: re bug 1059168 | 17:32 |
| ubot5 | Launchpad bug 1059168 in MAAS "MAAS should tell IPMI to PXE once" [High,Triaged] https://launchpad.net/bugs/1059168 | 17:32 |
| roaksoax | allenap: basically, each team we tell a machine to turn on, it will tell it to PXE boot | 17:33 |
| roaksoax | allenap: we don't have to manually configure the BIOS and tell it to PXE boot | 17:33 |
| roaksoax | allenap: it will do it on demand | 17:33 |
| roaksoax | allenap: sabdfl's request | 17:33 |
| roaksoax | s/team/time | 17:34 |
| rvba | roaksoax: I'm fixing the build now. The fix should land shortly. | 17:34 |
| roaksoax | rvba: awesome thanks | 17:34 |
| Daviey | roaksoax: maas is using squashfs by default now? | 17:48 |
| roaksoax | Daviey: for quantal yes | 17:49 |
| Daviey | roaksoax: and juju deploy precise uses old method, right? | 17:50 |
| roaksoax | Daviey: yes | 17:50 |
| roaksoax | Daviey: there are no squashfs images for precise, are they? if there are we could enable it oo | 17:50 |
| Daviey | there are not | 17:51 |
| Daviey | just wanted to check it worked | 17:51 |
| roaksoax | alright :) | 17:51 |
| roaksoax | smoser: around? | 17:58 |
| smoser | here | 17:59 |
| roaksoax | smoser: so, matsubara just pinged me about the commissioning stuff for IPMI since he mentioned that the cards lost its static address | 17:59 |
| roaksoax | smoser: since we are working with the assumption | 17:59 |
| roaksoax | smoser: that IPMI will alwas DHCP by design | 18:00 |
| roaksoax | smoser: so i think we need to address the fact that some people don't wan't to DHCP their IPMI cards, and will pre-configure them | 18:00 |
| roaksoax | smoser: what do you think? | 18:00 |
| roaksoax | matsubara: ^^ | 18:00 |
| smoser | roaksoax, i thought you were wokring under that assumption. | 18:01 |
| smoser | i thought if the card had an IP you reported it. | 18:01 |
| smoser | right? | 18:01 |
| roaksoax | smoser: nope not really. The assumption I was told to work bsaed on was if the IUPMI card is set Static addres network source, we should change it to DHCP | 18:02 |
| roaksoax | smoser: because it could be pre-configured | 18:02 |
| smoser | ah. | 18:03 |
| smoser | well i like your new suggestion better. | 18:03 |
| smoser | i'd say if it is set to static, and appears to be configured that you should leave it as such. | 18:03 |
| smoser | ideally such things are exposed in maas configuration | 18:03 |
| smoser | but... its a bit late for that i think | 18:04 |
| roaksoax | smoser: right, so I was thinking on simply passing a parameter such as IPMI_DHCP="yes" in commissioning_user_data | 18:04 |
| roaksoax | smoser: and if so, send --use-static regardless of what it is | 18:04 |
| roaksoax | to the command | 18:05 |
| roaksoax | rvba: another build error :( https://jenkins.qa.ubuntu.com/job/maas-merger-quantal/129/console | 18:06 |
| rvba | roaksoax: I've seen. Looks spurious to me. | 18:06 |
| matsubara | roaksoax, that'd be helpful. I'm setting up dhcp in the lab ipmi network, but it'd be easier to just use the static ip already configured | 18:07 |
| jam | rvba: so I'm pretty sure I did land the change for '?op=' but I'm also surprised that it would have landed if tests would then be failing. I can land a fix if you haven't already | 18:08 |
| rvba | jam: doing it right now: https://code.launchpad.net/~rvb/maas/fix-broken-build/+merge/128297 | 18:09 |
| roaksoax | smoser: something like this: http://paste.ubuntu.com/1262416/ | 18:12 |
| roaksoax | smoser: or better yet, enable it by default, so if the card is set to static, just use that IP address | 18:16 |
| smoser | it looks reasonable. | 18:16 |
| roaksoax | alright, i'll get that done then | 18:16 |
| smoser | maybe IPMI_CHANGE_STATIC_TO_DHCP=false | 18:17 |
| smoser | you did put the ipmi header there, but the name 'use_static' could mean so many things. | 18:17 |
| roaksoax | smoser: right :). Will do change that for something more appropriate | 18:19 |
| rvba | roaksoax: the fix has finally landed! | 18:28 |
| roaksoax | rvba: awesome! | 18:29 |
| smoser | allenap, is this known: | 18:52 |
| smoser | http://paste.ubuntu.com/1262473/ | 18:53 |
| smoser | http://paste.ubuntu.com/1262477/ | 18:55 |
| smoser | it was known. fixed in 1185 | 19:02 |
| roaksoax | matsubara: could you please configure one of your IPMI cards to static, and apply this patch to the commissioning_user_data conffile http://paste.ubuntu.com/1262508/ | 19:09 |
| roaksoax | matsubara: and re-enlist, and re-commission and see if it works (leaves it as DHCP and obtains the right address) | 19:09 |
| matsubara | roaksoax, sure, using the latest package: maas-0.1+bzr1170+dfsg-0+1192+117~ppa0~quantal1, I take? | 19:12 |
| smoser | roaksoax, anyone know anythign about "failed to upload?" | 19:14 |
| smoser | well, never mind. looks like we have one. | 19:15 |
| roaksoax | smoser: huh? failed to upload where? | 19:21 |
| roaksoax | matsubara: yeah | 19:21 |
| matsubara | ok, waiting it to be published | 19:23 |
| roaksoax | matsubara: ok, it doens't really matter what version of maas as long as it is greater than bzr1170 | 19:24 |
| roaksoax | matsubara: so you can just patch the file | 19:24 |
| matsubara | ah ok | 19:24 |
| roaksoax | smoser: for enlistment, do we have a metadata file we can pass same as commissioning? | 19:32 |
| smoser | user data for enlistment? | 19:33 |
| smoser | contrib/preseeds_v2/enlist_userdata | 19:33 |
| roaksoax | smoser: ah right lol, but we are pretty much going to use the same script for enlistment, commissioning | 19:38 |
| roaksoax | smoser: so it should probably live in a common place | 19:38 |
| roaksoax | smoser: any thoughts? maybe in a package? | 19:39 |
| roaksoax | cause i was just thinking on shipping it with maas-enlist | 19:40 |
| smoser | roaksoax, thats not a bad idea. | 19:41 |
| roaksoax | smoser: i'll make another ipmi package | 19:43 |
| roaksoax | err | 19:43 |
| roaksoax | another binary package for maas-enlist | 19:43 |
| smoser | ? | 19:43 |
| roaksoax | smoser: maas-commissioning | 19:43 |
| smoser | why separate? | 19:43 |
| smoser | due to dependencies? | 19:43 |
| roaksoax | smoser: becuase maas-enlist pulls curl, archdetect-deb, libavahi-core7, libavahi-common3 | 19:43 |
| roaksoax | yeah | 19:43 |
| smoser | good enough. | 19:43 |
| matsubara | roaksoax, enlisting now with your patch | 19:45 |
| roaksoax | matsubara: alright, enlistment wont be affected, only commissioning, but we need to enlist first so that commissioning changes take effect | 19:46 |
| matsubara | roaksoax, of the 4 nodes I booted, 3 are static and 1 is dhcp | 19:46 |
| roaksoax | smoser: oh btw... if the node is enlisted, and I make changes to commissionining-user-data they don't take effectd | 19:46 |
| roaksoax | matsubara: ok cool | 19:46 |
| roaksoax | matsubara: that should test it well | 19:46 |
| roaksoax | smoser: i have to re-enlist | 19:47 |
| smoser | roaksoax, relaly? | 19:47 |
| smoser | that is really bad. | 19:47 |
| roaksoax | smoser: yep | 19:47 |
| smoser | please open a bug for that. | 19:47 |
| matsubara | roaksoax, ok. the nodes are declared | 19:54 |
| roaksoax | matsubara: ok, now commission them please | 19:54 |
| matsubara | roaksoax, now I enter the ipmi configuration normally and commission? | 19:54 |
| roaksoax | matsubara: no | 19:55 |
| roaksoax | matsubara: don't enter IPMI, let it commission | 19:55 |
| roaksoax | matsubara: and see if the IPMI gets set | 19:55 |
| roaksoax | and not changed to DHCP | 19:55 |
| roaksoax | matsubara: so power them on manually once you've accepted&commission | 19:55 |
| matsubara | ok. they're on | 19:56 |
| roaksoax | matsubara: did you ifle a bug for the juju issue with the CPU count? | 19:58 |
| roaksoax | matsubara: do you know if it's been fixed? | 19:58 |
| matsubara | yes, i filed | 20:01 |
| matsubara | didn't test yet but i think so | 20:01 |
| roaksoax | matsubara: what's the bug number | 20:01 |
| matsubara | bug 1061286 | 20:02 |
| ubot5 | Launchpad bug 1061286 in juju "juju bootstrap returned ERROR Invalid 'cpu_count' constraint '1.0'" [Medium,Fix committed] https://launchpad.net/bugs/1061286 | 20:02 |
| matsubara | roaksoax, ok, nodes are ready | 20:02 |
| matsubara | and no IPMI config was set | 20:02 |
| roaksoax | matsubara: jum | 20:03 |
| roaksoax | matsubara: check in the commissioning-user-data that modprobe ipmi_si has arguments | 20:03 |
| matsubara | modprobe ipmi_si type=kcs ports=0xca2 | 20:04 |
| roaksoax | smoser: my bad | 20:04 |
| roaksoax | smoser: it does work | 20:04 |
| roaksoax | smoser: as expected | 20:04 |
| roaksoax | i wonder why i thouhght it didnt update | 20:04 |
| smoser | hey all | 20:53 |
| smoser | http://bazaar.launchpad.net/~smoser/maas/maas-pkg-test/view/head:/maas-ephemeral-test-quantal.txt | 20:53 |
| smoser | was mostly functional walk through of ppa test for me | 20:54 |
| smoser | i now do not have to touch the UI by hand. | 20:54 |
| smoser | whoowhoo | 20:54 |
| roaksoax | smoser: what do you think? http://paste.ubuntu.com/1262708/ | 20:56 |
| smoser | that looks reasonable to me, roaksoax | 20:59 |
| smoser | i'm out. | 21:01 |
| roaksoax | smoser: hold on | 21:01 |
| roaksoax | :/) | 21:01 |
| smoser | k | 21:01 |
| roaksoax | smoser: give me one min | 21:01 |
| smoser | holding | 21:01 |
| roaksoax | for you to approave | 21:01 |
| roaksoax | smoser: https://code.launchpad.net/~andreserl/maas/commissioning_improvements/+merge/128318 please :) | 21:02 |
| smoser | did you test it? | 21:02 |
| roaksoax | smoser: yes | 21:03 |
| smoser | you asked for review of "launchpad code reviewers" ? | 21:04 |
| smoser | what is doing that. | 21:04 |
| roaksoax | smoser: it does that automatically | 21:05 |
| roaksoax | smoser: julian (i think) changed it that way | 21:05 |
| smoser | well that is busted. | 21:07 |
| smoser | anyway. | 21:07 |
| smoser | i'm out. | 21:07 |
| smoser | and i revierwed. | 21:07 |
| roaksoax | smoser: awesome, thank you. have a good weekedn | 21:07 |
| roaksoax | jtv: if you are making changes to tx-tftp, please make htem in the ubuntu package in archive sa well | 21:37 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!