| basicer | So I have my curtin preseed doing some custom partition stuff, and it works okay, but grub fails to install. | 01:03 |
|---|---|---|
| basicer | Where can I see what commands its trying to run so I can fix it ? | 01:04 |
| === CyberJacob|Away is now known as CyberJacob | ||
| gmb | jtv1: I’ve realised that I forgot to finish reviewing https://code.launchpad.net/~jtv/maas/boot-image-dict/+merge/214210 for you; profuse apologies. I’ll get right on it once I’ve finished this branch. | 07:20 |
| gmb | bigjools: Do we have a sample of what bootresources.yaml should look like in order to pull in HWE kernels? | 07:21 |
| gmb | Just realised that my documentation reads “update bootresources.yaml appropriately” which is a bit like saying “use a goodly measure of spices” | 07:21 |
| bigjools | gmb: it's all in the subarchs | 07:21 |
| bigjools | so amd64/hwe-t for example | 07:22 |
| gmb | bigjools: Thanks. | 07:22 |
| bigjools | gmb: see http://maas.ubuntu.com/images/ephemeral-v2/daily/streams/v1/index.json | 07:22 |
| gmb | bigjools: Ah, perfect! | 07:22 |
| gmb | Excellent typo… “Hardware enoblement” | 07:23 |
| * gmb fixes | 07:24 | |
| gmb | bigjools: Could you use tags to set HWE kernels? | 07:40 |
| bigjools | gmb: no, it's driven entirely from subarchj | 07:41 |
| * gmb is still murky about tag magic | 07:41 | |
| bigjools | "tags are cheesy" | 07:41 |
| gmb | :) | 07:41 |
| gmb | bigjools: https://code.launchpad.net/~gmb/maas/hwe-docs/+merge/214476 | 07:50 |
| gmb | Cheese-free, hopefully | 07:50 |
| bigjools | gmb: cheers, will be on it shortly | 07:51 |
| bigjools | gmb: there;s code at the bottom of the diff | 07:56 |
| gmb | bigjools: Ah, sorry. colo-branch-using fail. | 08:08 |
| gmb | Fixing. | 08:08 |
| gmb | bigjools: Don | 08:10 |
| gmb | e | 08:10 |
| gmb | bigjools: Thanks for the review. All good points. On it now. | 08:21 |
| bigjools | gmb, allenap: https://bugs.launchpad.net/maas/+bug/1302156 | 09:08 |
| ubot5 | Ubuntu bug 1302156 in MAAS "maas upgrade 0072_remove_ipmi_autodetect fails" [Undecided,Incomplete] | 09:08 |
| jtv1 | bigjools: I'm just putting up a fix for bug 1302772. | 09:17 |
| ubot5 | bug 1302772 in MAAS 1.5 "update of maas-cluster-controller on trusty dumps traceback and crashes" [Critical,Triaged] https://launchpad.net/bugs/1302772 | 09:17 |
| === jtv1 is now known as jtv | ||
| bigjools | jtv: you're a star | 09:18 |
| bigjools | jtv: I shall leave one of gmb or allenap to review that, as I am EOD now. Thank you! | 09:18 |
| jtv | np | 09:18 |
| jtv | And I see that gmb assigned me the card. | 09:19 |
| gmb | I’ll review it. | 09:19 |
| gmb | Seems fair | 09:19 |
| jtv | According to the timestamp, about 3 minutes before I ever mentioned it. :) | 09:19 |
| gmb | :) | 09:19 |
| jtv | Thanks! | 09:19 |
| jtv | Did you finish reviewing my other branch? | 09:20 |
| gmb | jtv: I’m on it now. | 09:20 |
| jtv | Ah OK | 09:20 |
| * gmb designates self reviewmeister for this morning. | 09:20 | |
| jtv | Schnell! Schnell! | 09:20 |
| strikov | jtv: hey | 10:08 |
| strikov | jtv: how about extending TestMain to (1) have/pull multiple labels (2) have/pull .gz image to make sure that unpacking works | 10:10 |
| jtv | strikov: Useful in themselves, but would complicate a test that is already quite overweight and potentially brittle... It's always tempting to throw more complications into these tests, but also quite costly. | 10:34 |
| jtv | If we can unit-test these points, great. | 10:34 |
| jtv | (Today's a holiday for me, so not around much) | 10:34 |
| jtv | If we do the unpacking, then we can unit-test it. If it's Simplestreams' job, then it's really something for the Simplestreams tests. | 10:36 |
| strikov | jtv: Okay, thanks, got your point. Enjoy your holiday :) | 10:44 |
| jtv | Thanks! | 10:44 |
| strikov | jtv: I started to work on tests related to image unpacking (we do it, not ss) but met with one serious issue: euc2tgz tool (we use it to convert .img to .tgz) requires root permissions | 12:30 |
| tych0 | hi allenap, so it looks like i do want to trigger the celery job manually on the import process | 13:41 |
| tych0 | i guess you guys must do this for tests and stuff, is there a way to do that? | 13:41 |
| jtv | strikov: ah yes, that tool... one thing you can try is fakeroot. | 13:55 |
| jtv | It's like sudo, but fake. :-) | 13:55 |
| jtv | fakeroot runs a child process that thinks it's root, a lot like sudo, but it doesn't actually get any of the privileges. | 13:55 |
| jtv | tych0: you can just run a celery task (i.e. a function defined with the @task decorator) by calling its delay() method. | 13:57 |
| jtv | (Yes, the functions have methods :) | 13:57 |
| tych0 | jtv: ok, so i just import it and do footask.delay()? | 14:09 |
| tych0 | cool | 14:09 |
| jtv | tych0: Yes. And don't forget, when you need to patch out a celery task for a test, you actually need to patch out its delay() method! | 14:10 |
| tych0 | jtv: yep, we actually want to run this one this time :-) | 14:10 |
| jtv | Ah, that's easier. :) | 14:10 |
| jtv | During tests, celery is wired up to execute tasks synchronously. | 14:11 |
| jtv | In fact we've been sort of doubting the wisdom of that choice — because some of it gets run implicitly during tests for no good reason. | 14:11 |
| webbrandon | any of you ever ran into these when mining: http://paste.ubuntu.com/7217201/ ? I just started a priate network server and all my miners get that, trying to think of why? | 14:25 |
| jtv | Mining..? | 14:26 |
| webbrandon | oops | 14:26 |
| webbrandon | wromg chan | 14:26 |
| jtv | oic | 14:26 |
| jtv | gmb, rvba: I'm backporting my fix for the config bug (where the old "boot" section in pserv.yaml broke the upgrade). No objections? | 14:47 |
| rvba | jtv: sounds good to me. | 14:47 |
| tych0 | hi jtv, i'm getting http://paste.ubuntu.com/7217337/ | 14:51 |
| jtv | tych0: new one on me... one of the guys in Europe may be able to help. | 14:53 |
| tych0 | ok | 14:53 |
| tych0 | rvba: ^^ ? | 14:53 |
| tych0 | or allenap | 14:53 |
| jtv | Could be a celery change. | 14:53 |
| tych0 | i think this is probably something you've always had to do | 14:54 |
| tych0 | i vaguely remember rvba describing to me this before, at least | 14:54 |
| jtv | Hmm yes, it's not a change in celery — it's something we do. | 14:54 |
| rvba | tych0: yes, hang on… | 14:54 |
| rvba | tych0: Wrong celery config. | 14:56 |
| tych0 | ok, how do i pass the right one? | 14:56 |
| rvba | tych0: you have two celery config in /etc/maas, have a look at the upstart script to see what variable you need to define to load the right config. | 14:56 |
| tych0 | ok | 14:58 |
| tych0 | hi rvba, sorry for being slow, but: http://paste.ubuntu.com/7217420/ | 15:13 |
| tych0 | i'm not sure what variables to export exactly, based on my /etc/init/maas-cluster-celery.conf | 15:13 |
| AskUbuntu | MAAS / nodes have limited internet access after install | http://askubuntu.com/q/444563 | 16:03 |
| tych0 | allenap: any thoughts on my question above? | 16:18 |
| allenap | tych0: otp, biab | 16:18 |
| tych0 | allenap: sure, no worries | 16:20 |
| allenap | tych0: Are you setting CELERY_CONFIG_MODULE in the environment before invoking this? | 16:24 |
| allenap | It has a similar approach to Django’s settings module. | 16:24 |
| allenap | tych0: Are you trying to kick off the import job? That can be done via the remote API, but you don’t get any feedback, so I assume you want to run it directly? | 16:26 |
| tych0 | allenap: yeah, viz. the paste above | 16:28 |
| tych0 | i might be doing it wrong, though | 16:28 |
| tych0 | allenap: oh, if we can do that via the api that is good enough | 16:28 |
| allenap | tych0: I was looking at the wrong paste :-/ | 16:28 |
| tych0 | allenap: we are going to run the import directly | 16:28 |
| tych0 | and then run the celery job to do the import | 16:29 |
| tych0 | and then check if it imported everything | 16:29 |
| tych0 | so i guess it would be easiest if we could run it directly so it could fail for us | 16:29 |
| tych0 | but not super important, since it probably (:-) won't fail | 16:29 |
| tych0 | the import job itself is much more likely to fail | 16:29 |
| allenap | tych0: I don’t follow :) Do the import directly then run the celery job to do the import? Do you mean the celery job to report back to the region what’s been imported (report-boot-images or something like that)? | 16:31 |
| tych0 | yep | 16:32 |
| tych0 | the second one | 16:32 |
| tych0 | ideally we'd run that celery job manually too | 16:32 |
| tych0 | because we don't want to be stuck in a wait loop because it broke | 16:32 |
| allenap | tych0: Yeah :-/ Okay, I’ll see if I can figure out why that setting isn’t available. | 16:33 |
| tych0 | allenap: ok, cool, thanks | 16:34 |
| tych0 | i tried looking at celery | 16:34 |
| tych0 | scary stuff :-) | 16:34 |
| allenap | tych0: Instead of /etc/maas in PYTHONPATH, use /usr/share/maas | 16:37 |
| allenap | Or, both. | 16:37 |
| tych0 | allenap: bingo, thank you | 16:38 |
| tych0 | now i get back an AsyncResult object, which isuppose mean things worked? | 16:38 |
| allenap | tych0: /etc/maas/maas_local_celeryconfig_cluster.py will probably only be readable by root:maas, so make sure you’re running as root or in the maas group; you won’t get any warning if it can’t read that module because of permissions. | 16:39 |
| allenap | tych0: Might have :) I don’t know what one of those is. | 16:39 |
| tych0 | http://paste.ubuntu.com/7217779/ | 16:39 |
| allenap | I don’t even know what a result backend is. | 16:41 |
| allenap | rvba: Are you around? ^ | 16:41 |
| lazyPower | greetings maasers. I've run into an interesting issue with maas on trusty server that appeared after I rebooted the machine. http://i.imgur.com/QsOuhYZ.jpg | 16:41 |
| lazyPower | it has to do with iscsi target host, and the tgt bootstrapping, this error was hit before but skipped. Now that i've rebooted the maas region-controller (Which is also the cluster-controller) i now get halted on this during the pxe boot process | 16:42 |
| tych0 | allenap: hmm, ok | 16:42 |
| tych0 | it isn't showing up in the logs, so i don't think it is actually running | 16:42 |
| === deegee__ is now known as drussell | ||
| allenap | lazyPower: I suspect that’s https://bugs.launchpad.net/maas/+bug/1300548. That will be fixed in trusty soon, but if you look at the diff in that change you can see that all you need to do is create a missing symlink (and probably restart tgt). | 16:45 |
| ubot5 | Ubuntu bug 1300548 in maas (Ubuntu) "tgt targets do not persist after a reboot" [Undecided,Fix released] | 16:45 |
| allenap | tych0: Make sure you’ve got both /etc/maas *and* /usr/share/maas in PYTHONPATH. | 16:46 |
| allenap | I don’t think that’ll fix the bug you’re seeing, but it needs to be done anyway. | 16:46 |
| tych0 | allenap: ok, yeah, didn't fix anything :-( | 16:48 |
| lazyPower | aha! i knew i found it but closed the tab, and was unable to refind it. | 16:50 |
| lazyPower | thanks allenap | 16:50 |
| allenap | tych0: I think this is happening because, when you call .delay(), it expects to be able to send that task off to a queue. In MAAS’s case, RabbitMQ. | 17:07 |
| allenap | tych0: You need to find the function to call that actually executes the body of the code. | 17:07 |
| allenap | run() maybe, but that doesn’t seem to do anything here. | 17:08 |
| tych0 | hmm, ok | 17:09 |
| tych0 | just fiddling with somethign else, will look at it in a sec | 17:09 |
| rvba | allenap: tych0: back with a decent internet connection… | 17:10 |
| allenap | tych0: You’re better off just calling maas-import-pxe-files! | 17:11 |
| tych0 | allenap: that's what we're planning to do | 17:11 |
| tych0 | but we need to also trigger the celery job to have it collect the images | 17:11 |
| allenap | That’s all the task in p.tasks.report_boot_images does: calls sudo -n -E maas-import-pxe-files. | 17:11 |
| rvba | tych0: right, you can't use r.failed(): MAAS doesn't store the result of the tasks it sends. | 17:11 |
| tych0 | otherwise we can't create a node | 17:11 |
| tych0 | allenap: yeah, that's what we were doing before | 17:12 |
| tych0 | now maas does things via celery vs. a simple filesystem check | 17:12 |
| tych0 | so we hav eto trigger the celery job too | 17:12 |
| allenap | tych0: Yeah, ignore about the last 120 seconds from me :) | 17:13 |
| allenap | Possibly more :) | 17:14 |
| tych0 | ideally we'd be able to trigger the celery job manually as well | 17:14 |
| allenap | I was getting muddled between report_boot_images and import_boot_images. | 17:14 |
| tych0 | so i ncase that fails we get a non-zero exit code and know to inform the user | 17:14 |
| allenap | rvba: So, how do we execute a task directly? .run? | 17:18 |
| rvba | allenap: task.apply_async() | 17:19 |
| allenap | tych0, rvba: I’ve just realised what’s going to cause an issue. The credentials aren’t in p.cache.cache (fwiw, p.cache.initialize() needs to be called early too). | 17:20 |
| allenap | rvba: Doesn’t apply_async() arrange for the task to be called later? | 17:21 |
| rvba | allenap: yes, that's what I thought you meant. Oh, you meant call the task's code manually? | 17:21 |
| allenap | rvba: Yeah. I think .run() is the badger. | 17:22 |
| allenap | tych0: Getting the credentials into p.cache.cache will result in total hair loss. | 17:23 |
| allenap | report_boot_images() needs them to talk to the API. | 17:23 |
| allenap | tych0: Well… you might be able to fudge it. | 17:24 |
| allenap | tych0: Do you have a one:two:three credentials thing for the API to hand at the point you need to call report_boot_images()? | 17:25 |
| tych0 | yep | 17:25 |
| tych0 | so i can pass those in somehow | 17:25 |
| allenap | tych0: Here goes: | 17:27 |
| allenap | from provisioningserver import cache | 17:28 |
| allenap | cache.initialize() | 17:28 |
| allenap | from provisioningserver import auth | 17:28 |
| allenap | auth.record_api_credentials(“one:two:three”) | 17:28 |
| allenap | from provisioningserver import tasks | 17:28 |
| allenap | tasks.report_boot_images(http_proxy=…) | 17:28 |
| allenap | Watch out for the quotes above; my machine automatically changes them to typographical quotes. | 17:28 |
| tych0 | yeah, saw that | 17:28 |
| tych0 | r.e. the quotes | 17:28 |
| tych0 | allenap: cool, will try now | 17:29 |
| allenap | tych0: Eugh, ignore the http_proxy bit. I am yet again getting muddles. | 17:30 |
| allenap | muddled too. | 17:30 |
| allenap | tych0: I have to go now, but I’ll be back in ~2 hours. | 17:34 |
| tych0 | allenap: ok, it looks like this won't work exactly, but i think i can get it to work | 17:34 |
| tych0 | need to source /etc/maas/celery.conf and some other stuff | 17:34 |
| tych0 | but this is a lot further than i would have gotten myself | 17:34 |
| tych0 | i will let you know what i come up with | 17:35 |
| allenap | tych0: Ah, yes! Source /etc/maas/maas_cluster.conf and export MAAS_URL and CLUSTER_UUID. | 17:35 |
| tych0 | yep | 17:35 |
| tych0 | arg | 17:41 |
| tych0 | 403 forbidden | 17:41 |
| tych0 | http://paste.ubuntu.com/7218034/ | 17:41 |
| Fishy_ | whats it mean when I can't import boot images on a new install, and I get " File "/usr/lib/python2.7/dist-packages/formencode/schema.py", line 145, in _to_python value_dict, state) Invalid: The input field 'username' was not expected." | 17:45 |
| rvba | Fishy_: doesn't look good at all, could you please file a bug with the full stack trace, the version of MAAS you're using, the exact MAAS configuration, and what you've done to get that stacktrace? | 17:50 |
| Fishy_ | okay | 17:51 |
| Fishy_ | fresh install off CD | 17:51 |
| Fishy_ | https://bugs.launchpad.net/maas/+bug/1303935 | 17:59 |
| ubot5 | Ubuntu bug 1303935 in MAAS "unable to import boot images in fresh install" [Undecided,New] | 17:59 |
| rvba | Fishy_: ta. I can't have a look at it right now but we will get back to you quickly. | 18:01 |
| Fishy_ | cool! | 18:03 |
| Fishy_ | im sure its me | 18:03 |
| _bjorne | why im always get this answer in my access.log?! 200 is ok, 404 is error ----> GET /MAAS/metadata//2012-03-01/user-data HTTP/1.1" 404 200 "-" "Python-urllib/2.7 | 18:54 |
| tych0 | hi smoser, what can you tell me about the new import script and tftp timeouts? | 19:23 |
| tych0 | it looks like my /var/lib/maas/tftp is totally empty, i assume it shouldn't be? | 19:23 |
| smoser | that woudl seem bad. | 19:24 |
| smoser | but it might be goign to ta different place. | 19:25 |
| tych0 | smoser: http://paste.ubuntu.com/7218450/ | 19:25 |
| smoser | tych0, that seems plausible | 19:26 |
| tych0 | smoser: ok, so that might work (?) | 19:27 |
| tych0 | i could bomb boot-resources and try again | 19:27 |
| smoser | tych0, id' avhe to look closer, but generally it seems sane. | 19:32 |
| smoser | i dont knwo about tftp timeouts though | 19:32 |
| smoser | you ahve more info there? | 19:32 |
| tych0 | smoser: no, it pxe boots, gets an ip, and then never gets a kernel from tftp | 19:33 |
| tych0 | smoser: is there some more info i can collect somewhere? | 19:33 |
| smoser | does it get the pxe config ? | 19:36 |
| smoser | this is intel ? | 19:36 |
| tych0 | smoser: amd64, no, it doesn't get the pxe config | 19:37 |
| tych0 | PXE-E32: TFTP open timeout | 19:38 |
| tych0 | PXE-M0F: Exiting Intel Boot Agent. | 19:38 |
| AskUbuntu | MAAS Controller & KVM Help | http://askubuntu.com/q/444648 | 19:41 |
| smoser | tych0, is the pserv running ? | 19:42 |
| tych0 | well | 19:42 |
| tych0 | i restarted it but i don't think it came up, actually | 19:42 |
| tych0 | sec | 19:42 |
| tych0 | heh... | 19:43 |
| tych0 | http://paste.ubuntu.com/7218525/ | 19:43 |
| tych0 | upstart's maas-pserv.log | 19:43 |
| smoser | tych0, is it crashing on a tftp request? | 19:48 |
| smoser | or crashing just when run | 19:48 |
| tych0 | smoser: got it figure dout actually | 19:48 |
| tych0 | will file a bug shortly | 19:48 |
| tych0 | smoser: should i still expect things in tgt-admin --show? | 19:57 |
| smoser | tych0, yes | 19:57 |
| tych0 | now iscsistart is timing out | 19:57 |
| tych0 | and there isn't anything there | 19:57 |
| tych0 | (in tgt-admin --show | 19:57 |
| tych0 | ) | 19:57 |
| Jeffrey_ | Good afternoon, I am currently using MaaS 1.5 Trusty Thar and after a fast install using curtin on a node the server doesn't reboot. I was wondering if this is the expected action? I was using a previous MaaS 1.4 Saucy and after a fast install the machine would reboot and then be ready. | 19:58 |
| smoser | console log is the most useful. | 20:06 |
| smoser | if it as trusty ephemeral image | 20:06 |
| smoser | then you will have /var/log/cloud-init-output.log | 20:07 |
| smoser | and /var/log/cloud-init.log , looking for WARN will probably shoul you something | 20:07 |
| Jeffrey_ | smoser: Alright awesome thanks for your help! I'm going to go try some things now. | 20:09 |
| allenap | tych0: How did you get on in the end? | 20:36 |
| tych0 | allenap: https://bugs.launchpad.net/maas/+bug/1302772 was what i was hitting | 20:36 |
| ubot5` | Ubuntu bug 1302772 in MAAS "update of maas-cluster-controller on trusty dumps traceback and crashes" [Critical,In progress] | 20:36 |
| allenap | tych0: Do you have a workaround for that? | 21:01 |
| tych0 | allenap: yeah, just delete that whole section from the config file | 21:02 |
| allenap | tych0: We’ll have a fix for that in time for Trusty, so as long as you can bear it for now it should be fine. | 21:03 |
| tych0 | allenap: yep, that worked | 21:03 |
| allenap | Tip top. | 21:03 |
| === CyberJacob is now known as CyberJacob|Away | ||
| === CyberJacob|Away is now known as CyberJacob | ||
| lazyPower | looks like we have some new fun stuff on the horizon with import-pxe-files, i got an invalid key signature error | 21:20 |
| lazyPower | http://paste.ubuntu.com/7218962/ | 21:20 |
| lazyPower | however, re-running the command completed successfully. Maybe a temporary hiccup | 21:21 |
| tych0 | lazyPower: those images probably aren't signed by the archive key | 21:24 |
| lazyPower | its strange that re-running the command completed without issue. I think it was just a temporary hiccup? | 21:25 |
| tych0 | shoudl be signed by the key in ubuntu-cloudimage-keyring it hink | 21:25 |
| lazyPower | because if its failing crc validation and bailing on the whole process, it should have repeated the same behavior | 21:25 |
| === ubot5` is now known as ubot5 | ||
| === CyberJacob is now known as CyberJacob|Away | ||
| === bigjools_ is now known as bigjools | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
