| === Salty is now known as Guest45701 | ||
| === disposable3 is now known as disposable2 | ||
| === mimizone_ is now known as mimizone | ||
| === marlinc_ is now known as marlinc | ||
| benlake | I’m apparantly missing something regarding where to access the API. Is it provided on the same port as the WebUI (I’d assume so)? /api/version/ and /api/2.0/ both return “No Such Resource" | 15:18 |
|---|---|---|
| benlake | whelp. Just figured it out, /MAAS/api/version/ | 15:19 |
| benlake | that’s a tad unexpected. Had the impression /MAAS/ and /API/ would be respective UI and API roots. | 15:21 |
| benlake | The docs do not mention the MAAS_URL is the prefix. | 15:21 |
| pmatulis | benlake, please provide the URL to the doc you followed | 15:34 |
| benlake | https://docs.ubuntu.com/maas/2.1/en/api | 15:35 |
| pmatulis | benlake, ok, interesting. the intended page to get started with the CLI/API is: | 15:36 |
| pmatulis | https://docs.ubuntu.com/maas/2.1/en/manage-cli | 15:36 |
| benlake | err, I wanted to get started with the API. | 15:37 |
| pmatulis | we try to direct people at this top-level page: | 15:37 |
| pmatulis | https://docs.ubuntu.com/maas/2.1/en/intro-management | 15:37 |
| pmatulis | but i can see how you can end up directly on the API page | 15:37 |
| benlake | API Documentation is a giant top level button :D | 15:38 |
| pmatulis | right | 15:38 |
| benlake | I’m not using the CLI tool, I was looking for, and found, the API docs :P | 15:38 |
| benlake | which are great. Only trip was seeing all the /api roots paths, and not realizing that was in addition to whatever the configured MAAS_URL is | 15:39 |
| pmatulis | the API docs are generated from the maas code. i don't have any influence there | 15:41 |
| pmatulis | roaksoax, possible to add some clarity? ^^^ | 15:41 |
| roaksoax | benlake: cli is just a reflection of the API | 15:42 |
| roaksoax | benlake: it is auto generated | 15:42 |
| roaksoax | benlake: MAAS_URL is a config on /etc/maas/rackd.conf | 15:42 |
| benlake | I’m confused by what you’re attempting to convey | 15:42 |
| benlake | why is the CLI relevant to wanting to access the API directly and discoverying how? | 15:43 |
| benlake | and yes, I realize where the MAAS_URL is, it was just not apparent it needed to be the prefix to the paths referenced by the API docs. | 15:43 |
| benlake | the service provided on 5240 could provide both /api/ and /maas/ | 15:45 |
| benlake | nothing is wrong. I was just commenting that I tripped over /api needing to be /maas/api | 15:45 |
| pmatulis | benlake, in passing, for any general documentation issues you can open them here: | 15:46 |
| pmatulis | https://github.com/CanonicalLtd/maas-docs/issues/new | 15:46 |
| benlake | yeah, the threshold for making paperwork is tricky. | 15:47 |
| benlake | for a minor trip up that could easily be an expectation of mine simply not being the case; I leaned toward saying it here and seeing if it’s worth the issue. | 15:48 |
| benlake | thanks for he ref. | 15:48 |
| pmatulis | benlake, your comment is valid IMO. and IRC is fine to discuss. i really meant for any future issues | 15:51 |
| benlake | if I make interfaces changes to a controller, is there a way to get MAAS up to speed? Are the fabric references tied to interface names internally? | 15:56 |
| benlake | are PPAs added via MAAS available through the proxy? | 16:40 |
| * benlake is guessing not since Package Repositories says “…subsequently deployed” | 16:41 | |
| mup | Bug #1694759 opened: RSD Pod refresh shows ComposedNodeState is "Failed" <MAAS:Confirmed for newell-jensen> <MAAS RSD :Confirmed for newell-jensen> <https://launchpad.net/bugs/1694759> | 17:02 |
| mup | Bug #1694761 opened: Invalid Pod Resources error shows in API but not UI <MAAS:Confirmed for blake-rouse> <https://launchpad.net/bugs/1694761> | 17:17 |
| mup | Bug #1694761 changed: Invalid Pod Resources error shows in API but not UI <MAAS:Confirmed for blake-rouse> <https://launchpad.net/bugs/1694761> | 17:26 |
| benlake | Hmm, are the MAAS NTP settings used in the PXE boot images? I’m seeing ntp.ubuntu.com used during an attempted Release operation. | 17:31 |
| benlake | I’m also seeing massing ureadahead errors, but I think they’ve been there for a while. | 17:32 |
| benlake | s/massing/massive/ - by massive I mean like 29k+ entries in syslog | 17:32 |
| mup | Bug #1694761 opened: Invalid Pod Resources error shows in API but not UI <MAAS:Confirmed for blake-rouse> <https://launchpad.net/bugs/1694761> | 17:32 |
| mup | Bug #1694767 opened: RSD composition not setting local disk tags <MAAS:Confirmed for newell-jensen> <MAAS RSD :Confirmed for newell-jensen> <https://launchpad.net/bugs/1694767> | 17:33 |
| roaksoax | benlake: the ureadahead errors are being looked at, non-maas issue though | 17:48 |
| roaksoax | benlake: that's a bug in ureadahead | 17:48 |
| benlake | thanks for the confirmation | 17:48 |
| roaksoax | benlake: as far as the proxy, it is a cashing proxy , so your ppas would still use the default proxy | 17:49 |
| roaksoax | benlake: on NTP, the commissioning environment will configure ntp | 17:49 |
| benlake | neat @ proxy | 17:49 |
| benlake | Re: ntp, not seeing that happen | 17:49 |
| roaksoax | benlake: go to /etc/ and grep for the expected config | 17:50 |
| roaksoax | i'd need to check | 17:50 |
| roaksoax | benlake: unless it is systemd timesync that's doing those queries | 17:50 |
| benlake | yeah, its systemd | 17:51 |
| benlake | I’m seeing these things, which are likely not related at all, because a Release is failing (multiple tries) | 17:52 |
| benlake | I’ve deployed/released this node a few times over the last few days, and now it’s hung | 17:52 |
| benlake | are there any telltales in the logs that could confirm for me the booted envronment is/will attempt to erase didks? | 17:53 |
| benlake | also disks | 17:53 |
| roaksoax | benlake: did the machine enter disk erasing mode and started erasing ? | 17:56 |
| benlake | yes, then timed out | 17:56 |
| roaksoax | benlake: you should see progress in the events log | 17:56 |
| roaksoax | benlake: timeout when ? | 17:57 |
| benlake | while waiting for disk erase to complete | 17:57 |
| benlake | it never timed out before... | 17:57 |
| roaksoax | benlake: do you have an example of how it did ? | 17:57 |
| benlake | I popped into KVM to see what was up and I caught the tail end of “erasing disks complete” and then it powered off | 17:57 |
| benlake | it just timed out again, so maybe. what do you mean example? | 17:58 |
| benlake | “Node changed status - From 'Disk erasing' to 'Failed disk erasing'" | 17:58 |
| roaksoax | benlake: do you have a full log ? | 18:00 |
| benlake | sure | 18:00 |
| roaksoax | benlake: also /var/log/maas/rsyslog/<machine-name>/<date>/messages | 18:00 |
| benlake | would you only the latest release attempt and for me to remove ureadahead errors? | 18:01 |
| * benlake notices a lot of snapd nonsense attemting to access the internet | 18:01 | |
| benlake | that’s annoying. | 18:02 |
| roaksoax | benlake: the ureadahead is unrelated though | 18:02 |
| benlake | right, I was just going to remove it from the log so you don’t have to see it... | 18:03 |
| benlake | it’s 29k entries of noise | 18:03 |
| benlake | roaksoax: this is allt he runs sans ureadahead http://paste.ubuntu.com/24728711/plain/ | 18:10 |
| benlake | I just noticed this, which is interesting: cloud-init[3156]: request to http://10.128.1.130:5240/MAAS/metadata//2012-03-01/ failed. sleeping 32.: HTTP Error 409: CONFLICT | 18:10 |
| benlake | according to the logs, it says it erased the disks. MAAS state for this node is “failed disk erasing”. I guess I’ll mark it as Broken, Fixed, and attempt to Deploy it | 18:17 |
| roaksoax | benlake: so, i need a full node event log | 18:21 |
| roaksoax | benlake: and the syslog | 18:21 |
| roaksoax | to know what may be wrong | 18:21 |
| benlake | all I did was grep -v ureadahead, you want all of that? | 18:21 |
| roaksoax | benlake: i think wiith what I have wrt to cloud-init is enough | 18:21 |
| benlake | the only thing I stripped was ureadahead | 18:22 |
| benlake | if that is unrelated, you have everything | 18:22 |
| roaksoax | benlake: my thinking is that this could be the culprit: May 31 17:55:52 electron systemd[1]: Failed to start Apply the settings specified in cloud-config. | 18:23 |
| roaksoax | May 31 17:55:52 electron systemd[1]: cloud-config.service: Unit entered failed state. | 18:23 |
| roaksoax | May 31 17:55:52 electron systemd[1]: cloud-config.service: Failed with result 'exit-code'. | 18:23 |
| roaksoax | benlake: do you have a node event log ? You can go to the UI and grab it from there | 18:23 |
| roaksoax | specially the "full" | 18:23 |
| roaksoax | or you can grab it from the API on the 'events' endpoint | 18:23 |
| benlake | lemme see | 18:23 |
| benlake | oh, you want the MAAS events, got it | 18:24 |
| benlake | ugh, between timesyncd and snapd, this thing sits around timing out for 5 minutes. | 18:26 |
| roaksoax | benlake: http://pastebin.ubuntu.com/24728895/ | 18:27 |
| roaksoax | that's the issue | 18:27 |
| benlake | I saw that before pasting and was about to mention it | 18:28 |
| benlake | here is the entry in MASS UI, https://screencast.com/t/9ccpHGoc | 18:30 |
| benlake | is something wrong with that? | 18:31 |
| benlake | I went through the events exercise anyhow: http://paste.ubuntu.com/24728917/ | 18:34 |
| benlake | hmm, it seems cloud-init might not know to use a proxy | 18:37 |
| benlake | is it a project expectation deploying nodes have internet access? | 18:39 |
| * benlake is just wondering | 18:39 | |
| roaksoax | benlake: no. MAAS can work on completely offline environments | 18:45 |
| benlake | yeah, I see it can, I’ve been using it that way | 18:45 |
| roaksoax | benlake: but my guess of what's happening here is that sudo add-apt-repository ppa:<etc> is failing behind a proxy | 18:45 |
| benlake | just digging in the logs due to this PPA thing seems to show a lot of internet access attempts on these ephemeral images | 18:46 |
| benlake | roaksoax: aye, that seems to be the case. It not being configured to use one | 18:46 |
| benlake | https://askubuntu.com/questions/724224/cant-add-ppa-on-ubuntu-15-10-user-or-team-does-not-exist | 18:46 |
| benlake | old, but likely the same issue | 18:46 |
| roaksoax | benlake: the other thing you could do, is to add the ppa as a URL | 18:48 |
| benlake | I think I tried that, but the UI said it was invalid | 18:49 |
| benlake | prevented me from saving it. let me mess with it again | 18:49 |
| roaksoax | benlake: do it as 'add repository' | 18:53 |
| benlake | gotcha, does this look correct? https://screencast.com/t/lJy1PUpE49i | 18:55 |
| pmatulis | benlake, you want to add a PPA right? | 19:18 |
| benlake | yes, but see the discussion above | 19:18 |
| pmatulis | well, i know i added a PPA before so i'm confused. are you using the default proxy? | 19:25 |
| benlake | yes | 19:31 |
| benlake | are you sure your node didn’t have internet access when being deployed? | 19:31 |
| benlake | cloud-init attempts to use add-apt-repository when booted into the ephemeral environment and does not honor any proxy | 19:32 |
| pmatulis | oh. right, of course it did | 19:34 |
| pmatulis | enabling a PPA requires internet access | 19:36 |
| benlake | not sure MAAS devs expect that to be required. | 19:37 |
| pmatulis | if maas is supposed to be "offline ready" then i guess that's a bug. but i'm not sure how you can enable a PPA w/o internet | 19:37 |
| benlake | if they do, another update to the docs would be a good idea :) | 19:38 |
| pmatulis | can you file one? | 19:38 |
| benlake | if the env vars htto_proxy and https_proxy are set, it’ll use it | 19:38 |
| benlake | only if roaksoax confirms | 19:38 |
| benlake | *http_proxy | 19:38 |
| benlake | roaksoax: adding as non-PPA worked, thanks! Also the URL in my screenshot was completely wrong. I failed on that one. | 19:45 |
| roaksoax | :) | 19:55 |
| roaksoax | benlake: glad to hear it works now | 19:56 |
| pmatulis | benlake, you were able to add a PPA as a non-PPA? 'splain? | 19:56 |
| benlake | https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack | 19:57 |
| benlake | green link, “Technical details about this PPA" | 19:57 |
| benlake | shows the direct repo URL | 19:57 |
| benlake | can use that with the add repo option instead of add ppa. basically the PPA helpers just lookup that URL and fetch the repo’s public key for you | 19:58 |
| benlake | seems I have another auto IP address selection issue. The NTP server being set for deployed nodes is not the desired/functional IP | 20:28 |
| benlake | I like the cascade of NTP servers, region, rack ,etc. | 20:28 |
| benlake | but am I going to need to use the “force external NTP server” option here? | 20:29 |
| benlake | dang, but that would force the region controller to use itself as an NTP server. | 20:51 |
| pmatulis | benlake, thanks (for the PPA stuff) | 20:57 |
| pmatulis | benlake, what URL did you end up using? you said you made a mistake originally? | 22:55 |
| benlake | bad: https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack | 22:56 |
| benlake | good: http://ppa.launchpad.net/linbit/linbit-drbd9-stack/ubuntu | 22:56 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!