| braziercustoms | hello. I'm back again :D I am still having problems with conjure-up on 16.04. can anyone tell me why I keep getting this error? 2017-08-07 21:29:26,447 [DEBUG] conjure-up/openstack-novalxd - __init__.py:21 - Showing dialog for exception: cannot retrieve charm "cs:glance-258": cannot get archive: Get https://api.jujucharms.com/charmstore/v5/glance-258/archive: dial tcp: lookup api.jujucharms.com on 10.228.217.1:53: re | 01:45 |
|---|---|---|
| sarnold | braziercustoms: you're3 cut off at ":53: re" | 01:47 |
| braziercustoms | read udp 10.228.217.50:38257->10.228.217.1:53: i/o timeout | 01:48 |
| braziercustoms | @sarnold that is the last of it | 01:49 |
| braziercustoms | 16 machines are running but cannot access horizon of course | 01:49 |
| braziercustoms | @sarnold I just noticed that one is different than the last one. | 01:51 |
| sarnold | braziercustoms: is that a go-implemented dns lookup that's failing? | 01:51 |
| braziercustoms | im not sure what you mean "go-Implemented" | 01:51 |
| braziercustoms | sarnold: it is hitting a DNS forwarder but its never been a problem, you think thats the problem? it works if I put the URL in the web browser it tries to download a file. | 01:59 |
| sarnold | braziercustoms: i'm guessing here.. try something for "for i in `seq 1 100` ; do dig api.jujucharms.com @10.228.217.1 ; done ... see if you get any errors or all successes | 02:01 |
| braziercustoms | sarnold: dont see a failure | 02:05 |
| braziercustoms | answers each in 0ms | 02:06 |
| sarnold | braziercustoms: hrm. is the conjure-up error repeatable? | 02:06 |
| braziercustoms | well, last time seems like I got a different one but it was same type of failure | 02:07 |
| braziercustoms | sarnold: I'm trying to find it. give me a sec | 02:08 |
| braziercustoms | sarnold: here is a previous one https://pastebin.com/aHpjDAMP | 02:12 |
| braziercustoms | sarnold: so far every time I try to conjure-up, it fails at something "not retreivable" | 02:13 |
| braziercustoms | sarnold: that is every time in the last week or so. | 02:14 |
| sarnold | braziercustoms: I -think- file a bug against juju with ubuntu-bug juju-core .. unless you can track down if there is something wrong with the dns forwarder on 10.228.217.1 or the firewalls in between.. | 02:15 |
| braziercustoms | sarnold: I conjure-down, I will remove the controller (again) and start over see if there are any new errors. | 02:20 |
| braziercustoms | sarnold: :D you made me realize that my dns forwarder for my home network is replying on both interfaces.. its always been this way and didnt cause a problem before, but would it cause a delay? | 03:46 |
| braziercustoms | as I get ;; reply from unexpected source: 192.168.1.3#53, expected 192.168.1.2#53 | 03:47 |
| cpaelzer | jamespage: the OVS change we had was applied to master, I asked for branch-2.8 as well | 05:52 |
| cpaelzer | jamespage: but that said we could also just carry the patch on 2.8 as we know it is accepted | 05:52 |
| cpaelzer | jamespage: were you able to complete your check on OVS 2.8 and if so what is the summary and expected timeline? | 05:53 |
| cpaelzer | jamespage: instead of writing more walls of text I'll wait until you are around | 05:56 |
| cpaelzer | good morning everybody btw | 06:03 |
| dshap | Hey everyone -- all of a sudden today my production web server (running Ubuntu 14.04) started going down, and I'm trying to figure out why. By "going down" I mean I couldn't SSH into it, and when I rebooted the server (on EC2), I checked syslog and the first time I had this problem earlier today it showed memory issues, but then a few hours later it went down again and syslog didn't seem to show the same memory issues. What else should I l | 06:29 |
| dshap | into to diagnose what could be causing this? | 06:29 |
| ranjan | hi all, I have a Linux box whose / partition is 100% utilised when using df command, but then when using du -shx /, show there is a lot of free space. | 06:48 |
| cpaelzer | dshap: what do you mean by memory issues - out of memory kills? | 07:48 |
| dshap | cpaelzer: a couple lines like this: "dhclient: fork: Cannot allocate memory" | 07:48 |
| cpaelzer | and even being away as documentation @ranjan: this is a known thing and not "wrong" https://unix.stackexchange.com/questions/45771/df-vs-du-why-so-much-difference | 07:49 |
| cpaelzer | dshap: you seem not to be the first one https://forums.aws.amazon.com/thread.jspa?messageID=674000 | 07:51 |
| cpaelzer | dshap: but in general you'd need to track what is consuming your memory - assuming that really is the issue | 07:51 |
| cpaelzer | dshap: while usually next to useless/counterprodictive on a cloud you could set up some swap space to not die so immediately | 07:52 |
| cpaelzer | dshap: with that in place you could regularly track memory consumption | 07:52 |
| dshap | cpaelzer: That was my first thought when this happened the first time today. After it happened, I rebooted the server and my website came back online, but then 2 hours later it went down again -- that time, it didn't have any "Cannot allocate memory" messages in syslog | 07:52 |
| cpaelzer | dshap: either with basic "ps --sort -rss -eo rss,pid,command | head" or (my preference) with the tool smem | 07:52 |
| cpaelzer | dshap: what did it have in that second case - it must have had "something" I hope | 07:53 |
| cpaelzer | it doesn't shut down but loosing network connection right? | 07:53 |
| dshap | cpaelzer: According to the log, it doesn't appear to be shutting down, because I just see the reboot logs at the time that I initiate the reboot (from the AWS management console) | 07:54 |
| * kickinz1 back online | 07:54 | |
| dshap | cpaelzer: the last 2 things I see from sys log are: "dhclient: bound to <some_IP> -- renewal in 1701 seconds." and "CRON[1505]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)" | 07:55 |
| dshap | cpaelzer: the last 2 things before it went down the 2nd time* | 07:56 |
| cpaelzer | dshap: hmm - I might not have the best hints so other feel free to chime in, but if you really have not more after one of these reboots you need to set up some things to log more to disk | 08:10 |
| cpaelzer | dshap: you could also have auto-restarts if the HW under the cloud has issues, like https://forums.aws.amazon.com/message.jspa?messageID=275616 but I'd consider this unlikely to happen two times | 08:10 |
| cpaelzer | I tihnk you could find in the service health dashboard if such a case happened | 08:12 |
| cpaelzer | otherwise set up monitoring to close in on what issue you system in particular might run into | 08:13 |
| dshap | cpaelzer: got it. thank you for your suggestions! | 08:17 |
| lordievader | Good morning | 08:22 |
| cpaelzer | hi lordievader, good morning to you as well | 08:29 |
| lordievader | ranjan: That is quite possible. Probably some application which still has a reference to a deleted file. | 08:35 |
| lordievader | Du doesn't see those, df does. | 08:37 |
| lordievader | How are you doing cpaelzer? | 09:18 |
| cpaelzer | jamespage: I see the latest OVS upload still has the 15 errors in the self-tests | 09:19 |
| cpaelzer | lordievader: fighting the tide :-) how are you? | 09:19 |
| jamespage | cpaelzer: yes - looking at those now | 09:20 |
| cpaelzer | jamespage: I'll sync what we upload to Debian on Thursday afternoon - now that te patch with the pkg-config is accepted in OVS it will likely be the one with the multi-arch changes | 09:21 |
| cpaelzer | jamespage: but once I sync that from Debian it will fail to migrate until we have also uploaded a new OVS working with it | 09:21 |
| cpaelzer | jamespage: so ping me once your exploitation of OVS 2.8 makes and progress (or you are giving up for a reason) | 09:21 |
| cpaelzer | so we can coordinate those syncs/uploads as needed | 09:22 |
| lordievader | Doing allright here :) | 09:22 |
| === clvx is now known as Guest40784 | ||
| === LaserAllan is now known as Guest21980 | ||
| jamespage | cpaelzer: picked your patch for ovs pkg-config/dpdk | 11:01 |
| jamespage | cpaelzer: having to skip some tests on the dpdk build which I've reported upstream | 11:01 |
| jamespage | cpaelzer: I've also been able to re-enable s390x testing which is nice | 11:01 |
| cpaelzer | wow, that is great jamespage | 11:03 |
| cpaelzer | did you have to skip all 15 tests or were some fixable? | 11:04 |
| cpaelzer | jamespage: thanks for integrating my patch, as I said it is upstream now so it should be safe and not a maintenance burden | 11:04 |
| cpaelzer | jamespage: I sent you an optinal invite for the DPDK meeting, you certanily have enough but if you ever want to attend you now know time&date | 11:05 |
| cpaelzer | jamespage: so with the pkg-config applied we can upload the 17.05.1 including this change to Debian and sync from there | 11:06 |
| cpaelzer | jamespage: what is the timing on the OVS 2.8 release - do you know? | 11:06 |
| jamespage | cpaelzer: this month | 11:06 |
| jamespage | ~(timing of release) | 11:06 |
| cpaelzer | ok, so I'd then try to coordinate the upload to Debian and sync from there | 11:07 |
| jamespage | cpaelzer: re the tests - I've just deferred that to upstream tbh | 11:07 |
| cpaelzer | once the 17.05.1 is in Artful proposed I can ping you to upload the OVS 2.8 then | 11:07 |
| jamespage | cpaelzer: I'm a little time short this week as off from end of thursday for a couple of weeks :-) | 11:07 |
| cpaelzer | a couple you say | 11:07 |
| cpaelzer | I hope for vacation | 11:07 |
| cpaelzer | jamespage: so what shall we do about OVS then, should I try to get it to Debian and sync to Ubuntu earlier? | 11:08 |
| cpaelzer | jamespage: would that help you to close out the OVS upload before you leave? | 11:08 |
| jamespage | cpaelzer: it would but I'm happy to upload prior to that actually arriving in Ubuntu; it will dep-wait until 17.05.1 appears :-) | 11:09 |
| cpaelzer | and then fail on me while you are unavailable :-P | 11:09 |
| cpaelzer | but ok, sounds like a plan for now | 11:09 |
| cpaelzer | I can still whine about it when it happens | 11:10 |
| cpaelzer | jamespage: ok, I have asked my co-maintainer to upload to Debian whenever he is ready | 11:13 |
| jamespage | ta | 11:14 |
| cpaelzer | jamespage: I'll sync from there into artful once available, feel free to make a dep-waiting one being around in a-p if you want | 11:14 |
| jamespage | cpaelzer: I'll hold for as long as possible | 11:16 |
| cpaelzer | ok | 11:17 |
| jamespage | cpaelzer: good smell test at least - https://bileto.ubuntu.com/excuses/2889/artful.html | 13:45 |
| cpaelzer | jamespage: yeah doesnÄt seem too bad | 13:50 |
| === Guest13936 is now known as med_ | ||
| === med_ is now known as medberry | ||
| === admcleod_ is now known as admcleod | ||
| === medberry is now known as med_ | ||
| ahasenack | ubuntu-server meeting in #ubuntu-meeting if anyone is interested | 16:06 |
| madLyfe | this was the iso i installed: ubuntu-16.04.2-server-amd64 but im still showing kernel 4.4.0-89-generic. | 17:46 |
| sdeziel | madLyfe: that seems OK to me. Unless you expected the HWE kernel? | 17:51 |
| nacc | madLyfe: can you show `apt policy linux-generic-hwe-16.04` ? | 17:51 |
| nacc | madLyfe: yeah, so I'm thinking now the server ISO *doesn't* use the HWE kernel by default (not seeing it in the seeds) | 17:51 |
| nacc | madLyfe: which is a distinction from the desktop ISO I didn't realize | 17:52 |
| madLyfe | so run this command? 'apt policy linux-generic-hwe-16.04`' | 17:53 |
| nacc | madLyfe: that will just say if the hwe kernel is installed | 17:55 |
| nacc | madLyfe: if it's not, you can install it | 17:55 |
| nacc | (if you want) | 17:55 |
| nacc | madLyfe: but as we were discussing on #ubuntu, why do you want the hwe stack on your server? | 17:55 |
| madLyfe | that command didnt do anything | 17:56 |
| nacc | madLyfe: it should output something | 17:56 |
| madLyfe | my servers are miners, just trying to squeeze the most performance out of the CPUs as possible. | 17:56 |
| nacc | madLyfe: ok, and you've measured that the newer kernel is more performant? | 18:00 |
| nacc | madLyfe: in any case, back to taht command. It hsould either have outputted "Unable to locate package" or it would have outputted lines that show what versions are available/installed | 18:01 |
| madLyfe | i have no idea. havent tested it. thats why i was seeing if i had it installed or not | 18:01 |
| madLyfe | nacc: http://i.imgur.com/1RMDu1Z.png | 18:03 |
| oerheks | drop t ' at the end | 18:04 |
| oerheks | apt policy linux-generic-hwe-16.04` << | 18:04 |
| madLyfe | i did on the next two lines | 18:04 |
| madLyfe | where nothing happened | 18:04 |
| oerheks | those next lines are within your previous command, try fresh | 18:05 |
| madLyfe | manually typed it in again and same | 18:05 |
| madLyfe | even the clear command is doing it | 18:05 |
| madLyfe | wth | 18:05 |
| oerheks | apt policy gives a search, nothing gets installed or changed | 18:06 |
| madLyfe | no i mean clear isnt doing anything either: http://i.imgur.com/ZVGr7MB.png | 18:07 |
| oerheks | again, you are within your previous command, stop this with ctrl + c | 18:08 |
| sdeziel | madLyfe: type: ctrl-c | 18:08 |
| madLyfe | ah yes. still new. sorry | 18:08 |
| oerheks | :-) | 18:08 |
| oerheks | you will remember this next time, no worry | 18:08 |
| madLyfe | https://gist.github.com/5f60456c093419cd708b6f08faf98526 | 18:09 |
| oerheks | your prompt should give a name @ machine + $ ( as user ) or + # as root | 18:10 |
| oerheks | good, HWE not installed | 18:10 |
| oerheks | sudo apt install linux-generic-hwe-16.04 # if you *need* a newer kernel and such | 18:11 |
| madLyfe | well it says 4.10, but i dont have it? | 18:11 |
| madLyfe | "Installed: (none)" mean its not? | 18:12 |
| madLyfe | sudo apt install –install-recommends linux-generic-hwe-16.04 ? | 18:14 |
| nacc | madLyfe: right (none) means not installed | 19:57 |
| nacc | madLyfe: i'm not sure if you need install-recommends or not, but yes, that's the package you want | 19:57 |
| madLyfe | thanks. whats the worst that could happen by upgrading? | 19:59 |
| nacc | madLyfe: your system doesn't boot? | 19:59 |
| nacc | madLyfe: note, you're not upgrading, you're installing a differnet package | 19:59 |
| nacc | *a second package | 19:59 |
| madLyfe | its a pretty standard supermicro board. nothing proprietary on it. should be good. | 20:01 |
| === Epx998- is now known as Epx99o8 | ||
| === Epx99o8 is now known as Epx998 | ||
| Epx998 | Is there a good command to get the status of a netapp share thats mounted? | 20:55 |
| === jathan_ is now known as jathan | ||
| Epx998 | hmm | 21:14 |
| === hehehe is now known as Guest2979 | ||
| === Guest2979 is now known as hehehe | ||
| nacc | mwhudson: there's a few golang packages that are showing up in the 'out of date' from Debian list for server. Am I right to assume you've got a handle on them all? :) | 22:27 |
| mwhudson | nacc: er no but i can | 22:27 |
| mwhudson | nacc: where's the list? | 22:28 |
| nacc | mwhudson: http://reqorts.qa.ubuntu.com/reports/ubuntu-server/merges.html | 22:28 |
| nacc | mwhudson: there are ~11 for golang | 22:28 |
| nacc | mwhudson: it's ok if they are understood to be behind for a reason, I just want to make sure that we have a reason for each :) | 22:29 |
| mwhudson | the reason is that i haven't ripped out the shared library stuff yet i think :( | 22:32 |
| nacc | teward: are you planning on updating nginx to 1.13.3 this cycle? | 22:56 |
| hehehe | could be good | 22:59 |
| hehehe | teward: you going to do it | 22:59 |
| hehehe | - | 22:59 |
| hehehe | ? | 23:00 |
| nacc | hehehe: why would you ask the exact question I just did? | 23:00 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
