Arroyo1010hehe, I'm europe based, too :)00:10
Arroyo1010But I'm an owl00:10
sarnoldindeed :)00:11
Arroyo1010I'm actually making some progress. Turns out that the qemu "builder" in packer supports .img format. Now I need to figure out how to ingest the public key, etc00:13
Arroyo1010And, of course, I'm using the official Ubuntu cloud .img for that00:15
lordievaderGood morning06:07
ubernetsI have a Ubuntu 16.04 server, df -Th / shows 1.6 GB available space, there is only one partition in /etc/fstab . However I keep running into disk full errors from various programs. apt-get upgrade, apt-get autoremove, apt-get install, apt-get -f install show disk full and sometimes resulting dependency error. git pull also shows a disk full error message. Any idea what could be causing these errors even though the disk h06:08
ubernetsas 1.6 GB free space. I have rebooted the server twice already, still the same issues. Here is an output of df -h http://paste.ubuntu.com/25538619/ . And here is a failed apt-get upgrade showing the error message http://paste.ubuntu.com/25538626/ .06:08
lordievaderubernets: I'd do a watch on the df command while running the upgrade. Perhaps apt is downloading so much as to fill the disk.06:18
ubernetslordievader, I will look into that, but it happened with the git pull too. So I am skeptical that that's the cause06:19
lordievaderTrue. It makes for a strange issue.06:20
lordievaderCan you make files yourself still? Or does that too throw an error?06:20
ubernetslordievader, Yea I created a very small text file. It didn't throw an error06:25
lordievaderHmm, strange problem.06:25
ubernetslordievader, it gets stranger. I typed a rm command for the file and while using tab completion for the file name I got this error message: rm test-bash: cannot create temp file for here-document: No space left on device06:27
ubernetslordievader, the file was named test.txt. So there seems to be a problem with the /tmp folder ?06:28
lordievaderubernets: I saw multiple folders in the apt output complaining.06:28
ubernetsI typed tab completion after typing test06:29
lordievaderWhat kind of filesystem is on xvda1?06:29
ubernetsrm test<tab>, then it completed to "rm test-bash: cannot create temp file for here-document: No space left on device"06:29
ubernetsI think ext4 , double checking06:30
ubernetsYes ext406:30
lordievaderAre you able to reboot the server?06:37
ubernetsYes I rebooted it twice06:40
lordievaderubernets: What does 'du -hs /' report?06:42
lordievaderAround 7.8G of usage?06:43
ubernetsone sec06:45
ubernetslordievader, running the command now, it takes some time to complete06:52
lordievaderI'm sure it will.06:52
ubernetslordievader, here is the output http://paste.ubuntu.com/25538754/06:53
lordievaderHmm, unless that /var/lib/lxcfs folder is ~1.5G this seems fine.06:55
ubernetsWhy is access denied to root?06:56
lordievaderGood question. I wouldn't be suprised if AppArmor has something to do with it.06:58
ubernetslordievader, I want to show you something. THe files from the last paste have question marks for instead of permission flags07:00
ubernetslordievader, http://paste.ubuntu.com/25538796/07:01
lordievaderThat means you are not allowed to read the metadata from it.07:01
lordievaderNow that I think about it, might also be a userspace mount. Sshfs mounts can result in this.07:01
lordievaderAnyhow, I'd start cleaning up or extending the disk.07:02
ubernetsmore /etc/fstab : LABEL=cloudimg-rootfs/ ext4defaults,discard0 007:02
ubernetsHow should I clean it up?07:04
ubernetsYou mean just keep deleting more files?07:04
lordievaderIn a sane way, yes.07:06
lordievaderRemove packages you don't need, etc.07:06
=== jelly-home is now known as jelly
ubernetslordievader, I think someone is bruteforcing the server and it fills up the auth and bmtp logs07:28
lordievaderWhy do you think that?07:29
ubernetstail -f auth.log keeps showing up these kinds of messages : Failed password for root from port 59927 ssh207:30
ubernetsIP address from China07:30
ubernetsbtmp.log tripled in size in last few minutes07:31
lordievaderPublic ip?07:32
lordievaderIf so, configure fail2ban or something similar.07:32
ubernetslordievader, it's an ec2 instance and there are only 3 inbound ports allowed in the security group, but I see a lot of ssh2 attempts on all kinds of ports logged into auth.log .07:41
lordievaderOfcourse, if port 22 is publically accessible you get login attempts.07:50
ubernetsTHe attempts are made on ports like 59927 . See above07:51
ubernetsOh my bad07:52
ubernetsit's the from port07:52
cpaelzerjamespage: does UCA publish debug symbols as well?08:20
cpaelzerI tried to fetch them as I'd do on a "normal" ppa, but that didn't work yet08:21
fishcookeri've download http://releases.ubuntu.com/16.04/ubuntu-16.04.3-server-i386.iso then create the usblive with startup disk creator.. and the result is always failed08:34
fishcookerisolinux.bin missing or corrupt08:34
lordievaderfishcooker: Uefi or bios?08:40
fishcookerwill dd the image directly will solve the problem lordievader08:49
lordievaderBios is usually trouble free. You might try unetbootin. I'd leave the dd option as a last resort.08:51
fishcookerthanks for unetbootin lordievader, noted for the dd09:08
fishcookeri have dell server with 6 slots hard disk; with 2 sas disk 70GB then 2 disks 1TB, 2 slot will used as raid-1 for 2 sas disks what should i do with the rests resource ... let's say on the future i want to add 2 disk for the rest slot available should i go with lvm?09:09
AdillianMorning all. I just installed ubuntu-server and I can't connect to my lan let alone wifi. ifconfig has 'lo' with Local Loopback, and10:28
Adillian..and virbr0 with link encap:ethernet10:28
Adillianany ideas what I can try or where I can find relevant documentation?10:29
Adilliannever mind, fixed it10:58
=== bananajoe is now known as sammyg
lordievaderfishcooker: Concerning your question about lvm. It does sounds like you could benefit from using lvm.13:02
lordievaderAdding/removing disks is quite simple in lvm.13:02
=== beisner is now known as Beisner
ilmaisincan apparmor do following: let's say we have a port in the unprivileged range and we want only one user to be able to bind it14:00
sdezielilmaisin: no14:48
disposable2is there anything like bcache but in RAM? i don't care about data loss during power failure, this is for Ceph, data is replicated on other nodes. ideally i'd like something tiered (ram -> nvme ssd -> slow spinning disks).15:17
JanCdisposable2: you can have memory-backed block devices; not sure how is currently the best for what you want to do though15:31
rh10guys, go language suitable for system automation tasks? one employer require it15:31
rh10is it real write such king of things with go lang instead bash or python?15:32
JanCalthough I guess something bcache-like could be more optimized than having the extra layer15:32
ScottyAtHomewhat FS are people using for a RAID1 type system 16.04 server. tried BTRFS but it is a bit too faffy and difficult to get working when testing raid1 issues, has anyone else had those issues?15:33
naccrh10: you certainly can15:33
rh10nacc, thanks15:34
sdezielScottyAtHome: ZFS works pretty well on 16.0415:34
dpb1ScottyAtHome: you are running software raid?  mdraid?15:34
JanCI know people who write system automation "scripts" in C  ;)15:34
naccJanC: indeed :)15:34
JanCScottyAtHome: you can also use ZFS, or layered software raid + whatever filesystem you want15:34
ScottyAtHomesdeziel: might try it, how you finding it?15:35
ScottyAtHomedpb1: i am running the BTRFS raid, bit of a pain to get working when the root is on it and you are booting initramfs to get it back up on degraded15:36
sdezielScottyAtHome: I use it on all my physical machines15:36
dpb1ScottyAtHome: personally, I prefer zfs, or traditional raid + ext4.15:36
ScottyAtHomeJanC: have you used ZFS? if so how have you found it?15:36
JanCI have ZFS in one system, but not using mirroring15:37
sdezielI have not tried a root on ZFS though, for that I stick to mdraid + ext415:37
ScottyAtHomesdeziel: I use BTRFS on my desktop & laptop, but this is the first time of having it on a server in raid1 and it is a pain. How you find the raid1 or which ever raid you use when testing fro failures?15:38
sdezielScottyAtHome: mirroring on ZFS works really well, done many rebuilds, lost many drives but 0 data :)15:38
ScottyAtHomesdeziel: thattttt sounds like a sensible idea15:38
JanCZFS might have the same issue: you're not used to it (yet)  :)15:38
ScottyAtHomeJanC: true. it was BTRFS when I had to start updating Grub stuff to just get it to accept degraded I thought this is no good for ssh ing into to sort15:39
ScottyAtHomeDoes anyone know where LXD containers are kept? as I want to keep the containers on an easily adjustable FS15:41
sdezielScottyAtHome: when you'd install LXD, it will ask you if you want to create a zpool for it's storage15:41
rh10nacc, quiestion, how handy is it? write in go. probably there is a lot of useful libs in that case15:41
ScottyAtHomesdeziel: that sounds better.  Whe you have had a degraded raid with ZFS will the system still boot? is the unbroken drive/s still usable?15:43
naccrh10: probably better asked in a go channel15:43
naccrh10: not sure what you mean, anyways15:43
rh10nacc, you're right15:43
sdezielScottyAtHome: my bootup doesn't depend on ZFS as my root FS is on md+ext4. That said, ZFS mounted what remained of the pool and was still usable15:44
dpb1ScottyAtHome: yes, filesystems can be mounted degraded.15:44
sdezielScottyAtHome: then when I replaced the faulty drive, rebuilding (resilver) was fast as it only rebuilds the data, not the full disk like md does15:45
ScottyAtHomedpb1: automatically or manually? I want one that    is automatic so I can ssh in to sort the problem out.15:45
ScottyAtHomesdeziel: thanks for the info, that is useful.  Glad I have come on here to ask as it is quicker than it has been testing the system out.15:46
disposable2ScottyAtHome: I always use ZFS for everything but /. No support for zfs in grub-efi and no installer support force me to use btrfs for /. Only once did i have an unbootable system after a powercut, so i had to boot up from usb disk and run "btrfs check" or something similar against my /. After that, grub would boot it up again.15:48
disposable2ScottyAtHome: one day, we'll have solaris-style boot environments (whether it's with btrfs or zfs, i don't care) and we'll all live happily ever after.15:50
ScottyAtHomedisposable2: when I have been testing BTRFS it gets stuck at boot and goes into intitramfs, were you able get around this? I need the server to be remotely accessed to sort out the issues.15:50
ScottyAtHomedisposable2: lol, what were solaris style boot environments?15:51
disposable2ScottyAtHome: unless you've taken a picture of the screen while it was stuck in initramfs, i don't have an answer for you. i've only ever had that 1 problem with btrfs and resolved it within 5 minutes.15:52
disposable2ScottyAtHome: Boot Environments are FS snapshots you can boot into if your system gets broken after an update. this has saved my neck countless times. on solaris, BEs were presented in grub as options you could boot into.15:53
disposable2ScottyAtHome: when it's properly integrated, a new boot environment is automatically created after something like 'apt upgrade'.15:54
disposable2ScottyAtHome: of before15:54
disposable2OR before15:54
=== JanC_ is now known as JanC
JanCdisposable2: do you mean snapshot-based?15:56
disposable2JanC: yes, but with good integration with grub and package manager.15:57
JanC('apt-btrfs-snapshot' already exists)15:57
ScottyAtHomedisposable2: is your / on raid?15:59
disposable2JanC: does it automatically create a new entry in grub that you have to confirm as valid (after a boot) or it will automatically rollback to last confirmed-as-working?15:59
disposable2JanC: it's the overall system integration that makes BEs useful.16:00
JanCI assume there are distros which implement this, but I doubt it's well-tested enough to be used by default in a major distro16:01
JanC(IMO btrfs isn't ready to be used by default either)16:02
ScottyAtHomeJanC: i don't know what else is though with all the features BTRFS brings. I understand ZFS might be closer but the licensing is a bit dodgey16:03
disposable2JanC: i had to stop using btrfs for anything other than / when i discovered it's quickly degrading performance with many snapshots/clones.16:04
JanCthat's one issue, but I was thinking about getting all edge case data loss bugs fixed  :)16:06
JanCbcachefs might also become useful at some point16:06
JanCand who knows, maybe HammerFS one day...16:08
JanCbut OpenZFS is probably the only core filesystem code base that is really mature right now...16:08
ScottyAtHomeJanC: Thanks for the information.16:08
JanC(outside the legacy ones like XFS & ext4, of course)16:08
JanCScottyAtHome: all just my opinion, of course  :)16:09
ScottyAtHomeJanC: but sounds like experienced opinion.16:09
JanCnot really that experienced, but based on experiences from others I read16:10
ScottyAtHomeJanC: would you use ZFS for root?16:13
JanCI have no experience with that16:13
JanCalso, would depend for what (probably not for an important server)16:14
ScottyAtHomewhat would you use for an important server?16:14
JanCit would really depend, and you probably want to ask someone who's actually running lots of important servers  :)16:16
naccon some level, the fs is a little less relevant for an 'important server', the hardware and backup story is probably the higher priority16:17
naccnow, some fs give you the backup story16:18
naccbut, honestly, i'd expect most 'important servers' to run the "legacy" (funny word that) fs that JanC mentioned16:18
naccas they have been around and are "known stable"16:18
naccZFS is too new to really be on those long-running machines, IMO16:18
JanCnacc: by "legacy" I meant "traditional" filesystems, without integrated snapshots, raid, etc.16:20
naccJanC: i know :)16:20
naccJanC: just not heard of ext4 called that16:21
JanCand ZFS isn't really new16:21
JanCit's just new on linux  :)16:21
naccright, sorry, i meant in the context of this channel16:21
JanCbut the core code base is the same (it's mostly not a re-implementation)16:21
naccJanC: yeah, that's my understanding too16:22
JanCso most of the code base is well-tested and has seen quite a bit of real-world use16:23
=== Beisner is now known as beisner
=== dnegreir1 is now known as dnegreira
=== keithzg_ is now known as keithzg
hehehe_offwhy such quiet channel19:38
cliluwWhat does disabling a user account do? Does it just prevent logins to that account?20:53
=== hehehe_off is now known as hehehe
lordcirth_workcliluw, it prevents password logins - ssh keys can still get it21:31
lordcirth_workin*.  All --lock does is prepend '!!' to the password hash so it can't be matched.21:32
cliluwlordcirth_work: I think you're talking about disabled password. I'm thinking about disabled /account/.21:32
lordcirth_workcliluw, well then you'll need to be more specific about what you mean by disabling an account.  What command are you running?21:32
cliluwlordcirth_work: usermod --expiredate 121:34
lordcirth_workcliluw, that ought to keep them out, but I haven't needed to use it myself.  You can always delete the account, though you'd need to be careful of uid's.21:38

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!