/srv/irclogs.ubuntu.com/2014/05/29/#ubuntu-server.txt

lorenzTJ-: You remember monday when we tried to figure out how to get the server back online? I then reinstalled and today it happened again. Exactly the same problem. Approximately the same installation, running less than 36 hours, then hung up (network didn't work anymore, local console login timed out). I couldn't do anything besides rebooting and exactly the same problem appeared again.00:09
pakcjohow can I be sure that slapd gets started before krb5kdc?00:09
TJ-lorenz: !!!! on bare hardware or VM, or both?00:11
lorenzTJ-: Again on both.00:11
lorenzTJ-: I already checked initramfs, removed anything that isn't on my host machine and rebuilded. Seems the same. I now got detailed file logs on what has changed in the last 36 hours.00:12
Gallomimiaim having trouble with the installer not giving keyboard access anymore after entering the recovery mode in its grub menu00:14
=== xnox_ is now known as xnox
Gallomimiait's been working fine for days. then suddenly no keyboard in installer00:15
TJ-lorenz: have you done a memtest86+ on the bare metal?00:16
lorenzTJ-: No, I haven't. But I have done one  on the KVM host, so it should boot at least there.00:16
RoyKthe only way to run memtest is on bare metal00:16
RoyKlorenz: a kvm host should be bare metal00:17
RoyKand it probably is00:17
RoyKlorenz: using ecc on this one?00:17
lorenzRoyK: I ran it on the host.00:17
RoyKall way through?00:18
lorenzRoyK: for a night00:18
RoyKshould do00:18
TJ-lorenz: do you test it on the VM only after it fails on bare metal?00:18
lorenzRoyK: I got standard DDR3-1600, but my host system never crashed in a whole year of usage (5-8h/day)00:18
RoyKlorenz: probably not memory issues then00:19
lorenzRoyK: The server first crashed after 20d uptime and now after 36h00:19
lorenzRoyK: the 36 hour-one is a fresh install00:19
RoyKlorenz: wrote this little program to just stress memory from inside the OS, it made my system crash easily when memory was bad00:19
TJ-lorenz: I've had modules go 'funny' in severs after a long time... turned out to be vibration-related... reseating the modules after removing them fixed it00:21
lorenzTJ-: Ok, I'll see. I'll also run a quick memtest.00:21
RoyKlorenz: http://karlsbakk.net/tmp/memstress/00:21
RoyKlorenz: run that over for some time - if that can't crash the system, neither can the vm00:22
RoyK(IMHO)00:22
SachiruAnyone using ZFS on Linux for production on limited systems? How much RAM is the minimum requirements for it? Somehow I can't find that in the documentation.00:25
lorenzRoyK: It normally runs on bare metal00:26
lorenzRoyK: KVM is just for debugging00:26
RoyKSachiru: 2GB should do, but the more, the better00:26
RoyKSachiru: just don't use dedup unless you know what you're doing00:27
SachiruNo dedupe00:27
SachiruCool00:27
SachiruIs that 2GB just for ZFS ARC, or does that already include the OS?00:27
RoyKthe zfs arc should be large enough to sustain most use00:27
SachiruAlso is it safe to use ZRAM with ZFS?00:28
RoyKno, arc is separate on linux and freebsd00:28
SachiruAh.00:28
SachiruSo 2GB for ARC + Whatever OS needs00:28
RoyKit's only on illumos/solaris where it's somehow integrated00:28
RoyKit's dynamic, so don't worry00:28
RoyKzram?00:28
SachiruYep00:28
RoyKwhat's that?00:28
SachiruThe compressed swap in RAM thingy00:28
RoyKprobably00:29
Sachiruhttp://en.wikipedia.org/wiki/Zram00:29
RoyKSachiru: how much memory do you have on the box?00:29
Sachiru8GB RAM, but 4GB shall be consumed by VMs.00:29
RoyKadd a small ssd for l2arc00:29
RoyKif the VMs reside on zfs, set aside 8GB or so for SLOG00:30
RoyKjust a small partition for SLOG00:30
RoyKit'll boost performance a bit00:30
SachiruUh00:30
SachiruI don't think I'd need L2ARC00:30
RoyKon the SSD, that is00:30
SachiruOr a SLOG00:31
SachiruPerformance isn't really necessary (low use, test/dev environment VMs)00:31
RoyKSLOG is very good for sync writes and VMs do a lot of that00:31
SachiruYeah, but I don't have the budget for an SSD00:31
SachiruSadly00:31
RoyKjust get a cheap SSD00:31
SachiruThe VMs are just small AMP stacks00:31
SachiruBut sure, when I get into the money I'll add an SSD00:32
RoyKjust trying to talk about how to make zfs better00:32
RoyKno offence00:32
SachiruNone taken00:32
SachiruSo you're sure that ZFS can fit into my needs?00:32
SachiruCan I limit/throttle ARC on a linux system?00:32
RoyKyes00:32
SachiruCool.00:33
RoyKbut I don't remember how00:33
RoyKtry #zfsonlinux00:33
RoyKthey'll know00:33
SachiruThanks!00:35
RoyKnp :)00:35
SachiruOriginally I was planning LinuxRAID, but, lack of transparent filesystem compression.00:35
RoyKthe sad thing about zfs is the lack of flexibility00:35
RoyKthe good thing aboud MD is the revese00:36
SachiruIn that you can't define partitions as differing RAID levels, right?00:36
RoyKyou can add and remove drives from an MD raid without issues00:36
RoyKyou can't add another drive in a raidz on zfs00:36
SachiruTrue00:37
SachiruBTW00:37
Gallomimiareading all that made me hopeful for a decent answer to my problem of getting a system to boot with raid5, encrypted filesystem, and LVM. the root filesystem is on there. but in the mean time i've been tinkering with it and can't get past the language select screen on the 14.04 server installer cause the keyboard just dies00:37
RoyKthat'll require block pointer rewrite, which has been thought to be in the works for 5+ years00:37
SachiruIs it possible to run LVM on top of ZFS?00:37
bekksSachiru: sure, but it makes no sense.00:37
bekksZFS is a volume manager already.00:37
SachiruOk.00:37
Gallomimiai thought about zfs for this system but it didn't sound like something i wanted00:37
Gallomimiablock pointer?00:38
RoyKGallomimia: don't do it. boot the root fs from a pair of mirrored drives or even a tiny drive and use separate data drives00:38
SachiruGiven a choice between linuxRAID and ZOL, if the primary concern is stability and secondary performance, which would be the common recommendation?00:38
Gallomimiauh...00:38
bekksSachiru: dont use ZOL for mirroring your boot devices.00:38
Sachiru@Roy: What I was planning was LinuxRAID two 8GB flash drives for boot/OS00:38
bekksSachiru: USe it for your data stuff.00:39
SachiruOS swap + VM storage on ZFS mirrored 2TB drives00:39
Gallomimiahow much space does the "root" fs need? do i need to mount everything in the root dir from separate volumes?00:39
SachiruLinux will boot even if it doesn't have swap right?00:39
bekksSachiru: Use an OS which support that natively, then.00:39
RoyKSachiru: sounds reasonable00:39
Gallomimiadoesn't need swap00:39
RoyKSachiru: just keep the data and the root separate00:39
SachiruOr should I keep swap on root drive?00:40
bekksYou need swap. :)00:40
bekksSachiru: Keep it on the root disk.00:40
RoyKbekks: no, you don't00:40
Gallomimiayeah. YOU need swap00:40
SachiruOk00:40
bekksRoyK: For suspend2disk - you need swap, of course.00:40
SachiruA minimal install of lubuntu shouldn't eat up 8GB of space on the flashdrive right?00:40
RoyKbekks: for a server?00:40
SachiruAlso no plans of suspending00:40
SachiruFor a server00:40
Gallomimiame either00:40
Gallomimiaif i need to suspend, better to shut down00:40
SachiruThis is intended to be a KVM server. Lubuntu simply for the lightweight GUI00:41
RoyKSachiru: then install ubuntu server without the gui00:41
SachiruAll it will do is start, stop and manage VMs. Everything else will be in a VM inside, and Lubuntu because I'm not yet that comfortable with configuring KVM from the command line00:41
RoyKno need for a gui on a server00:41
bekksRoyK: for various use cases, you actually need swap, on a server.00:41
RoyKbekks: no, you don't. you need swap when you get a memory leak, but the swap will fill up and it'll take longer to find out what happened00:42
RoyKbekks: because with swap, it'll just slow down, you can't log in blablabla00:42
Gallomimiamy server's a desktop. but i want to do some server stuff on it. all im trying to do is boot up the rescue mode of the server install. my keyboard dies at that point. any ideas?00:42
SachiruSo what's the consensus? Swap on rootFS (LinuxRAID mirror on two 8GB flash drives)?00:42
SachiruOr swap on ZFS data drive?00:42
RoyKbekks: without swap, it'll crash gently00:42
RoyKSachiru: no00:43
Gallomimiadont put your swap on a usb drive00:43
Gallomimiawow.00:43
bekksYou do. Like the requirements of Oracle RDBMS clearly state. So the general assumption "you do not need swap on a server" is wrong.00:43
RoyKgeowany`work: no!00:43
RoyKGallomimia: no§00:43
SachiruRootFS would be on usb flash drives00:43
SachiruSo swap on ZFS data drive then?00:43
SachiruOk00:43
RoyKSachiru: no00:43
SachiruWhere would I put swap then?00:43
RoyKSachiru: keep the root and swap on the same media00:43
SachiruBut root is USB flash00:44
SachiruOr should I discard swap altogether?00:44
RoyKthen set vm.swappiness = 100:44
Gallomimialistening to oracle for advice on running open source software sounds like listening to the advice of the surgeon general telling you cars are perfectly safe, cause they cause more business for surgeons than any other market00:44
RoyKso linux won't swap at all unless it really has to00:44
SachiruOk00:44
SachiruIs it required for swap on Linux to be the same size as RAM?00:45
RoyKif you have the root on usb sticks, set vm.swappiness=100:45
RoyKno00:45
SachiruOr can I have 2GB swap with 8GB RAM?00:45
RoyKjust make a small thing like 128MB00:45
SachiruOk00:45
RoyKif you set vm.swappiness=1, it won't use swap00:45
bekksGallomimia: do you actually earn money with running Oracle RDBMS? If not, I gently discard your post.00:45
RoyKrunning oracle on ubuntu?00:46
RoyKthat's not supported afaik00:46
RoyKso you won't make much money ;)00:46
Gallomimiatrue enough, but can we agree that oracle or its software is not anywhere close to this topic?00:46
bekksRoyK: It isnt. It just tried to point out that the general assumption "you do not need swap on a server" is wrong.00:46
SachiruOk00:47
RoyKbekks: you don't00:47
SachiruThanks for the advice00:47
bekksRoyK: I did, read the backlog please.00:47
Gallomimiai ran servers with no swap for awhile. i eventually put swap in. and the problems of memory leaks were as the rest of the channel mentioned00:47
RoyKbekks: if you need swap on a server, you have too little memory00:47
bekksRoyK: If you do not read the requirements section, no one can help you.00:48
SachiruI'll do  / and 256MB SWAP on USB flash drive with VM.swappiness=10% (for ZRAM), then /home and /VM-Storage on the ZFS data disks00:48
RoyKbekks: I know linux uses swap a lot to make room for active memory. it's true. but it's also true that some versions of the kernel abuse this rather a lot, and start swapping out even with lots of gigs of memory available00:48
Gallomimiathe only time i needed the swap, was when there was a memory leak. and only then just so i could kill the server between levels and not in the middle of a game :P00:48
bekksRoyK: which doesnt change the point that the general assumption is wrong.00:48
Gallomimiaanyway, in ubuntu it's possible to have more than one swap file00:49
RoyKbekks: I've been using linux for a few years, almost 20, I know a bit about how it handles memory00:49
Gallomimiaon different volumes if desired00:49
bekksRoyK: I am, too. But still there is software out there that requires swap, no matter wether it is used or not. So again, the general assumption is wrong.00:50
lorenzTJ-: So, Memtest is now running, after one pass no errors. But I'm letting it run. What about the system?00:50
Gallomimiaagain you're correct in a general sense, but this guy told us his entire usecase and it doesn't actually need swap. aside from the fact that he's using up most of his memory in vm's00:51
RoyKbekks: no, software doesn't require swap. all memory usage is virtual. if the memory consumption is higher than what linux has, it swaps out. if vm.swappiness is set high, it swaps out early to free rarely-used pages for those who use them00:51
RoyKbekks: but no software require swap00:51
bekksRoyK: So did you read the Oracle RDBMS requirements section, actually?00:51
Gallomimia1. are we using oracle rdbms?00:52
RoyKbekks: I don't run oracle, so no00:52
RoyKbekks: I use open source software only00:52
bekksI can safely consider this discussion to be pointless then.00:52
lorenzI can confirm that, on many of my PCs and server I don't have swap set up and everything worked fine. If you have enough RAM it doesn't matter.00:52
Sachiru@lorenz: One pass isn't sufficient in my experience00:52
TJ-lorenz: how do you mean?00:52
SachiruI've seen several systems where it required two or three passes to catch intermittent errors00:53
RoyKbekks: when was oracle a part of this discussion before you introduced it?00:53
Gallomimia2. just because this one piece of software says it needs swap space doesn't mean all or any software actually needs swap space. it's much more likely that the software at hand is poorly written and abuses its privilege of having swap space.00:53
lorenzTJ-: Was a response to the swap discussion.00:53
SachiruMy current practice is to run memtest for at least 12 hours (three passes) before declaring it "semi-stable"00:53
SachiruUnless it's ECC RAM, in which case two passes is enough.00:53
Gallomimia3. no more oracle in FOSS discussions please00:53
lorenzSachiru: Yes, I'll let it run.00:54
bekksRoyK: It never was - again, I proved that the general assumption is wrong. However, I'm not going to discuss this any further, since you dont want to see the difference between the "Sachiru use case" and "the general assumption".00:54
TJ-lorenz: I wasn't reading it :)00:54
lorenzTJ-: What about the system? I can give you full change lists, but I've already looked through them and there's nothing interesting there.00:54
RoyKbekks: there's a little difference between running linux systems for 10+ years in production and "assumptions". I know linux. I know how it works. It doesn't need swap until the midden hits the windmill00:55
RoyKbekks: it's not assumptions - it's what I've learned over the years00:55
TJ-lorenz: I'm at a loss, really. Aside from memory corruption being flushed to disk I can't imagine what could cause those symptoms00:56
bekksAnd you havent touched Oracle yet and you are assuming how it should work. I'm off of this pointless discussion.00:56
RoyKbekks: so please, tell me why we should have a ton of unused gigabytes of swap on our servers ;)00:56
RoyKbekks: I Don't Use Oracle. Got it?00:56
bekksSo please just read the requirements section where this point is explained.00:56
bekksAnd I am off of this discussion. Got it?00:57
RoyKare we discussing oracle or linux?00:57
RoyKbekks?00:57
lorenzbekks: Noone here cares about Oracle requirements. It is simply not relevant.00:57
* RoyK diverts bekks to #ubuntu-offtopic00:57
=== Lcawte is now known as Lcawte|Away
RoyKSachiru: no, it's not *needed* to have swap on linux00:59
RoyKbut a small section of swap is always good00:59
RoyKsay, 1GB or 500MB or so00:59
RoyKso that linux can swap out things not in use there and use the remaining memory for something useful01:00
lorenzTJ-: I currently have a very similar system, which (after an upgrade from 12.04 to 14.04) runs for around 2 months. No problems there. My newer host (which is the one that fails) has a newer i7 4770, while the other host has the predecessor i7 3770. Any known issues with Haswell CPUs on Linux?01:01
RoyKSachiru: if you have a fast drive and little memory, it may be good to keep a 1-2GB swap space and setting vm.swappiness=10001:01
RoyKSachiru: if you have sufficient memory, it's no need01:01
lorenzRoyK: vm.swappiness=100 just kills the SSD because of the writes01:02
RoyKSachiru: if you have low memory and *slow* swapspace, set vm.swappiness=1 or so01:02
RoyKlorenz: no, it won't swap that much01:02
RoyKlorenz: it only swaps when it finds it necessary01:03
lorenzRoyK: I thought at 100 it would swap almost everything01:03
RoyKlorenz: and most good SSDs these days can handle rather a lot of writes, even cheap MLCs01:03
pakcjohello01:03
RoyKlorenz: no01:03
RoyKlorenz: it's just an index - at 100 it'll try to work out which pages to swap out earlier. the default is 6001:04
lorenzRoyK: Ok, something learned :)01:04
RoyK:)01:05
SachiruWell01:08
SachiruI was planning on using ZRAM01:08
Sachiru(compressed swap in RAM)01:08
SachiruI guess I can just use ZRAM and no swap01:08
SachiruThen vm.swappiness = 100%01:08
SachiruSo that it uses compressed swap in RAM always (essentially a dirty way of compressing RAM transparently)01:08
lorenzTJ-: Added files since install http://paste.ubuntu.com/7540516/ , Modified files since install http://paste.ubuntu.com/7540519/  (Excluded /usr/share/man, /var, /usr/share/doc)01:09
TJ-lorenz: As I said the other day; I blame one of those PPAs!01:11
SachiruOne more query:01:14
lorenzTJ-: Well Owncloud is certainly not the problem (Just a web application), Elasticsearch and Percona XtraDB are just database servers, so they aren't even started at the time it fails. That leaves Suricata and Ksplice. I already removed Ksplice because it integrates into initramfs and others and is therefore a candidate for the failure, but even after a complete uninstall it didn't work better. Suricada does not really interact with01:15
lorenzthe kernel (besides NFQUEUE), but I'm trying to remove it and see what happens.01:15
SachiruAs I said, rootFS shall reside on mirrored 8GB flash drives (Sandisk Cruzers). Should I max out the 8GB, or just partition 7GB and leave 1GB for free space (garbage collection, trim and all that).01:15
SachiruOr do USB drives not have TRIM?01:15
lorenz@Sachiru It depends on the USB adapter and the drive.01:17
SachiruSo, google the drive and research controller if it implements wear leveling01:17
SachiruIf yes, leave slack for overprovisioning, if no, allocate all and hope that OS is sane enough to wear-level on its own?01:18
patdk-lapyou do know it's rare for a usb flash drive to do garbage collection or trim?01:18
patdk-lapand I don't know of any that do it with <128gigs of flash01:18
lorenz@Sachiru I would assign the full 8GB, even that is barely enough for many applications01:19
SachiruNo intended applications01:19
lorenzpatdk-lap: I use a lot of Kingston KC300 60GB SSDs and they do TRIM01:19
SachiruThis is intended to be VM host01:19
SachiruFlash drives only host boot data for OS01:20
SachiruEverything else is stored onto ZFS data drives01:20
patdk-laplorenz, that is a usb stick?01:20
lorenzpatdk-lap: No, but still flash01:20
SachiruAlso a quick google shows that Sandisk Cruzers have very basic wear leveling (no trim or garbage collection but better than nothing yes?01:20
patdk-lapI said flash usb01:20
patdk-lapsure, you can get sata flash at 4gigs that do trim01:21
patdk-lapbut not usb01:21
SachiruAlso, query: RAID mirror of 2 flash drives where the /boot and / partition resides ensures that the system can boot from either of the two drives, correct?01:21
patdk-lapSachiru, nope01:21
SachiruOk01:22
patdk-lapyou have to make sure grub is installed to both01:22
SachiruYes01:22
SachiruAnd that too01:22
patdk-lapand that bios is set to boot both01:22
SachiruForgot to say that, sorry01:22
patdk-lapand hope the first one doesn't fail in a half state01:22
davidbowlbywhat's the best way to backup my old vmware ubuntu server instance and restore it to my openstack instance?  I'm not looking to do the full vmdk move because it takes too much space.  I've been playing with tar to zip the files and restore, but I'm obviously overwriting stuff that the cloud image uses to mount the boot volume.  Any help would be appreciated.  :)01:22
patdk-lap!best01:22
patdk-lapstupid bot01:22
davidbowlbyI've used the VMDK transfer method for my win2k8r2 VMs, but I figured since both the source and destination are running the same release of ubuntu I could probably do some form of backup/restore01:24
lorenzTJ-: Checked memtest again, still no errors at 3 passes.01:24
davidbowlbyit would save me a ton of space because I have quite a few ubuntu VMs in my old environment01:24
Sachiru@patdk-lap01:24
SachiruWhat would you recommend then?01:25
SachiruI want to make it so that the base OS can boot from either of the two drives01:25
patdk-lapyou can't01:25
SachiruSo that if one fails I can pull it out01:25
SachiruAnd put the other in01:25
lorenzSachiru: BTRFS can do that (I think)01:25
patdk-lapyou just have to hope the first one when it fails, fails badly enough, it boots the second01:25
SachiruAh01:25
SachiruHmm01:25
patdk-lapheh? btrfs seems to do a lot of things it can't01:25
lorenzpatdk-lap: Well it has checksumming and integrated raid, so it would detect where the faulty data is.01:26
patdk-lapthe only way to solve that fully, is to use a real raid card01:26
SachiruIs there an easy way to clone a USB drive?01:26
patdk-laplorenz, and at that point? your boot already failed01:26
patdk-lapand it won't boot the second disk01:26
patdk-lapso no, btrfs didn't fix it01:26
SachiruI'm thinking just use one flash drive, do weekly clone01:26
SachiruTo another01:26
patdk-lapit just didn't run the corruption, at best01:27
lorenzSachiru: A simple dd would work01:27
SachiruIn case primary fails just unplug it and plug in the latest clone01:27
SachiruDoes that work?01:27
patdk-lapwhy wouldn't it?01:27
SachiruOk01:27
SachiruIf I do LinuxRAID (2-way mirror) of the flash drives then unplug one01:27
SachiruIf I plug it back in later could I use MDADM to rebuild/re-clone the drive?01:28
lorenzyes, that would work01:28
lorenzbut it is a lot of work01:28
patdk-lapjust keep bitmaps turned on01:28
SachiruI think I'd go with LinuxRAID then01:28
SachiruLinuxRAID doesn't do "rebuild the moment the drive is plugged back in" right?01:29
lorenzNo01:29
patdk-lapmake sure mdadm is setup correctly to boot in degraded mode01:29
lorenzAt least not in default config01:29
SachiruOk01:29
SachiruThanks01:29
SachiruNow I know what to do. Thanks all! You've been such a great help01:29
lorenzTJ-: I left my VM running and it has made it to the promt where it says that /tmp, /boot/efi and /home aren't ready01:34
lorenzTJ-: Would point to a udev problem01:34
TJ-lorenz: Can you checksum the boot files (vmlinuz, initrd.img) and compare them against the originals before the problem?01:35
lorenzHas anyone of you had udev problems lately?01:35
lorenzTJ-: Well, there are no originals (at least no this time)01:35
lorenzTJ-: That system was installed, booted and hung01:36
TJ-lorenz: You have pretty amazing Linux-killing superpowers!01:36
davidbowlbycould I just boot my vmware ubuntu server and use rsync in some way to transfer its files to the openstack ubuntu server?01:36
davidbowlbyI have several servers for different purposes and don't want to lose their configs or data.01:37
lorenzTJ-: Yay! But I never asked for it :D01:37
lorenzTJ-: initrd.img is (nearly) the same as on my local host01:43
lorenzTJ-: Nope, my diff call was not recursive.01:44
lorenzTJ-: The diffs are ok, I also tried to boot it with my local initrd. It booted the kernel, but didn't do more than the old initrd.01:49
TJ-lorenz: something very weird then, it must be disk image corruption somewhere01:49
lorenzTJ-: BTRFS -- So no disk corruption possible01:50
lorenzTJ-: Booting with host vmlinuz and initrd works, but doesn't make it any better01:54
lorenzTJ-: I got something! I removed all udev and plymouth upstart jobs and now it boots quite quickly to the point where it fails to mount /etc/fstab volumes. Then I started a maintaince shell, which surprisingly worked and now I got the log message that mountall failed (but after 19 seconds, not the usual 300)02:04
lorenzTJ-: That's also the first boot which logged into kern.log!!02:06
TJ-how about starting udev now, manually02:08
lorenzTJ-: Ok, done that. I'll try now to continue the boot process.02:09
lorenzTJ-: Mountall seems to fail now.02:10
SachiruAnyone know how to configure memory limits for ZRAM on ubuntu? There's nothing in the official documentations and if there's anything on the forums about it 2 hours of googling apparently isn't enough.02:12
lorenz/etc/init/zram-config.conf02:12
lorenzprobably there (I can't check that because I don't have ZRAM)02:12
SachiruNope, no such file, and if it did exist it's empty (not even a comment)02:15
Gaba1after I install ubuntu server from USB it will not find the HDD to boot. I try ubuntu 12 and that works fine. anyone have any ideas?02:41
=== Gallomimia_ is now known as Gallomimia
=== Gaba1_ is now known as Gaba1
Gaba1hello, I am having an issue where I create a USB boot drive and when the installation gets to GRUB there is an error and then when I go to complete the installation all I see is a little white blinking underscore03:18
cfhowlettGaba1 verify your ubuntu ISO and the boot USB03:19
cfhowlett!md5sum03:19
ubottuTo verify your Ubuntu ISO image (or other files for which an MD5 checksum is provided), see http://help.ubuntu.com/community/HowToMD5SUM or http://www.linuxquestions.org/linux/answers/LQ_ISO/Checking_the_md5sum_in_Windows03:19
TohshI have a clean install of server 14.04 on an intel i5 3rd gen based machine. I am unable to boot normally without the nomodeset flag set on the kernel line in grub. It will attempt to boot and then freeze when adding swap. This has never been an issue in the past on previous releases for me. I have verified my UUIDs match in fstab and blkid. Any idea?03:25
=== thumper is now known as thumper-afk
=== ValicekB_ is now known as ValicekB
=== Ursinha-afk is now known as Ursinha
RoyKbeamed-up-zombie-day!07:48
sarnoldRoyK: do you have the day off?07:48
RoyKyep07:49
sarnoldnice07:49
=== Lcawte|Away is now known as Lcawte
Tazmainhi all how do I upload to a ftp that uses a username and password ? And where is the rsync.conf file in ubuntu I cannot find the default one.10:35
ikoniayou create the rsync.conf10:37
bekksTazmain: Use a ftp client. And you can provide the credentials on the rsync commandline.10:37
ikoniaand you upload to an ftp site that needs a username and password by puting in the username/password when prompted by the client10:38
Tazmainso do I set the location in rsync as the ftp client ? or ftp url ?10:40
bekksrsync has no gui, it is a command line utility.10:41
Tazmainthat I know10:41
Tazmainyou need to say in the config the location you are backing up to10:42
bekksYou dont need any config, you need the command line only.10:42
TazmainI want to automate it, doesn't it require a config then ?10:43
bekksNo.10:43
bekksIt requires a command line.10:43
Tazmainreally ? I am on the rsnapshot howto page and it says 4.1 create the config file. But okay if i just require command line I can make a bash script. So in the command I will also have to put the username and password then ?10:45
Tazmainbekks, if you run rsync as a deamon doesn't that require a config. Just trying to clear my confusion10:47
=== Ursinha is now known as Ursinha-afk
bekksTazmain: you dont run the client as a daemon.10:54
bekksTazmain: you are connecting using the _client_ to the ftp site, which doesnt require a config but a command line only.10:54
=== thumper-afk is now known as thumper
TazmainI meant rsync as a daemon10:58
bekksYou dont use the rsync client as a daemon.11:03
bekksAnd you do not configure the server to connect anywhere, thats what the client does.11:03
TazmainOh I understand11:04
bekksI told you three times now ;)11:04
Tazmainbekks so I could go rsync backup /var/www/stuff ftp ftp.location.com/var ?11:06
bekks"/var/www/stuff ftp ftp.location.com/var" doesnt make much sense. you can use rsync to transfer data to an ftp location, yes.11:12
Tazmainbekks okay do I need to mount the ftp location or how would I do the command? Could you give me an example if that isn't too much of a bother11:16
=== Ursinha-afk is now known as Ursinha
bekksyou dont need to mount the ftp location but you specify the credentials and the transfer protocol on command line. "man rsync" shows you how to do it, and you want to use the -e option.11:21
=== medberry is now known as Guest36553
Tazmainbekks, thank you very much I will have a look at that otherwise I will use curlftpfs and just rsync to that. Only thing I still want to ask is if I want rsyn to run everyday do I need to make it a daemon or just a cron task11:58
sarnoldcron task is probably fine12:02
sarnolddaemon mode rsync is nice if you have many clients all reading (or writing) to the same places12:02
Tazmainsarnold, so would you recommend cron ?12:20
sarnoldTazmain: yes12:20
bekksTazmain: the daemon is for creating an rsync server, nothing else.12:37
=== matsubara is now known as matsubara-afk
foolhardyI cannot seem to mount a "linux" partition: http://pastebin.com/DpJt5fx013:56
foolhardyany advice13:56
foolhardy?13:56
sarnoldfoolhardy: "Linux" is just a partition type, it doesn't tell you what is actually on that block device13:57
foolhardyhow can I find that out?13:57
sarnoldfoolhardy: try file -s /dev/sdb113:58
bekkssudo blkid13:58
foolhardy/dev/sdb1: sticky x86 boot sector, code offset 0x013:58
foolhardysudo blkid did not return anything regarding this new disk13:59
foolhardyonly the original os disk13:59
bekksSo there is no filesystem found.14:00
sarnoldyou could try leaving off the -t <foo> and see if mount can just figure it out14:02
sarnoldif not, try ext2, ext3, exg4, btrfs, zfs, ntfs, ntfs-3g, vfat, etc..14:03
=== swebb is now known as zz_swebb
bekks -t auto14:07
=== dannf` is now known as dannf
moparisthebestcan I get do-release-upgrade to just download all needed packages and not change anything else?14:17
moparisthebestbecause I have to do the upgrade remotely over a slow connection, so I'd like to have it download everything while I don't have to watch it, then just apply things later14:18
rbasakmoparisthebest: back up /etc/sources.list and /etc/sources.list.d/. Then replace all mentions of "saucy" with "trusty" (for example). Then run "apt-get update", then "apt-get --download-only dist-upgrade".14:23
rbasakThen change the files back. Correction: /etc/apt/sources.list and /etc/apt/sources.list.d/14:23
rbasakThen you'll have everything downloaded in /var/cache/apt/archives/, and a later upgrade will use those files instead of downloading against, except if there is anything new since you did that.14:23
rbasakinstead of downloading again14:24
moparisthebestah yea, guess that should work14:24
moparisthebestyea it should get the vast majority for me, thanks!14:24
=== medberry is now known as Guest71256
pakcjoHello, I think kerberos is starting before slapd and it should be the other way around, how can I confirm/fix this?14:58
sarnoldpakcjo: check if they have old-style init.d scripts or upstart scripts, that'll decide how you fix it..14:59
pakcjolooks like upstart15:00
pakcjosarnold: at least for slapd: http://pastie.org/923543515:01
moparisthebesthehe ok rbasak: Calculating upgrade... Failed The following packages have unmet dependencies: *snip* E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.15:06
moparisthebestso is there a way around that?15:06
rbasakI don't think that should happen unless you've got third party packages installed.15:06
rbasakIf that's right then you'll need to resolve those yourself.15:06
moparisthebestwell, this is a lucid 10.04 box15:06
moparisthebestit worked for downloading precise 12.04 packages15:07
moparisthebestbut failed with that when I tried trusty 14.04 :(15:07
geniiMight want to ppa-purge and try again15:08
moparisthebestthere is nothing in /etc/apt/sources.list.d/ never was15:09
geniimoparisthebest: When you say "this is a lucid 10.04 box " ... "it worked for dowloading precise 12.04 packages" ... Do you mean you upgraded 10.04 to 12.04 or that you installed 12.04 packages onto a 10.04 system?15:11
moparisthebestwell aptitude is getting the majority of them, that'll be good enough15:11
=== matsubara-afk is now known as matsubara
moparisthebestgenii:  no I'm trying to pre-download most of the packages I'll need for do-release-upgrade15:12
moparisthebestI'll be upgrading to precise, then immediatly to trusty15:12
moparisthebestso I changed my sources.list to precise, apt-get update, apt-get -d dist-upgrade15:12
rbasakOh, I see. I'm not aware of any way to get the trusty package downloads without upgrading to precise first.15:12
moparisthebestwhich worked, downloaded packages and did nothing else15:12
moparisthebestand then changed sources.list to trusty and did the same which broke a little15:12
rbasakI would strongly suggest redeploying trusty rather than upgrading all the way from lucid.15:12
rbasakUse configuration management, and test first.15:13
moparisthebestyou think so?15:13
moparisthebestI actually have a server I originally had 6.X on that currently has 12.0415:13
moparisthebestI've never upgraded twice in a row like this, but hopefully it won't have any issues15:14
rbasakThe general approach is much more reliable and reproducible. Know what you have deployed, rather than carrying issues forward.15:14
moparisthebestyea makes sense, what configuration management do you use?15:14
rbasakWe do support upgrading, but the nature of server distributions is that they involve local customisations, and upgrade scripts cannot always carry that forward.15:14
moparisthebestall I've ever used is backups and such :)15:14
davidbowlbyThis sounds like you should walk through it a couple times.  I suggest booting up a virtual machine running 10.04 and attempt the upgrades a couple times before playing with a production system.15:24
davidbowlbyyou can hose a VM over and over ;)15:24
moparisthebesthmm, never saw the --sandbox option for do-release-upgrade before15:27
moparisthebestthat should be pretty safe?15:27
moparisthebest-s, --sandbox         Test upgrade with a sandbox aufs overlay15:28
jpdsmoparisthebest: Safe as an upgrade gets.15:29
jpdsmoparisthebest: https://wiki.ubuntu.com/AufsBasedUpgrades15:32
moparisthebestah thanks I was trying to find more info about it15:33
davidbowlbyit wouldn't take you long to run through it in virtualbox15:33
moparisthebestwell except I don't have another exact copy of this machine in a VM15:34
davidbowlbyyou don't need an exact copy to get the procedure down15:34
davidbowlbyI'm talking about nailing down how you're going to do it15:34
davidbowlbythe rest is just data15:34
moparisthebestI've upgraded 10.04 to 12.04 on multiple desktops/laptops/servers before, it was just around the time 12.04.1 was released, so, awhile ago :)15:35
davidbowlbynot saying there can't be issues with packages you have installed15:35
davidbowlbybut you can do a dpkg list to find out what's on it15:35
davidbowlbybut you're not talking 10.04 to 12.0415:35
davidbowlbyyou're talking 10.04 to 12.04 to 14.0415:35
moparisthebestI've also upgraded from 12.04 to 14.0415:35
davidbowlbybut you're asking about the double upgrade15:36
moparisthebestthe same computers that *used* to run 10.04, and 8.04 before that15:36
bekksdavidbowlby: 10.04 to 12.04 to 14.04 includes 10.04 to 12.04.15:36
davidbowlbypersonally I'd want to walk through it on something I don't care about first15:36
moparisthebestI'd think it'd be about the same15:36
davidbowlbythinking and knowing are two different things15:36
moparisthebestyea the only thing that's giving me pause is I have to drive 10 miles if it won't reboot or something hehe15:36
=== zz_swebb is now known as swebb
davidbowlbyexactly, so why not be sure about it15:37
davidbowlbycreate a 10.04 vm15:37
davidbowlbywith the same packages installed15:37
davidbowlby(you can get a list from dpkg)15:37
davidbowlbyand then try it15:37
davidbowlbyif it goes well, you have a procedure15:37
davidbowlbyif it breaks, you didn't break a server 10 miles away15:37
davidbowlby10 miles lol, try 1600 miles away15:38
bekksmoparisthebest: Just use do-release-upgrade -s15:38
moparisthebestyea it's a good idea, just not sure if I want to try it or just screw it and roll the dice, like you said, it's not like it's 1600 miles away or more like the rest of my servers15:38
davidbowlbyI'd do the VM and document a procedure in confluence for future use15:39
davidbowlbybut that's me :D15:39
moparisthebestwell and this is my last computer on 10.04 so I don't ever need to do it agai15:39
davidbowlbyah15:39
davidbowlbybut you can always share it online for other suckers in your position15:39
moparisthebesthmm yea15:40
davidbowlby#contribute15:40
davidbowlbylol15:40
MTughanI'm trying to provision an Ubuntu 14.04 image on an OpenStack installation and am switching to using WaitConditions. I get a URL from the wait condition and install the heat-cfntools package (to get cfn-signal), but running cfn-signal returns a Python import error: "No module named heat_cfntools.cfntools". I can't find the module anywhere on the system.15:54
MTughanAm I just missing a package that needs to be installed? Is something missing from the package? Something else entirely going on?15:54
fridaynextanybody here use phpvirtualbox on ubuntu server? any comments on performance?16:08
fridaynexti'm currently remoting into KVM from a VM on my OSX laptop, and that's too kludgey for me. I want to simplify.16:08
fridaynextI already have OpenVPN installed, so it seems phpvirtualbox would be a perfect companion to access my media server's VMs from anywhere.16:09
=== medberry is now known as Guest71557
davidbowlbyMTughan, did you install heat-api-cfn  ?16:24
MTughandavidbowlby: No, because I think that's the server component.16:25
davidbowlbyI think heat-engine is16:25
MTughanLooking at its dependencies, you may be right.16:25
davidbowlbyhttps://packages.debian.org/hu/sid/web/heat-api-cfn16:26
MTughanYeah, the name suggests that it's the API endpoint, but maybe it's not.16:26
davidbowlbyI'd say based on your error message, it's a good thing to try ;)16:26
MTughanIt's weird though, looking at the OpenStack git repo for heat_cfntools, the module's in that repo. http://git.openstack.org/cgit/openstack/heat-cfntools/tree/16:27
davidbowlbyThis package contains the CloudFormation (CFN) API.16:27
davidbowlbyI assume you installed heat-api16:27
davidbowlbythat error is specific to cfn though, so I think apt-get install heat-api-cfn will fix it16:27
MTughanIf the packages were available, perhaps.16:28
MTughanE: Failed to fetch http://nova.clouds.archive.ubuntu.com/ubuntu/pool/main/h/heat/python-heat_2014.1~rc2-0ubuntu3_all.deb  404  Not Found [IP: 91.189.88.153 80]16:28
davidbowlbyhmm interesting16:28
davidbowlbysudo apt-get update16:28
MTughanI'm trying an update now.16:28
davidbowlby;)16:28
davidbowlbygmta16:28
MTughanNormally, I have cloud-config do an update on boot. But I think that was skipped this time because of an issue I had while testing.16:29
MTughanYeah, now it's working.16:29
davidbowlbysweet, I just started playing with OpenStack last week16:30
davidbowlbygot tired of esxi hardware support continuing to dwindle16:31
davidbowlbymigrated my mysql server last night using rsync, that was nerve racking, but went well16:33
davidbowlbyluckily I was smart and used DNS in my environment lol16:33
MTughandavidbowlby: Neither heat-api nor heat-api-cfn fixed it.16:36
MTughanI had thought because I didn't do an update that heat-cfntools might not've been up-to-date, but upgrade didn't touch it or fix it.16:37
=== pHcF_ is now known as pHcF
lordievaderGood evening.17:00
=== andreas__ is now known as ahasenack
=== medberry is now known as Guest45057
=== Ursinha is now known as Ursinha-afk
pakcjowhich is the init.d script for dhcpd?18:03
PryMar56 dhcp3-server18:05
pmatulispakcjo: if you want something lighter and progressive you may consider dnsmasq.  it doesn't have all features though18:07
pakcjoPryMar56: I don't have that (Ubuntu 12.04.4 LTS)18:09
pakcjoI have isc-dhcp-server and isc-dhcp-server618:10
PryMar56pakcjo, I checked lucid18:10
pakcjoI'm asking because it seems that there was a change, /etc/dhcp instead of /etc/dhcp3/18:11
pakcjois there a better tool than update-rc.d to see which init.d scripts are enable?18:12
PryMar56pakcjo, ls -al /etc/rcS.d  ; ls -al /etc/rc2.d/18:12
PryMar56if you see symlinks to /etc/init.d/ the service is setup18:13
pakcjohmm interesting, I have a S07dhcp3-server but it is not in /etc/init.d/18:14
pakcjoso, is this a new upgrade bug?18:15
PryMar56pakcjo, that is a dead symlink18:15
PryMar56its harmless18:15
pakcjoPryMar56: yes, but I don't have anything to start dhcp with...18:16
PryMar56pakcjo, run update-rc.d against the new isc-dhcp...18:17
pakcjoPryMar56: which? isc-dhcp-server or isc-dhcp-server6? why I have 2?18:18
PryMar56do you want to hand out ipv6 addresses? I don't18:19
pmatulispakcjo: there may be an upstart job18:19
pmatulis/etc/init18:20
pakcjooh 6 is for ipv6, thanks ;)18:20
pakcjopmatulis: oh yes, there is :)18:20
timhansengood afternoon18:23
pakcjohello timhansen18:23
timhanseni just created a new 14.04 server on linode. installed lamp, and setup a virtualhost. added the entry into my local hosts file, but when i go to the fake domain, i’m getting the index view of the html directory (i removed the default ubuntu index.html)18:25
timhansenhere’s the output of my virtualhost: https://gist.github.com/willc0de4food/cc014d072cf42eb538dc18:25
timhansenany ideas as to what i did wrong?18:26
pakcjois there a way to set order of init.d scripts?18:30
pakcjoif I manually change the symbolic lick from S06 to S05 will that work?18:35
pakcjo(is that the correct way of doing it?)18:35
pmatulispakcjo: is this kerberos & slapd again? normally those links should not be changed18:38
pakcjopmatulis: yes18:38
pmatulistimhansen: remove 'Indexes'?18:39
pakcjoI have to start them manually after a reboot... I was thinking of adding them to /etc/rc.local but there should be a better way...18:39
timhanseni figured it out. i’m absent-minded & didn’t name the vhost with .conf -.-18:40
=== sync0pate is now known as tcdev24
=== tcdev24 is now known as sync0pate
pmatulispakcjo: doesn't kerberos only need access to slapd when it receives a request?  why would it depend on it to start up?  are you also storing kerberos/kdc config information in slapd?18:44
pakcjopmatulis: yes, ldap is used as db for kerberos18:45
pmatulispakcjo: so what is the current problem?18:45
pakcjopmatulis: kerberos tries to start, fails with couldn't connect to ldap server, slapd starts18:45
pakcjoi need slapd to start before kerberos18:46
pmatulispakcjo: i wonder why?  it must do a check when it starts18:46
pakcjoa check?18:46
pmatulispakcjo: yeah, it obviously does a check of some sort18:47
pmatulispakcjo: would be great if you could pastebin some logs from kerberos18:47
pmatulispakcjo: but it's weird that such a problem has not already been identified18:48
pakcjothere isn't much: kadmind[1333](Error): Can't contact LDAP server while initializing, aborting18:48
pakcjohttps://bugs.launchpad.net/ubuntu/+source/krb5/+bug/65243318:49
uvirtbotLaunchpad bug 652433 in krb5 "Init script dependency error: krb5-kdc starts before slapd" [Low,Confirmed]18:49
pmatulisah18:49
pakcjofrom 2010 XD XD XD18:49
pmatulispakcjo: that bug explains things fairly well.  were you not able to get a workaround out of it besides manually changing the symlinks?18:57
pakcjowell the bug report suggest to change order19:00
pakcjoand rc.local XD19:03
pakcjoanother reason to get rid of slapd19:03
sixBBhello.  I am exeperiencing problems with installing/upgradiing udev.  Thie configure process hangs.  I've googled and not tound anything pearticularly helpful19:17
sixBBany suggestions?19:17
=== croftworth is now known as leeve
=== leeve is now known as croftworth
pakcjowhat is the correct way to start/stop services in /etc/init/?20:26
Patrickdkservice xxxx (start|stop)20:29
pakcjothasnk Patrickdk20:30
pakcjoso start xxxx and stop xxxx is for something else?20:31
pakcjoor they are aliases?20:31
Patrickdkdunno20:31
lordievaderpakcjo: There are many ways, using initctl is another way ;)20:31
Patrickdkhmm, start/stop is aliased to initctl20:32
lordievaderAh, that is new to me. :)20:32
pakcjothanks20:33
MTughanpakcjo: service using start and stop in the script for start and stop.20:33
MTughanAnd status and reload for those two as well, apparently.20:33
MTughanAll of which are aliased to initctl by softlinks.20:34
MTughan"restart" is just accomplished by calling stop followed immediately by start.20:35
Patrickdkah, service is just a script, that runs init.d or initctl for init20:35
MTughanExactly.20:35
MTughanShell script, even. Thought it might be Python, but I guess that'd be overkill.20:35
Patrickdksince upstart, I switched to using service for everything20:36
MTughan(sorry, I've been doing a lot of Python lately)20:36
PatrickdkI've been doing lots of lua scripts lately :)20:36
Patrickdkvery low latency :)20:36
MTughanYeah, service should be your goto for service-y stuff (no pun intended).20:36
pakcjonested db in ldap is a bad idea right? is there any documentation about it?20:47
pmatulispakcjo: it makes things unecessarily complex20:48
pakcjopmatulis: do you have any link about where I can read about it? or a guide to switch/port to hmm "lineal?" model?20:49
pakcjoflat model20:49
pmatulispakcjo: no, but IMO it should just be a matter of changing the slapd config so it points to the new location20:50
pmatulispakcjo: i idle in #openldap , you might get better opinion there20:50
pakcjopmatulis: it's nested vs flat or is there a better terminology for not nested?20:50
pakcjothanks20:50
pmatulispakcjo: i don't think 'nested' is a great term20:51
pakcjo:)20:51
pmatulispakcjo: it usually implies one thing working within another.  but in this case, it's just a matter of directories.  a location and not something in the logical/working aspect20:51
pakcjopmatulis: thanks20:52
pmatulisex: nested kvm.  a hypervisor which is actually a guest of another hypervisor20:52
pmatulispakcjo: i know i have at least one cleanup/re-install shell script that would wreak havoc if i had a database located within the directory of another database ;)20:56
pakcjo:)20:57
__dan__hi guys, hope you're all well, been having consistent hangs with btrfs defrag on ubuntu 14.04 stock kernel, was wondering if there is a guide to updating the kernel to latest and also updating the btrfs supporting programs? any guidance appreciated, cheers :)21:14
Patrickdkdon't use btrfs till it's stable?21:20
__dan__when will that piece of string reach it's end?21:25
__dan__also, I'm not using the more cutting edge stuff like dedupe or parity raid, I'm comparing between ZFS on Linux and btrfs on the same hardware and would like to evaluate btrfs latest, preferably in a safe non-hacky manner I was hoping to get some help with that21:27
Patrickdkdunno, watch btrfs.wiki.kernel.org21:27
__dan__uh huh21:27
Patrickdkthey don't say btrfs is stable yet, at all21:27
Patrickdkonly that the ondisk format is stable, not that the code/runtime is stable21:27
__dan__well, I believe the man page etc was changed last year to reflect it's relative stability from where it was when that man page was originally written21:28
Patrickdkwell, consider the wiki was updated less than a month ago21:28
Patrickdkand says NOT STABLE21:28
PatrickdkI would assume it's not stable21:28
__dan__the situation is a little more complicated than you seem to be giving credit21:29
rberg___dan__:  check here for builds of upstream kernel releases.. http://kernel.ubuntu.com/~kernel-ppa/mainline/21:29
Patrickdkhow is it more complicated?21:29
__dan__because some parts of the codebase are more mature than others21:29
__dan__hi rberg_ thank you I will take a look21:30
rberg_as for userland you may need to build it yourself.21:30
* Patrickdk just hopes he can use btrfs, without dedup/compression/snapshots/...21:31
Patrickdkjust to store files21:31
Patrickdkwithout it exploding and becoming completely corrupted21:31
__dan__ah yes I saw this earlier, I'm running trusty and would like to keep it that way, is it OK to run the utopic kernels? and is there a way to set it up to update automatically? (ie. an apt repo that has latest 3.14 for instance?)21:31
Patrickdkhad it do that way too many times21:31
rberg_heh the way btrfs and zfs do snapshots is REALLY nice..21:32
* __dan__ nods :) I'm a FreeBSD refugee and used to running ZFS but ZFS on Linux is highly not recommended for 32-bit machines - guess what I'm running :/21:32
__dan__to be fair it's not the best idea on 32-bit FreeBSD either but I ran it with no issues whatsoever after the initial tune-up period21:33
Patrickdkthere are still 32bit systems?21:33
* __dan__ rolls eyes21:33
Patrickdkthought they died out with pentium D cpu's21:34
methoanyone successfuly migrated bind to windoz server?21:40
=== elliotd123_ is now known as elliotd123
__dan__metho: I'm no DNS expert but can't you just configure windows as secondary and have it pull the zonefiles from bind primary?21:48
__dan__can't say I know anyone crazy enough to migrate DNS to windows though sorry :) i dare to even ask your use case21:49
qman__Migrating a DNS zone to MS DNS isn't hard, but it's not an ubuntu question21:51
__dan__maybe on some level he came here so we could talk him out of it :)21:52
qman__As for migrating _from_ MS DNS _to_ bind, making bind the secondary, transferring the zone, then reconfiguring as a master is a good method21:54
Joe_knockHello there23:05
Joe_knockWhenever I SSH into my remote server, I get a small introduction about the system, how do I make this introduction appear whenever I want?23:06
lutostagJoe_knock: I believe you are talking about motd (message of the day), which is created by the scripts that live in /etc/update-motd.d/23:16
Joe_knocklutostag: is there a simple command to make it display? Or were u telling me by saying motd?23:16
Jordan_UJoe_knock: cat /etc/motd23:20
Joe_knockJordan_U: Thank you23:22
Jordan_UJoe_knock: You're welcome.23:23
=== Lcawte is now known as Lcawte|Away

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!