[00:25] happy birthday AlanBell === chalcedny is now known as chalcedony === issyl0 is now known as Guest96732 [06:18] gord: :( [06:20] will not be online today, at the hospital wth the bf getting his tonsils out, anyone looking for me email me [06:21] thanks popey [06:21] I am 6^2 now [06:38] happy b'day AlanBell :) [06:42] thanks [06:43] wife just came down and asked if there was anything interesting happening on facebook [06:43] "apparently it is my birthday" [06:43] "oh yeah, happy birthday" [06:53] hahaha :) [08:24] Morning all [09:00] HELP! :) [09:00] selinuxium, Hi it look like you're trying to type a letter, can I help? [09:01] Trying to start up PC today and the encrypted drive is unmounted. [09:01] encrypt-mount-private does nothing.. === daubers_ is now known as daubers [09:01] selinuxium: Encrypted using the installer? [09:02] Not sure if an update may of borked something [09:02] hey daubers, yup [09:02] selinuxium, encrypted home? [09:02] JGJones: yup [09:02] you get any output with encrypt-mount-private? [09:04] in home only two files... a README.txt and an Access-Your-Private-Data.desktop [09:04] JGJones: non at all... [09:04] JGJones: unless I run a root... [09:05] if run as root I get Inserted auth token XXXXXXXXXX into the user session keyring and an fopen [09:06] Did you save the original passphase when setting up the encrypted home? [09:06] Oh.... Sorry... Morning everyone! :) [09:11] Morning all [09:13] Morning DJones [09:14] you got any experience with encrypted home parts not mounting at boot? [09:14] selinuxium, Did you save the original passphase when setting up the encrypted home? [09:15] JGJones: Emailed it to myself.. [09:15] Morning all! [09:17] hey popey [09:17] o/ [09:19] selinuxium, you're saying that when you run ecryptfs-mount-private in terminal, it doesn't even ask you for the password? [09:20] JGJones: nope.. [09:27] hmm - you should be prompted for a password when running that command. [09:28] I assume you've rebooted just in case? [09:30] JGJones: several times... [09:30] Not too much on this PC.. [09:31] You can boot into a LiveCD/USB and recover your encrypted home [09:32] Happy to just flatten and start again, but would like to be able to find out how to fix... :) [09:33] JGJones: recovering it? or just be able to read it? [09:33] selinuxium, reading it [09:33] there's a guide you could follow for reading it in a LiveCD - http://www.cyberciti.biz/faq/ubuntu-mounting-your-encrypted-home-from-livecd/ [09:34] I would try reinstalling ecryptfs-utils? [09:35] hang on, did you try ecryptfs-recover-private? [09:36] JGJones: nope... [09:36] you'll need to run that command as root [09:36] it'll search for encrypted private directories [09:36] OK.. :) running it atm. [09:37] tum te tum tum tum..... [09:44] JGJones: Unwrapping passphrase and inserting into the user session keyring failed [-5] :/ [09:47] so ecryptfs-recover-private did find your encrypted home and prompted your for your login passphase? [09:49] or did you use your mount passphase (that's the random string of alphanumeric characters)? [09:49] JGJones: yup.. GOing to check to see if I have any othedrs knocking about... [09:50] selinuxium, I just wanted to clear up - which passphase did you use? [09:50] mount passphase or login password? [09:53] JGJones: passphrase [09:53] did you try your login password? [09:53] JGJones: nope. [09:53] oh... [09:53] numpty! [09:54] the error message was saying unwrapping passphrase - it have your mount passphrase as I understand it and you need login password to continue [09:55] JGJones: Private data mounted read-only at /tmp/ecryptfs.XXXXXX :) [09:55] going for a reboot and see... :) [09:55] lovely - at least your data's not screwed - obviously it's time to find out why it's not doing it at boot. [09:56] all data available in the /tmp directory [10:01] oh - what's missing? [10:02] .... [10:02] * JGJones slaps self... [10:02] need to read properly [10:02] ALL data available - not Some data available. [10:05] Hello. [10:13] Hello [10:15] selinuxium, did a reboot work alright? [10:23] JGJones: nope... Stil same issue... [10:23] Interesting... May just get data off and rebuild... :) [10:24] try reinstalling ecryptfs-utils [10:24] apt-get --reinstall install ecryptfs-utils [10:25] JGJones: think that will be the next step.. [10:36] JGJones: THat worked! :) [10:37] JGJones: very strange, but learnt lots about ecrypt I didn't know.... CHeers :) [10:41] good morning everyone. [10:43] hey brobostigon [10:43] hey selinuxium [10:45] Hmm... just examined a RAID that has a 5.2TB block device that apparently has a 167PB file on it [10:45] stupid metadata fail [10:46] interesting situation, [10:58] i want to get some better performance out of my software raid setup on my microserver [10:58] I should post my question to askubuntu [10:58] yes, I'll do that [11:01] morning all [11:01] popey: What do you get from it at the moment? [11:01] o/ davmor2 [11:02] morning daubers how's the windows experiment going? [11:02] davmor2: Good. I'm writing another blog post, but the task I'm doing for that one is taking some time :) [11:03] daubers: ah you're trying to print something right ;) [11:03] davmor2: heh, you joke, but it took me 15 minutes to get the office printer installed on the blasted thing [11:04] daubers: haha [11:04] It discovered the network printer easily, but doesn't bother automatically sorting the driver. Asks me to select it from a list which doesn't contain it, then click a button to try and find it on Win Update, 5 minutes of waiting and then go through the list again to find it [11:05] In ubuntu it was just "Goto the printers setting box, click add, it appears in the list, click ok, it works" [11:05] daubers: at least you found it I had one where I did all that and it still didn't show up so off the the manufacturers website I went and got it manually no fun at all [11:05] davmor2: Rubbish [11:06] daubers: what that you have to use windows or it's no fun at all tracking down drivers :D [11:06] the second part :p [11:07] Most of the stuff I do everyday is now platform agnostic, so I haven't really noticed that much getting in my way [11:08] daubers: the best is if you have a generic piece of hardware (like a webcam) and windows say oh I know what that is, says it works and it does and you can't persuade it different and can't find the driver details on the supermarkets web site and no info on Google that's real fun [11:09] daubers: can you tell I used to fix pc's for a while? [11:10] davmor2: Just a tad [11:11] it doesn't work even [11:12] Hmmm [11:12] May have to switch USB stick suppliers, these ones are twice as slow as the last batch :( [11:13] daubers: no the last batch you were using linux it's windows that is slow ;) [11:13] davmor2: These are in linux :) Can't use windows at work, _everything_ is linux based in the workshop [11:15] haha [11:30] daubers: just writing it up :D [11:30] popey: Fair enough [11:30] its a lot to write up [11:30] so I am doing it in one place :D [11:30] popey: If you've got mroe than 3disks in there, then I'd suspect the slow bit to be your network (depending on RAID mode) [11:31] nope, not network bottleneck [11:31] this is all local IO [11:31] Desktop grade drives? [11:31] its got 8 disks (9 if you include the boot disk) [11:31] yes [11:31] RAID 5? [11:31] no [11:31] 10 [11:31] Urgh [11:31] There's your problem :) [11:32] i've never known software raid not to suck [11:34] * popey wonders whether more useful information is to follow [11:34] popey: You have 2 4 disk stripes mirrored? [11:34] my software raid does not suck [11:34] there, now you know [11:34] Mine doesn't suck, I just wonder if I can do better [11:34] popey: Also what size disks? [11:34] and RPM [11:35] 2x2TB and 4x500G [11:35] Urgh, that will also cause problems [11:36] sorry [11:36] 4x2TB and 4x500G [11:36] 4x2TB is md0, 4x500G is md1 [11:36] md0 and md1 are one giant lvm [11:37] So they're concatenated? [11:37] however md1 is new to the mix so very little io there as lvm uses the md0 disks first [11:37] popey: So I'd suspect you get maybe ... 90 MB/s? [11:38] popey: what speed RPM are the disks? [11:38] Streaming [11:38] http://askubuntu.com/questions/49022/how-do-i-improve-my-server-disk-performance/612 [11:38] oimon: one sec [11:39] popey: http://www.redhat.com/magazine/009jul05/features/lvm2/#fig-striped ? [11:39] trying to figure out [11:39] I'd rather not run smartctl against them [11:39] any other way to find out [11:39] MartijnVdS: I'd avoid that on mismatched drive sizes [11:40] popey: Create a big file from /dev/zero with dd [11:40] It will give you speeds :) [11:40] popey: check /sys for serial numbers, look up on manufactuerer's website [11:41] yeah, rummaging [11:46] popey: Ideally in your situation, I'd move the 2TB drives into a RAID5. Depending on the file size I'd set the stripe width to something between 128k and 256k and I'd setup ext4 with the stripe-width option to match [11:46] I switched from RAID5 to RAID10 because I was led to believe that write performance was better [11:46] I'd then rsync mirror that onto the external box, as that will be limited in performance to a single SATA channel (i.e. the esata link) [11:47] I'd be surprised by that, as for every write to the block device you're doing two write (i.e. one to raid0 1 and one to raid0 2) [11:48] If you're using it as a metadata controller for a san where the writes a maybe a couple of blocks in length, then I'd agree. [11:48] how can I tell what stripe width is currently in use? [11:49] should say in mdstat [11:49] 64K [11:49] I take it that's the default? [11:49] Yep, whats the average file size you're copying? [11:49] no idea [11:49] lots of small files though [11:50] small as in less than 1MB or less than 100MB? [11:50] less than 1M [11:50] (small for me is less than 1GB_ [11:50] this is a home server [11:50] not a corp san [11:50] 64k should be ok then. [11:51] so 4x2TB as RAID5, not 6? [11:51] I'd RAID 5 it, there's little performance difference between 5 and 6, but in that setup you would be better off with raid 5 for capacity [11:51] i used a program called iozone to map my drive performance. you can then graph the results. e.g. http://ubuntuone.com/p/zUW/ [11:52] popey: There's also a lovely program called dstat which will give you a MB/s count (on screen and to a file) [11:54] popey: OOI, whats the total size of all the data that's being copied? [11:54] it'd be nuice if ubuntu one links preserved file extensions [11:54] yep...it's an ODS file [11:54] ~700GB [11:54] BigRedS: preserved it here [11:54] you don't know until you click it though [11:55] popey: no, I mean in the link itself, so oimon's link becomes http://ubuntuone.com/p/zUW.xls [11:55] not for me it doesnt [11:55] popey: You're using rsync to copy? [11:55] so before clicking on it I'd know OOo is going to try to run away with all my memory [11:55] daubers: cp [11:55] which virtual machine software is favoured these days? [11:55] virtualbox [11:56] cheers [11:56] popey: You're achieving about 99.5MB/s, which is close to the theoretical cap of what 2 desktop drives in a stripe could do in that situation [11:57] allow about 50MB/s per desktop drive [11:57] hmm [11:57] popey: Move it into RAID5 and you should get about 50MB/s more [11:58] heh [11:58] thats going to be fun :D [11:58] moving it all about [11:58] popey: This does depend on the speed you can read the data off the source drives though :) [11:58] sustained 90.5MB/s shouldn't be sniffed at. [11:58] err 99* [11:58] source drives? [11:58] it's cp [11:58] from dir to dir on same disks [11:59] popey: Ah! In that case it's even worse than that :) Certainly in that scenario you really shouldn't sniff at the performance you're getting [11:59] * oimon wonders if MooDoo is awake [11:59] hah [11:59] hmm, i could fail out half the raid 0 to make two disks available [11:59] I find that if i benchmark writes to /dev/null, it blows the water out of /dev/sda [11:59] and make a degraded raid 5 from two 2TB [12:00] reading+writing on the same drives _really_ hurts any spinny disk system [12:00] then move the data over and expand when done [12:01] or just leave as is, if this is the best I can expect [12:01] 10K RPM drives help? [12:01] popey: I'd leave as it. 10K drives will help, SSD will help even more :) [12:02] yeah, 2TB SSD... [12:02] not that I'm using all that space [12:02] * daubers was happily pulling 1.5GB/s from an 8 disk raid 5 with SSD's last week [12:02] popey: The other thing you could do is use rsync, so you're not copying files that haven't changed [12:03] thats not technically possible I dont think [12:03] Why not? [12:03] its a clean copy [12:03] You copy the files then blat the other directory? [12:03] _I_ don't do this [12:03] rsnapshot does [12:03] and no, thats not quite how it works [12:04] Hmmm [12:04] so you're using rsnapshot into a dir, then making a copy of that dir? [12:05] no [12:05] rsnapshot is a wrapper around rsync/cp [12:05] workflow is fun [12:06] http://rsnapshot.org/screenshots.html [12:06] :D [12:06] note cp -al [12:07] uses hard links so you're not copying all files every time [12:07] and rsync ensures you're not transferring all files every time [12:08] Ah! Now I think I understand [12:10] Yeah, that cp will be your bottleneck in rsnapshot. If you want to get a true idea of what you're disks can supply you with, then you'd need to benchmark one disk in the set and multiply up. Generally desktop drives are between 50/60MB/s [12:11] Any idea what chipset the server uses? [12:11] yes, see the askubuntu thing [12:11] at the bottom [12:11] I mean North/South bridge [12:11] oh, uhm [12:12] http://paste.ubuntu.com/627905/ [12:12] That'll define how many pci-e (or equiv) lanes are going to your sata controller [12:12] 00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] (rev 40) [12:12] thats the internal one [12:12] 02:00.0 RAID bus controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev 01) [12:12] thats the external one [12:13] why does "ATI [...] SATA Controller" still look so wrong [12:13] AMD RS785E/SB820M chipset [12:13] (not saying it's wrong, it just looks jarring) [12:16] when that cp is running (which it is now because I have rsnapshot run at midday [also, yay, lunch]) I can't even get an ls in a reasonable time [12:17] more RAM might help I guess? [12:17] disk cache [12:17] Mem: 1925784k total, 1901548k used, 24236k free, 952244k buffers [12:17] Swap: 3905528k total, 12868k used, 3892660k free, 19772k cached [12:18] without a block diagram I'd be tempted to say the most you can expect from that would be 250MB/s (one pci-e 1.1 lane) [12:18] more ram will help the ls [12:18] I also try to stream video from it [12:18] its my get_iplayer box [12:19] and that struggles when an rsnapshot kicks in [12:19] Yeah, you'll hit the iops limits of the spinny disks [12:19] so it ends up doing the backup near constantly [12:20] as soon as one finishes, the next is ready to start [12:20] the pitfalls of a low powered server :D [12:21] popey: To improve that I'd use you're external box as the get_iplayer repo [12:21] As a seperate volume [12:21] good call [12:21] I'd also see if you can nice down the rsnapshot during the day, or just let it run at night when you're less likely to be using it for streaming vids [12:22] good call [12:22] * popey modifies his cron job [12:23] Otherwise, the only way to improve it's performance is more ram, better proc, SSD's etc. You might get a bit more by tweaking the ext4 stripe_width settings, but it won't do miracles [12:23] we have horrendous problems deleted bazillions of tiny files [12:23] ta [12:24] popey: superhub replaced, v+ box replaced \o/ [12:24] yay [12:24] Bloke couldn't find anything wrong though [12:24] we run snapshots with hard linking too, sometimes takes days to reduce the number of files [12:25] nice -n19 should do it [12:37] Meh, locked myself out of the router's web interface. Silly me [12:43] guys - anyone done much work on xml + xslt in IE? === bigcalm__ is now known as bigcalm [12:56] If the superhub didn't reboot every time I added a mac/ip pair, this wouldn't be so messy [12:57] andylockran, Some experience, why? [13:18] s-fox: IE is returning the transformed XML as a string, whereas firefox returns it as a document fragment [13:21] just tried the tweetdeck app inside chrome...ugh :( [13:37] popey: I had a play around and posted an answer on your AU question. [13:39] nice [13:39] thanks! [13:40] * popey times writing to the disk that isnt in the array first [13:44] 52s :D [13:44] whilst an rsnapshot is running [13:45] Probably not the fairest of test [13:46] mdadm does seem to have some odd defaults... Unless they're there for a reason I don't yet understand. [13:47] box only has 2GB RAM [13:47] I have 8GB here I'm memtesting in another server which I'll put in later [13:48] I only have 6GB in this at the moment (with a desktop running on it too) so that might be enough to boost the caches right up. [13:48] popey: it's an Atom, isn't it? [13:48] AMD [13:48] similar though [13:48] laptop cpu [13:55] hmm, only 4GB here, not 8GB [13:55] ah well [14:00] * bigcalm returns from lunch to find his 'net connection still up [14:00] So far, so good [14:01] :D [14:01] Hmmm these seem interesting... http://www.businessdirect.bt.com/products/seagate-500gb-momentusxt-hybrid-sata-300-2-5--7200rpm-32mb-70JK.html?utm_source=retention+email&utm_medium=email&utm_content=I900&utm_campaign=email140611 [14:01] Sorry, should of cut the crap off the back of that URL [14:02] selinuxium: gets fairly mediocre reviews http://www.pcpro.co.uk/reviews/hard-disks/360301/seagate-momentus-xt-500gb [14:03] Hmm... Saw some benchmarking and it looked good.. [14:03] Although TechRadar were backflipping in joy by the end of their review: http://www.techradar.com/reviews/pc-mac/pc-components/storage/disk-drives-hdd-ssd-/seagate-momentus-xt-500gb-710193/review [14:05] Could imagine a couple raided in a Media PC would be pretty sweet... [14:06] I'm tempted to put an SSD in my laptop to double the speed of it. [14:07] * dwatkins looks into hybrid drives as well [14:07] bah [14:07] If you had space for 2 2.5" disks, I'd go with a pure SSD and a pure mechanical. Much more predictable performance. [14:07] unetbootin made a usb stick that wont do memtest [14:08] * popey tries a memtest usb thing [14:09] memtest is tiny, I think I have a 512 MB USB stick somewhere... ;) [14:09] sorted [14:09] its running now [14:10] if it passes I'll put that ram in my microserver [14:11] memtest would actually fit within 512 KB. It's only 160KB + boot overhead [14:13] Oli``: that dd takes 8 seconds when the box is fairly idle [14:14] stripe_cache_size doesnt exist here [14:16] popey: Sounds like sequential write is bombing along nicely then [14:16] yeah [14:16] I'll have a google through some of those settings [14:17] earlier daubers did some calculations here and seemed to think I'm getting expected throughput for non-enterprise 7200rpm drives [14:18] 120MB/s on RAID10 is 60MB/s per pair so yeah, I guess that isn't terrible but it's not great. [14:20] popey: Please write a blog post about this! [14:20] Oli``: popey isn't doing straight streaming reads, he's read/writing from the same volume. So what he's getting is relativley good for a software RAID10 [14:21] Oli``: the copy in the post seems to be writing at ~90MB/s to the volume (so 90MB/s to each pair of drives simultaniously as well as reading the stuff from the disk) so it's pretty good [14:22] usually i get better results writing over the network than from disk-disk [14:23] oimon: One disk to a second disk (either over network or seperate physical disk) is always quicker than a copy from the disk to the same disk. Less head movement required in the first instance [14:24] Daviey: which bit? [14:25] ooo, need to go buy wifeys birthday pressie [14:25] ubuntu earrings? [14:25] * daubers needs to get a birthday card for his wife on the way home [14:25] already booked a party at pizza hut for tomorrow evening \o/ [14:25] i don'ti have a new theory, no one is allowed to complain about missing/moved/hidden system trays in linux until they have had to code something that implements it =\ [14:25] popey: that should now be the number one priority for you [14:27] daubers, a question for you re your 30 days with Windows...it's Windows 7 yes? Does it solve that annoying problem with USB where if I plug in a USB device...say a mouse into 1 port of USB, it'll say it's looking for driver and then it's ready...there's a delay... [14:27] and then other time you put in mouse again...but into a different usb port and does the whole installing driver crap all over again? [14:27] JGJones: no, it still does that [14:27] Worse with printers - get Printer, Printer (Copy), Printer (copy 2) etc? [14:27] how annoying [14:27] sigh. [14:28] lol [14:28] MartijnVdS, I get it all the time on a laptop with Windows...ok NOT that annoying, but still...I shouldn't need it to install drivers for the same sodding hardware everytime. [14:29] or USB memory stick. or get a laptop for Windows with a single USB port(!) [14:36] popey: generally, about your RAID experience. [14:36] tuning, throughput etc [14:37] guys I got an initramfs error again, I am not on the live disk. Here is the output of fdisk -l http://paste.ubuntu.com/627966/ [14:37] what has happened at line 12? [14:42] AlanBell, popey ? [14:43] safiyyah: Thats nothing to worry about. Just means your partition doesn't end on a cylinder boundary on the disk (not an issue, but with older spinny disks can lower performance marginally) [14:43] safiyyah: See http://en.wikipedia.org/wiki/Cylinder-head-sector for more info on cylinder/head/sectors on disks [14:46] Also... http://www.reghardware.com/2011/06/16/retro_gamer_day_of_the_tentacle/ <- Best point and click game _ever_ [14:46] DoTT!! [14:46] o/ [14:46] \o/ even [14:47] okay so continue with mounting the broken installation [14:47] ? [14:49] daubers? [14:58] btw how do you kill the ubuntu software centre? I x killed it but still it isn't dead as I cannot use synaptic or command line [14:58] !aptfix [14:58] If an APT front-end crashed and your database is locked, try this in a !terminal: « sudo fuser -vki /var/lib/dpkg/lock;sudo dpkg --configure -a » [14:59] dutchie it stills says locked [15:00] http://paste.ubuntu.com/627976/ [15:00] * BigRedS has always just rmd the lock file [15:00] rmd? [15:01] i had that a couple of days ago. apt locked, nothing running [15:01] first time i've ever seent hat [15:01] lol ali it's not funny [15:01] i have an initramfas error again [15:01] popey not around to help me lol [15:02] initramfs* [15:02] how did you solve it ali1234 ? [15:03] it fixed itself after a while [15:04] safiyyah: you could try resizing the first partition (while booted from a live cd) [15:04] safiyyah: to match the boundary [15:04] safiyyah: you can use a graphical tool for it (gparted?) [15:05] !initramfs [15:05] hmm [15:05] then for debugging: [15:05] !grub [15:05] GRUB2 is the default Ubuntu boot manager since 9.10 (Karmic). Lost GRUB after installing Windows? See https://help.ubuntu.com/community/RestoreGrub - For more information and troubleshooting for GRUB2 please refer to https://help.ubuntu.com/community/Grub2 - See !grub1 for releases before Karmic (9.10) [15:07] what does the initramfs error say? [15:07] maybe you just need a new intird? [15:28] BigRedS: I don't remember what it said [15:28] is there a command I can run? [15:29] mkinitrd will create a new initrd image which may help. But the important bit is that it creates one different to the current broken one [15:29] and to do that, you need to know what's broken [15:30] for which you need the error [15:30] (or a log if it did eventually boot into something) [15:30] Okay [15:30] I will reboot and write the error down and come back [15:30] thank you guys [15:58] okay am back [16:01] aha, with errors? [16:02] right my initramfas error reads as follows http://paste.ubuntu.com/628013/ I would like to add that I have not plugged in any new USB devices but I did run the updates that pop up twice in the last couple of days [16:03]