[07:36] <mapps> hmmmmmmmmmmmm]
[08:49] <MartijnVdS> http://www.estofex.org/cgi-bin/polygon/showforecast.cgi?text=yes&fcstfile=2014060806_201406062211_1_stormforecast.xml
[09:29] <brobostigon> morning boys and girls.
[09:56] <foobarry> finished my massive lancaster in time for d-day
[09:56] <foobarry> http://i.imgur.com/YzSsD5e.jpg
[09:59] <foobarry> http://i.imgur.com/D16flk6.jpg
[10:07] <bigcalm> Good morning peeps :)
[14:02] <NET||abuse> quick question, i have an external disk, what's the fastest fs to run for a backup, offloading my backup files from laptop, about 100GB worth, and updating :)
[14:03] <NET||abuse> so i want to put ext4 on the disk, but there are options, eg, btrf, xfs, fat32... just being informed from the dropdown on gparted
[15:13] <daftykins> NET||abuse: "KISS"
[15:52] <foobarry> hilarious bing advert thread on facebook
[15:53] <foobarry> read the comments https://www.facebook.com/BingUK/posts/842486845766093
[15:54] <daftykins> there aren't any
[15:54] <daftykins> requires login to see perhaps?
[16:05] <foobarry> mebbe
[16:59] <bigcalm> popey: ping
[17:09] <popey> yo
[19:32] <bigcalm> Argh
[19:32] <penguin42> argh?
[19:33] <popey> ARGH!
[19:33] <bigcalm> df shows that /tmp is at 100% usage with 1.9GB
[19:33] <bigcalm> du -chs /tmp shows 136K
[19:33] <bigcalm> This is on a client's server
[19:33] <bigcalm> Where /tmp is its own mount
[19:33] <ali1234> foobarry: the really hilarious thing is when you go to bingiton.com and click "learn about the study" at the very bottom of the page
[19:33] <penguin42> bigcalm: Is it a real /tmp or is it a shmfs?
[19:33] <penguin42> bigcalm: One thing that causes that is a deleted file that's still open
[19:34] <bigcalm> From mount: /dev/mapper/vg_root-lv_tmp on /tmp type ext3 (rw)
[19:34] <bigcalm> I see
[19:34] <penguin42> nod, ok, so probably a deleted file that's still open
[19:34] <bigcalm> How would I find such a file? :)
[19:35] <penguin42> bigcalm: You could try lsof | grep deleted
[19:35] <penguin42> bigcalm: Deleted files don't free the space until every user closes them
[19:36] <bigcalm> Ta
[19:37] <bigcalm> Wow, that's a lot of open files
[19:38] <bigcalm> All of them by mysql :)
[19:40] <bigcalm> Screw it, I'm going to stop mysql and see what breaks
[19:40] <bigcalm> That's what weekends are for, right?
[19:40] <penguin42> bigcalm: but you greped for deleted ?
[19:41] <bigcalm> penguin42: yes
[19:44] <bigcalm> penguin42: http://discworld.cuth.eu/dump/deleted.txt
[19:45] <penguin42> wth is cuth?
[19:45] <bigcalm> It's me :)
[19:45] <penguin42> bigcalm: mysqld     3353        mysql  253u      REG              253,2  1902923776         23 /tmp/ML9IzY9V (deleted)
[19:45] <bigcalm> Yep
[19:45] <moreati> bigcalm: mario?
[19:46] <bigcalm> Heh, hiya moreati :)
[19:46] <bigcalm> moreati: how was the barcamp?
[19:46] <penguin42> bigcalm: You got some type of dump/backup script that does crazy things like that?
[19:46] <moreati> good thanks
[19:46] <ali1234> you can view the file contents through /proc
[19:46] <ali1234> if you haven't already restarted it
[19:47] <bigcalm> I know what the content is. I'm trying to make things usable again
[19:47] <bigcalm> If I stop mysql, I'm guessing that the /tmp files will become visible or actually get deleted?
[19:47] <ali1234> they get deleted yes
[19:47] <ali1234> all you really need to do is make mysql close that file
[19:48] <bigcalm> Know how?
[19:48] <ali1234> well, what's in the file?
[19:48] <bigcalm> Oh, um
[19:48] <bigcalm> How do I "view the file contents through /proc"? :)
[19:49] <ali1234> look in /proc/3353/fd with root
[19:49] <ali1234> that's all the file descriptors opened by mysql, by fd number
[19:49] <ali1234> it will be one of those
[19:49] <bigcalm> Ta
[19:49] <ali1234> possibly 23? not sure how to read the output of lsof
[19:51] <ali1234> yeah 23
[19:52] <bigcalm>  /proc/23 doesn't exist
[19:52] <ali1234> /proc/3353/fd/23
[19:52] <penguin42> bigcalm: do an ls -l /proc/3353/fd  it'll show you what fd is
[19:52] <penguin42> bigcalm: for pid 3353
[19:55] <bigcalm> I seriously don't understand what I'm doing now :)
[19:56] <ali1234> it's probably a temporary file for a query
[19:57] <ali1234> try 'file /proc/3353/fd/23'
[19:57] <ali1234> or hexdump -C
[19:57] <bigcalm> ali1234: that would make sense as things broke when I tried to run a query to delete a lot of data from a table
[19:57] <ali1234> did you do a massive join?
[19:57] <bigcalm> # file /proc/3353/fd/23
[19:57] <bigcalm> /proc/3353/fd/23: symbolic link to `/data02/mysql/mysql/tables_priv.MYI'
[19:58] <bigcalm> No, I did a massive delete
[19:58] <ali1234> hmm that's the wrong fd then
[19:58] <bigcalm> The table in question is currently over 15GB
[19:58] <bigcalm> I'm trying to delete some of it so that the server's hdd doesn't get full and break the client's website
[19:59] <ali1234> what query did you do?
[20:00] <bigcalm> DELETE FROM tb_mass_email_recipient WHERE enu_status = "sent" AND dat_sent < @archive_date;
[20:00] <bigcalm> Oh, and this before it:
[20:00] <bigcalm> SET @archive_date = DATE_SUB(NOW(), INTERVAL 30 DAY);
[20:00] <ali1234> hmm...
[20:01] <ali1234> well, most likely you'll have to restart mysql
[20:01] <bigcalm> I need to sort out the server so that I can start again and set the interval to be 365 days and slowly work towards now
[20:01] <ali1234> is that query still running?
[20:01] <bigcalm> No, it broke after 2 hours
[20:01] <ali1234> ah... well, that's definitely the problem then
[20:01] <bigcalm> And now I can't view any table because /tmp is full
[20:02] <bigcalm> Here goes
[20:02] <ali1234> restart it, and try running the query with a limit clause
[20:02] <ali1234> i dunno if that will help, but it might
[20:02] <ali1234> i mean maybe it's trying to do the delete atomically
[20:03] <bigcalm> I've just issued stop, it's taking a while
[20:03] <bigcalm> Yay, /tmp is showing 1.8GB free now
[20:04] <bigcalm> There are 2.2 million rows in this table I'm trying to archive
[20:08] <bigcalm> Great, crashed tables. Why am I not surprised?
[20:08] <ali1234> well yeah.
[20:09] <bigcalm> I may regret having issued REPAIR TABLE from my SQL client
[20:11] <bigcalm> Now I know to keep an eye on /tmp
[20:18] <bigcalm> Whom ever designed this one table - I wish them harm
[20:23] <ali1234> caching outgoing email into a table seems like a common design pattern
[20:24] <ali1234> amusing when someone moves the website to another server and doesn't realise it requires a cronjob to actually send the email
[20:30] <bigcalm> But wasn't a job to clear out the cached data once the emails had been sent
[20:33] <ali1234> yeah that's low priority :(
[21:38] <bigcalm> Crikey, I'm having to run the query at most 10 days at a time. Might even have to drop it to 1 day per query
[21:38] <bigcalm> Well, query set
[21:38] <bigcalm> I'm also running OPTIMISE TABLE after the delete to free up space
[21:44] <Armag3dd0n24> what is the best mobile network for free Internet?
[21:44] <Armag3dd0n24> obviously in the UK lol
[21:44] <ali1234> bigcalm: maybe try DELETE QUICK, see http://dev.mysql.com/doc/refman/5.0/en/delete.html
[21:44] <ali1234> or add a limit clause
[21:45] <bigcalm> ali1234: will give it a go, ta
[21:46] <ali1234> "If you are going to delete many rows from a table, it might be       faster to use DELETE QUICK followed by       OPTIMIZE TABLE. This rebuilds the       index rather than performing many index block merge operations."
[21:46] <bigcalm> Ta
[21:46] <bigcalm> Though it's also about freeing up space on the HDD
[21:46] <ali1234> yeah, the optimize should do that
[21:47] <bigcalm> Ah, still use the optimize then, goodo
[21:47] <ali1234> if you just delete a row it would just leave a gap in the table file
[21:48] <ali1234> so it has to remove that gap... that's probably the bottleneck if you just use regular delete
[21:48] <bigcalm> I have a feeling I should run optimize on some other client's tables :)
[21:48] <bigcalm> I see
[21:50] <ali1234> i'm uncompressing 120000 tar.bz2 files, each one is about 1MB and contains around 50-100 files... the whole thing is on a XFS filesystem using deadline schedule... guess how long it will take?
[21:51] <bigcalm> A while?
[21:51] <penguin42> so 120GB of data uncompressed or compressed?
[21:52] <penguin42> so ok 120GB of data but dominated by small files - which ain't XFSs strong point mind
[21:52] <penguin42> ali1234: I'll toss my bet in at 15mins
[21:53] <ali1234> yeah, and deadline optimizes for reading... and it's a 5400 rpm drive
[21:53] <penguin42> ohhhhh
[21:53] <ali1234> it's been going for over a day and it's done about 30%
[21:53] <penguin42> ouch
[21:54] <ali1234> i changed it to cfq and it's going much faster now
[21:54] <penguin42> wow that big a difference?
[21:54] <ali1234> seems about 4x faster
[21:55] <ali1234> still freezes up the computer every now and then
[21:55] <penguin42> probably when it realises it's filled RAM up with stuff to be written
[21:56] <ali1234> yeah... i've got about 8GB in the disk cache and 1GB used on swap
[21:58] <ali1234> once i've decompressed it all then i'm going to run findup -m on it and recompress it into larger archives, this should give a 10x decrease in compressed size if my experiments on a small sample are correct
[22:00] <ali1234> how do i check the free inode count on xfs?
[22:00] <penguin42> xfs is always a bit of a mystery to me
[22:00] <ali1234> df -i apparently
[22:00] <ali1234> i forgot this drive was xfs...
[22:00] <ali1234> should have reformatted it before i started
[22:01] <penguin42> oh df -i is just the standard
[22:01] <ali1234> hmm. only used 8% of inodes... should be okay
[22:10] <bigcalm> I'm considering setting up a rack in my garage once I move house. That said, I don't know where to start in sourcing parts. Should I just buy something from HP/Dell/eBay or build something from scratch?
[22:10] <ali1234> you mean the actual rack itself?
[22:11] <ali1234> you can build a rack fairly easily, you just need some L-bar and some DIY skills
[22:13] <bigcalm> Both rack and server to go in it
[22:13] <ali1234> it depends how fancy you want it really
[22:14] <bigcalm> I need a powerful set-up for running multiple VMs at the same time. The HP Proliant microserver I have isn't up to the job.
[22:15] <bigcalm> It'll basically take over the job of my dev server
[22:15] <ali1234> you could just purchase one of those ubuntu cloud boxes ;)
[22:16] <shauno> I'm curious why you're aiming for a rack?  they only really make sense where you require the density
[22:16] <ali1234> yeah, i'd just go with a tower for this
[22:16] <shauno> almost any other scenario a tower is going to be more machine for less money
[22:18] <ali1234> also i wouldn't put computers in a garage, they will rust
[22:18] <ali1234> been there done that
[22:18] <ali1234> same goes for cellars
[22:20] <ali1234> i'd get a tower, put one of these in it: http://www.scan.co.uk/products/icy-dock-mb996sp-6sb-6-sata-hot-swap-525-bay-backplane-raid-cage and a load of these: http://www.scan.co.uk/products/960gb-crucial-m500-25-sata-iii-6gb-s-slim-7mm-ssd-marvell-500mb-s-read-400mb-s-write-80k-iops
[22:21] <ali1234> lots of RAM, and i5 or i7
[22:21] <bigcalm> I either need a quiet yet powerful machine (my laptop is just about up to the job, but I don't want to have it as my dev server). Or a not so quiet machine that I can stick somewhere out of the way
[22:21] <ali1234> that would be quiet if you go i5 ^
[22:22] <ali1234> and get lots of fans instead of one huge one
[22:22] <ali1234> not silent mind, but you wouldn't hear it from another room
[22:22] <bigcalm> Thinking about it, the new house has a utility room that is more part of the house than the garage
[22:24] <penguin42> ali1234: I have a crucial v4 on my desk with bad sectors and less than a year old
[22:25] <ali1234> v4 is like 2 generations old though
[22:27] <penguin42> hmm
[22:28] <bigcalm> Hotel booked for the RAT \o/
[22:53] <bigcalm> Looking for bare bones machines, keep being shown ultra compact bricks
[22:54] <bigcalm> I'm really out of the game
[22:57] <ali1234> don't buy a barebones
[22:57] <ali1234> why would you even do that?
[22:57] <ali1234> you'll get a crappy loud case with noisy fans and a rubbish PSU if you do that
[22:59] <bigcalm> Okay, looking at other things now
[22:59] <bigcalm> http://www.ebuyer.com/581296-hp-proliant-ml310e-gen8-724160-035
[22:59] <bigcalm> Though I have no idea what Xeon CPUs are like
[23:00] <shauno> if your laptop is ordinarily up to the job, xeon's likely to be overkill.  and they rarely hit a nice bang/buck ratio unless you actually need that overkill
[23:01] <bigcalm> So they are generally higher performance than the 4th gen i7 I have in this laptop?
[23:02] <ali1234> laptop i7s aren't the same as desktop i7s
[23:02] <penguin42> bigcalm: The single socket xeons are very close indeed to the equivalent spec i7
[23:03] <shauno> now that I think of it, I'm not actually sure what a xeon's selling point is.  it used to be dollops of cache, but desktop processors have caught up with that pretty well
[23:10] <penguin42> I thought it was ECC memory
[23:15] <penguin42> although it looks harder to compare these days - no 100% matches to the same model as the xeon
[23:31] <popey> evening all
[23:31] <popey> uh, morning all
[23:31] <penguin42> hey Popey
[23:31] <popey> hows tricks?
[23:31] <penguin42> yeh ok, yourself?
[23:31] <bigcalm> Morning popey
[23:32] <popey> tickety boo
[23:32] <bigcalm> popey: hotel booked
[23:32] <popey> which one?
[23:32] <bigcalm> Good question
[23:32] <popey> holiday inn?
[23:32] <bigcalm> Yes, that one
[23:33] <bigcalm> £59 quid with Expedia
[23:33] <bigcalm> And 8% cash back via quidco
[23:33] <popey> oh nice
[23:33] <popey> good deal
[23:34] <bigcalm> Yeah, lot cheaper than getting a room in Farnham
[23:34] <popey> interesting hotel
[23:34] <bigcalm> How so?
[23:34] <popey> So, Queen Victoria used to get the train from London to Farnborough
[23:34] <popey> Then a carriage from the station to that hotel
[23:34] <popey> She would stay there
[23:35] <bigcalm> I've used Holiday Inn else where in the country. Hopefully the quality is standard
[23:35] <bigcalm> Cool
[23:35] <popey> And then inspect the troops on the nearby parade grounds
[23:35] <popey> it has a lot of history
[23:35] <popey> we looked at that place for our wedding reception
[23:37] <bigcalm> Big place
[23:37] <bigcalm> Quite looking forward to it now
[23:38] <popey> yay
[23:39] <bigcalm> Next need to book a hotel for oggcamp
[23:39] <bigcalm> Which might be the Holiday Inn as well :D
[23:40] <bigcalm> Know if anybody has booked hotels yet?
[23:41] <bigcalm> I can hear movement up stairs
[23:41] <bigcalm> Might be time to go to bed
[23:41] <bigcalm> Toodles
[23:44] <popey> i have no idea about hotels for oggcamp
[23:51] <dutchie> surely the venue itself is the obvious choice
[23:52] <popey> yeah, bit spendy though
[23:53] <dutchie> i wonder if there are any guest houses in wolvercote
[23:54] <dutchie> i guess there are the travelodge and the holiday inn right there
[23:55]  * dutchie could probably just stay with postgrad friends in central ox though :D