[00:05] [Jonathan Riddell] 12.04 Feature Freeze This Week - http://blogs.kde.org/node/4536 [00:05] [Jono Bacon] Ubuntu Accomplishments Lens - http://www.jonobacon.org/2012/02/15/ubuntu-accomplishments-lens/ === gary_ is now known as Guest36891 [00:21] supposing I had a zillion png files and I wanted to convert them to jpg [00:21] if i put them all on a NFS share and then write a converter program that silent skips png files when a jpg already exists (or rather, if creating it fails due to it already existing), could I run that across multiple machines - ie a cluster? [00:21] it seems like someone would have tried this before. does this type of system have a specific name i can google for more information? [00:22] ali1234: sounds like a trivially parallel task, but you need to manually parallelize your problem set [00:22] it is trivially parallel yes [00:22] i'm not actually doing png to jpg [00:22] even if this means "for i in a*.png" on machine 1, "for i in b*.png" on machine 2, etc [00:23] but the idea is the same: each input file creates exactly one output [00:23] it's a lot slower task as well so bandwidth isn't an issue [00:23] do i really need to split the work like that? can't i just let the NFS handle it for me? [00:24] ie try: for i in *.png... except OSError: pass [00:24] or whatever exception you get when you try to create a file that already exists [00:25] th ereason i ask is because i'll be writing that code anyway so that the whole job can be continued if it is stopped [00:28] i'd like to be able to add and remove machines fromt he cluster at will too :) [00:35] well i'm just going to implement it and see what happens [00:42] ali1234: your problem if you do it in order is obvious, no? [00:42] the obvious problem is that every node will attempt to process every file [00:42] machine 1 sees no foo.jpg, so starts converting foo.png. machine 2 sees no food.jpg, so starts converting foo.jpg. machine 1 finishes conversion, writes foo.jpg [00:42] that supposes that my program doesn't attempt to open the file *first* and then lock it [00:43] that needs to be an atomic operation of course [00:43] the output file i mean [00:43] if that fails, it silently goes to the next file [00:44] the the first instance opens foo.jpg, opens foo.png, converts, writes to foo.jpg [00:44] the second instance attempts to open foo.jpg, fails, then goes to foo1.jpg [00:44] this seems like an obvious idea. it might mean a lot of collisions if all instances were converting several files per second, but they are not [00:45] the actual conversion does about 1 file per minute per CPU core [00:47] actually it would do this: [00:47] if output file exists and is exactly 42 * 32 bytes, skip input file [00:47] what you really want is a parallel job scheduler. [00:47] else attempt to open output file for exclusive writing [00:47] if that fails, skip input file [00:47] else, process the input file [00:48] let's say you have a million images, and 10 machines. you don't split it into ten 100,000 lots, you split it into a whole lot of 60 image lots. [00:48] submit this whole bunch of 60-image jobs to the scheduler [00:48] i specifically don't want to write a job scheduler [00:48] so use one of the shelf? believe ir or not, other humans have done this stuff before [00:49] i've looked at every available system for doing this [00:49] yeah, shoulda stayed in my game rather than trying to help [00:49] they all either don't work like this, are so low level you have to write most of it yourself, or would require me to rewrite the whole thing in java [00:52] this is about the time that people tell me just to use hadoop [00:53] why not just have 1 comp do it over a longer period of time? [00:55] because computers get rebooted in the real world [00:55] therefore the software must be able to skip already processed files [00:55] the easiest way to do that is just to check if the output file already exists [00:55] so, as i said, i have to implement that check anyway [00:56] now, i can either: [00:56] 1) implement the check in a slightly nuanced way that makes it possible to cluster the software with zero extra work [00:57] or 2) go and spend a month learning hadoop and then rewrite the entire software for a feature that will save me perhaps 1 week of processing time [00:58] or index the files in a separate text file, numbered, then process them in that order..... but logging to a file the ones done [00:58] that's basically 3) implement my own half-assed job control server [00:58] so you know which file to start on next time [00:59] yeh [00:59] the worst of all possible options :) [00:59] half-assed is better than been fully assed, if you can't be assed [00:59] ;) [01:00] if it is a one off, I wouldn't go to the trouble of setting something great up [01:00] just do something to get it done [01:00] anyone know how I view POSTDATA in firefoxes web console? [01:01] It was always turned off by default, but now I can't see the on button for it anywhere [01:01] aha, right click on the net button :P [01:20] this is absolutely a job for condor. but i'm sure ali1234 already rejected that one. [01:20] Anyone know a little about RSA and mind me poking them with a couple questions? [01:28] condor eh? [01:29] well let's see. it works by wrapping system calls so my software will run in it [01:29] what it doesn;t do is any kind of job control [01:29] i would need to implement that myself [01:29] so let's say i implement that using the scheme i've outlined above [01:29] then i could run my software on NFS [01:29] *or* i could run it inside condor [01:30] so, condor doesn't actually help me at all [01:31] nobody in the history of computing has ever thrown a bunch of shell scripts at a job scheduler before, clearly [01:32] i'm sure they have [01:33] i think i'm going to go to bed. you have fun with your unique little computing problem [01:33] see here's the problem [01:33] you have totally failed to answer my very simple original question [01:33] all you;ve done is say "that's easy, use this", and then suggested a solution that doesn't actually help at all [01:33] and made lots of sarcastic remarks [01:35] " if i put them all on a NFS share and then write a converter program that silent skips png files when a jpg already exists (or rather, if creating it fails due to it already existing), could I run that across multiple machines - ie a cluster?" [01:35] yes. [01:35] there. question answered. [01:35] was that really so hard? [01:36] however, the scheme you keep outlining for doing it is retarded. but you're really not interested in hearing it. [01:37] but since anything anyone ever in the history of creation has ever said which goes against your preferred technical solutions is clearly wrong, i'll leave it as an exercise to the reader as to why [01:38] nothing says "it's bedtime!" like six years running a parallel computing facility, eh? [01:41] on the contrary, i'm very interested in hearing it, assuming it is a logical, fact-based reason, rather than an argument from authority [01:41] which is so far all you have presented [01:42] isn't there a command to run a command on another machine? [01:42] hamitron: "ssh" [01:42] hmmm, it was something else I recall [01:43] or maybe to run a command on lots of machines [01:43] * hamitron shrugs [01:43] ali1234: fine. in my experience, this is a task which is suited to a batch job scheduler. there are two main classes of batch job schedulers - cycle scavangers like xgrid or condor, and dedicated managers like torque and gridengine [01:45] ali1234: if i were to try and accomplish your task, i would write a bash script which could translate an array job name (e.g. 1098[4].hostname) into some unique set of images to process (an hour is a good measure), submit a big freaking array job, and let the scheduler keep throwing out chunks of work to needy nodes as & when free nodes become available [01:46] ali1234: e.g. if you have a million images numbered 0000001 to 1000000, then do some basic division of $PBS_ARRAYID in your submission script so a single job works on, like, 60 images [01:47] ali1234: batch job scheduling, and shell scripting, are solved problems. "for i in foo do myapp $i" isn't rocket science either. [01:49] how will that system handle a restart of every machine in the entire cluster? [01:49] the job gets marked failed by the scheduler, and is subsequently rescheduled. [01:49] by "the job" i mean "the specific jobid of the job running on a failed device", i.e. one piece of an array juob [01:50] or more than one piece [01:51] worst case the failed jobs get marked as held, and need to be released by the scheduler admin [01:52] so this system requires a database of jobs which are either currently running or failed? [01:52] and presumably the completed ones too [01:53] ali1234: batch submission systems use a daemon on a "master" node, which handles submission of jobs to nodes, and accounting of status. you can read historic information such as job run time, queued jobs, etc [01:55] e.g. http://www.clusterresources.com/torquedocs/commands/qstat.shtml === Lcawte is now known as Lcawte|Away === Twinkletoes_ is now known as Twinkletoes [02:01] i'm totally going to bed now, i have to drive to cambridge tomorrow [02:01] ok. goodnight === joar_ is now known as CosmicB [07:23] morning all [07:54] Morning folks [08:00] morning === himcesjf1 is now known as himcesjf [08:52] aloha [08:52] morning [09:04] Morning [09:30] Morning! [09:31] * BigRedS is surprised each morning to find that 'morning' doesn't tab-complete [09:31] Good morning :) [09:48] From apticron this morning [09:48] * /etc/lsb-release, /etc/issue, /etc/issue.net: Bump version number to [09:48] 10.04.4 in preparation for the point release. [09:48] Surely it should wait until April is here before that change is made? [09:50] * czajkowski waves [09:50] Ello czajkowski :) [09:51] Do I want this as my wedding ring or will people think it's tacky? http://www.kinektdesign.com/product-gear-ring.php [09:53] bigcalm: I think you'll spend forever cleaning it and getting fluff and dirt out of the groves [09:53] czajkowski: that seems likely :( [09:57] gord: new ssd may not fit in x220. bay is 7mm, drive is 9.3mm ⍨ [09:57] Goodness [09:57] popey, yeah you have to be careful, intel ones you can make fit [09:58] too late, ordered the ocz one [09:58] I didn't consider that when I bought one for my laptop. It does fit though thankfully [09:58] http://www.ocztechnologyforum.com/forum/showthread.php?89168-Vertex-3-X220-compability&p=645699&viewfull=1#post645699 [09:58] might fit, others have managed [09:59] I thought there was a standard for laptop drives? [09:59] Good morning all! :) [10:00] Good Morning! [10:01] hmm on the latest update I swich to a different application [10:01] I click on terminal on the left hand side menu and instead of showing me this irc window as before it launches a new terminal [10:01] oh but I do seem to have hud [10:01] * czajkowski peers at gord you broke my laptop didn't you :p [10:02] popey: found it http://thereifixedit.failblog.org/2012/02/03/white-trash-repairs-when-in-doubt-close-your-eyes-and-shove/ [10:04] haha [10:04] Thought you'd like that [10:04] bigcalm: there are numerous standards ☹ [10:05] popey: I think you mean BigRedS ;) [10:09] bah [10:09] just managed to slide another 9mm drive in [10:09] missus [10:09] No worries :) [10:10] Oh my [10:10] such a tight squeeze, the "VOID if removed" stickers came off [10:10] Whoops [10:10] if this goes in, it's never coming out! [10:10] (again, missus) [10:10] Take an angle grinder to it [10:11] some people take the shell off the device [10:17] Ah, that's the lovely thing about standards :( [10:19] and compiz crashes [10:19] bugger [10:21] Why do my chair arms keep sinking? [10:21] Silly Aeron === Lcawte|Away is now known as Lcawte [10:30] * popey wants aeron [10:32] * bigcalm gives popey an Aero [10:32] Sorry, the n fell off [10:32] ah id love an aero [10:32] nice mint one [10:34] good morning everyone, [10:42] morning [10:42] morning MooDoo [10:49] Hayley's laptop has a bad case of the malwares. Tempted to wipe and put Ubuntu on it. Think she might notice the difference though [10:50] bigcalm: not if you theme it to look like windows lol [10:51] or temporarily dualboot [10:52] wubi? [10:52] If MS Office could run 100% perfectly without any faffing on Linux, then I think she would give it a go [10:54] bigcalm: http://www.ylmf.org/en/ [11:07] bigcalm: just tell her it's office 2013 [11:07] "complete redesign" [11:08] reformat. "forget" to back up something important like sims saves. that's always educational for windows users. [11:09] morning all [11:10] morning davmor2 [11:10] MooDoo: morning dude [11:13] davmor2: how are you br'a :) [11:14] MooDoo: tired, but not as tired as czajkowski I wager [11:15] davmor2: why what's she been doing? [11:15] Hi davmor2 [11:16] MooDoo: she staying just up the road shhhhhhh [11:16] bigcalm: hey dude how's life [11:17] davmor2: huh? [11:17] DJones: I don't think that she cares that much about the look of the OS, more that MS Office won't work 100% [11:17] davmor2: still full of curry from last night [11:17] You? [11:18] MooDoo: She has a mini sprint/training with mrevell for her new job so got the 06:23 train to wolves this morning and her hotel is in stone throwing distance [11:18] bigcalm: nice [11:20] \o/ ssd installed [11:21] o/ [11:23] * jpds watches davmor2 throw some stones. [11:23] jpds: no czajkowski needs to be in the hotel for me to hit her with them :P [11:24] jpds: the worrying things is would throw them back :D [11:24] davmor2: Mean. [11:26] * czajkowski waves [11:28] * davmor2 gives czajkowski an empathy hug early trains and working all day == no fun [11:28] I can haz XKCD progress bar in Ubuntu plox? [11:28] I wants XKCD progress bar. [11:28] it's not the worst [11:28] and I get to learn stuff [11:28] so win win tbh [11:29] And you're in Wolverhampton, the centre of the Linux world! [11:29] (I might be embellishing the truth a little) [11:32] just a tiny bit [11:38] Myrtti: man, yeah [11:38] Myrtti: pull random facts from wikipedia based on the displayed time [11:38] Oh man that needs to be implemented [11:39] IKR?!?! [11:39] ☺ [11:39] had to decipher IKR. I like. [11:40] bigcalm: also the centre of Wolverhampton has pork roll shops [11:40] * popey uses gparted to copy/paste his partition from rust to ssd [11:40] Are any of the Thomas train engines black? [11:41] (from the "on topic questions" file) [11:42] Does anyone know the etiquette about finding sponsors for uploads? Is it considered ok to prod someone on IRC? [11:43] i think one is [11:43] happen to know its name? [11:43] the story is that I was waving at it while canoeing on Sunday and then didn't notice the approaching rapid and ended up mashing my face down some rocks [11:43] * Myrtti throws money at the computer screen [11:43] Donald and douglass are black :) [11:43] Mavis [11:43] please someone implement XKCD progress bar [11:43] and Neville [11:44] http://www.myfavoritetoys.com/character-guide/thomas-friends-character-guide.html?limit=all [11:44] "Neville is a black steam engine who looks more like a diesel engine. " [11:44] yeah i think it was neville [11:45] the passengers were waving back :( [11:45] aside: the Llangollen railway is cool beans [11:49] popey, gord: I looked into the code for lenses but it was over my head for now any chance you could throw together an xkcd lens :D [11:50] you know it's what lenses were designed for :D [11:50] i actually don't like xkcd :P so no :P [11:51] gord: okay how about a general COMICS LENS then :D [11:51] RSS lens would be good actually [11:52] heretic gord [11:53] i would like a 'comic' lens which has scopes for multiple online comics like xkcd, dilbert etc [11:53] search for "wedding" and it finds comics for that topic [11:53] or type "today" and get all of todays comics [11:53] I bet there'll be a meme and a lolcats one before too long [11:53] * brunogirin got irc running on phone so no excuse to miss meetings anymore [11:53] uau [11:53] yay even [11:54] popey: as I say I looked at it but the code was way over my head currently for a novice python guy [11:59] even the python lens examples? [12:00] gord: yeah I had a look I'm just not there yet [12:04] gord: don't forget unlike you, I used to drive a Lorry for a living, I've only been learning python for the last few month in any sort of serious fashion and only to make automation easier :) I really am a newbie at programming [12:05] gord: I'll get there just not this release :D [12:05] davmor2: really? [12:05] have you tried programming python *while* driving a lorry? [12:05] it's possible with care and some custom input devices [12:06] gordonjcp: you need all your wits about you with the prats on the road nowadays, especially in a gas tanker :D [12:09] that's true [12:09] text-to-speech works well even in a noisy cab [12:10] and then I used a gearstick-mounted PTT button from a radio to enter stuff in Morse Code [12:10] mostly for controlling an MP3 player, but one day I realised it would work with vim too... [12:10] programming python while you float weightlessly in space is possible, as proven by XKCD [12:18] Myrtti: and back to needing a comics lens again ;) [12:19] I think you all need help with your unity lenses, but I'm just using my computer here thank you very much [12:22] czajkowski: how was you journey up? [12:29] aptitude really needs a --dontbestupid switch [12:31] removed linux-image ? [12:31] haha, not quite [12:31] every time I use it it seems to want to uninstall half my system [12:32] and sometimes it proposes a solution that involves removing two packages, and then when it goes on to say what it's actually going to do, the REMOVED list is about nine screenfuls long... [12:33] Maybe it doesn't need those things [12:33] well, it doesn't *need* them, but I normally want them :) [12:33] Is it a list of suggested removals or a list of what will be removed? [12:34] anyone fancy confirming bug 932718 on 12.04 ? [12:34] Launchpad bug 932718 in unity "Keyboard overlay gets in the way when zooming" [Undecided,New] https://launchpad.net/bugs/932718 [12:34] bigcalm: nah, it's whats to be removed [12:34] stuff like libapache2-mod-php5 [12:34] my impression's probably not helped by only normally turning to aptitude when apt-get is a bit confused [12:35] I sometimes have to use aptitude to force a full upgrade (usually linux images being held back) [12:36] i never use aptitude [12:36] nah, nor i generally. Mostly because of it's fondness for breaking things [12:38] I only seem to need to use it on my parents' viglen mpc-l running 10.04 [12:51] NIS - you lose, I win ! [12:56] ypbind && yplol [12:57] I have defeated it [12:58] hail me, worship me [12:58] ypwin [13:11] Amazon UK - you fail [13:11] AWS has mostly put me off buying things from Amazon [13:11] AWS ? [13:11] I get a feeling of impending doom at abot the second 'a' of the URL [13:12] could you explain, I don't know what you are talking about [13:12] and it sounds curious [13:28] Amazon's cloudy service [13:28] herrro [13:29] er czajkowski go out and come back in, you don't greet us like that now do you [13:29] I do [13:29] herrro [13:29] pah! you're not the czajkowski we know an love, who are you? get her davmor2 [13:30] byebye [13:31] hugs czajkowski :) [13:33] BigRedS: so you don't use amazon shopping due to your concerns on their aws [13:33] (didn't realise you meant aws as their cloud service rather than a shopping function) [13:33] ikonia: haha, well, every time I type the URL I get flashbacks [13:34] Nah, really I've nothing much against it, but I definitely don't have any particular fondness for the company any more [13:34] a little dissapointing, thought you may have had a juicy story of whiy it was bad [13:34] Er, poor I/O, poor networking, poor scalability of their infrastrucutre, poor support [13:34] basically it's pretty bad at all of its selling points [13:34] ooh, I know the cloud service....you're bad mouthing that to the wrong person :) [13:35] ahhhh [13:35] oh, you wanted shop horror stories? Nah, none of those [13:35] I just thought you may have had a juicy story on why AWS->Shopping was linked [13:35] oh, no [13:35] well isn't the shopping hosted on aws? [13:35] I thought that's what aws grew out of. But, yeah, no bad times with the shoppping thing [13:37] it's hosted on "a" aws platfom, not the public one [13:38] oh, yeah [13:43] it's a bold statment to not host it on the public platform ;) [13:43] haha, yeah [13:45] though I like the allegation that aws basically grew out of amazon having built a pretty good foundaton for the shop and then renting it out [13:45] it might well be true, I'm just being a bit cheeky as Amazon have annoyed me today === sagaci_ is now known as sagaci [13:46] haha, I've left the amazony stuff now. Amazon don't get to annoy me daily any more [13:55] mrevell: czajkowski knocking your connection about? :P [13:58] no I'm being good ! [14:00] Where's the fun in being good? :) [14:07] bah I keep setting off hdd by hitting alt with irc [14:07] >:( [14:20] pancro: herrrrro fancy seeing you here [14:23] czajkowski: hola! [14:24] czajkowski: I tend to hang out here but I'm usually quiet [14:24] indeed [14:24] pancro: what did you have for todays lunch ? [14:26] czajkowski: oops, I haven't posted it yet, have I? kidney bean soup with onions and red wine :) a labour of love, I started soaking the beans last night! [14:27] I had yummy sandwiches and fresh fruit [14:27] so there :p [14:27] gord: popey in irssi when you alt 1 is hud jumping out for ye and being annoyingly in the way ? [14:28] don't use irssi [14:28] clearly :) [14:28] but yeah until a fix in compiz lands, hud will launch on other key combinations if you are too quick with them [14:28] i dont have hud installed it seems [14:28] ii unity 5.2.0-0ubuntu4 Interface designed for efficiency of space a [14:28] should that have it? [14:28] or do I need to install it separately? [14:28] that won't have it [14:28] it landed last night but I alt key for all channels on irc and alt 1 keeps launching hud [14:29] it didn't land ;) you are running the staging ppa or something [14:29] * czajkowski knew gord would do it [14:29] fecker;) [14:31] gord: get the vertex 3 ssd.. oh my golly [14:31] popey, how much did it set you back? [14:31] too much [14:31] think i might wait till a US trip and pick one up for half the price ;) [14:31] http://www.ebuyer.com/260707-ocz-240gb-vertex-3-ssd-2-5-sata-iii-6gb-s-read-550mb-s-write-520mb-s-vtx3-25sat3-240g [14:32] hah [14:32] geez, you don't need to read 550MB/s :P [14:32] AlanBell: what's that CRM software yoyu keep pimping to people? [14:36] BigRedS, http://www.vtiger.com/ i believe [14:44] AlanBell: do you have a cms recommendation ? [14:46] CMSMS or WordPress or Drupal [14:46] Depends what you need to do with it [14:46] anyone in here running 12.04 and use a terminal for irc? alt tabbing through my applications will not display my irc terminal? have to keep restarting it [14:47] diplo: aha, ta! [14:47] czajkowski: I do at home, if you nag me this eve I'll see if I can replicate. [14:47] I normally do it in guake [14:48] so if I alt tab through applications I can see it [14:48] but it wont launch [14:48] if I click on terminal on the side bar a new terminal pops out [14:48] if I minimise every single last applcation [14:48] there is my irc window [14:48] :/ [14:49] odd [14:49] I'll not go into my normal diatribe about working in unity right now, given your new employer :) [14:49] BigRedS: good of you [14:58] * czajkowski huggles davmor2 [14:58] czajkowski: you tired yet [14:58] nope [14:58] had tea and bickies so am sorted [14:59] mmmbickies [15:00] * hamitron is trying to cut down [15:03] christel, czajkowski: see when I read that I saw tea and brickies which had a whole different connotation [15:05] how do I find out what package provides a binary in /usr/bin? [15:06] apt-file ? [15:07] davmor2: haha [15:07] davmor2 reads "Dave Amour" to me [15:08] ikonia: hmm, apt-file -f search /usr/bin/java shows nothing [15:08] java is a symlink check out /etc/alternatives [15:09] nothing for /etc/alternatives/java either [15:10] Azelphur: if you do ls -la /usr/bin/java is it a file or a link ? [15:10] it's a link to /etc/alternatives/java [15:10] Azelphur: and then ls -la /etc/alternatives/java [15:11] ikonia: the magical trail ends at /usr/lib/jvm/java-6-openjdk/jre/bin/java which apt-file also doesn't know about [15:11] now that is surprising [15:11] I'm assuming /usr/lib/jvm/java-6-openjdk/jre/bin/java is actually a file [15:11] yep [15:12] but still, I'm mainly trying to get it to use java-7 so if anyone knows what the arg is for update-alternatives that'd be just as handy [15:12] Azelphur: sudo apt-file update first ? [15:12] AlanBell: then sudo apt-file search /usr/lib/jvm/java-6-openjdk/jre/bin/java [15:12] oops [15:12] sorry Azelphur ^ [15:12] or /usr/bin/java [15:13] yep that's got it, ty :P [15:13] there you go [15:13] there is no escape [15:13] and there we go, update alternatives fired off using java 7, much better \o/ [15:27] hello [15:31] hi === chalced is now known as chalcedony [15:40] hello zleap [15:55] hi [15:56] sorry was akf [15:56] afk [15:56] Quiet afternoon [15:56] yeah === myrtti is now known as Myrtti === liam_ is now known as Guest5582 [17:07] [Liam Proven] Still using Ubuntu 10.04? You may find this info about backports of newer kernels very useful. - http://liam-on-linux.livejournal.com/28391.html [17:16] o/ [17:17] choo choo [17:21] afternoon [17:22]