| rick_h_ | jcastro: how do you do a "Show HN"? I only see ask/jobs/submit | 04:20 |
|---|---|---|
| Blazeix | rick_h_: pretty sure you prefix the title with "Show HN" | 04:24 |
| Blazeix | http://www.hnsearch.com/search#request/all&q=SHow+HN&sortby=create_ts+desc | 04:24 |
| rick_h_ | ok yea, I found a big thread discussing making it a tab | 04:27 |
| rick_h_ | but looks like it's shot down | 04:27 |
| rick_h_ | well, don't want to submit and go to bed, so maybe for tomorrow | 04:27 |
| snap-l | Good morning | 13:36 |
| rick_h_ | morning | 13:54 |
| rick_h_ | 499 | 14:29 |
| rick_h_ | views on your busiest day, August 18, 2012 | 14:29 |
| rick_h_ | lol, damn one short | 14:29 |
| snap-l | rick_h_: Did the API url change or something? | 14:33 |
| snap-l | I can't seem to bookmark | 14:34 |
| snap-l | via the extension | 14:34 |
| snap-l | I'm getting a 502 error when bookmarking from the site | 14:35 |
| rick_h_ | snap-l: no, 502 from the site? using 'add bookmark'? | 14:36 |
| snap-l | Yes | 14:36 |
| rick_h_ | k, sec will look | 14:36 |
| snap-l | bbl | 14:41 |
| brousch | I found my old 1 gallon aquarium in the basement and got some fish for it | 14:44 |
| rick_h_ | wtf it works locally | 14:46 |
| brousch | Hm, yeah 502 for me too | 14:47 |
| rick_h_ | ok yea saw the error that time | 14:48 |
| rick_h_ | whoosh file lock error because I've got import going I bet | 14:48 |
| rick_h_ | there's a huge import going on that's killing me | 14:49 |
| brousch | so add doesn't go through celery? | 14:56 |
| rick_h_ | the import does | 14:56 |
| rick_h_ | but it's throwing new bookmarks at whoosh like no tomorrow | 14:56 |
| rick_h_ | and the web site gets locked out | 14:56 |
| brousch | right, so the normal add doesn't get put in the same way? | 14:57 |
| rick_h_ | I just killed celery and got it to work | 14:57 |
| rick_h_ | still slow | 14:57 |
| brousch | It seems like all adds would go through the same process to be added | 14:57 |
| rick_h_ | no, normal add happens in request. Because just saving a bookmark hits whoosh to index the page title, description, and extended description | 14:57 |
| rick_h_ | they do, sqlalchemy events, but two people are talking now | 14:57 |
| rick_h_ | the web site and the celery importer | 14:57 |
| rick_h_ | so they're fighting over who has lock access to the whoosh index | 14:57 |
| rick_h_ | and this is when you get fun with file based data stores | 14:58 |
| brousch | Yeah, I'm probably just showing my ignorance | 14:58 |
| rick_h_ | so when the website is adding all is well. It's sharing a single lock for all of it's threads | 14:59 |
| brousch | I figured the importer would throw each add into celery, then that add would be processed by whoosh as it has resources | 14:59 |
| rick_h_ | whoosh only lets one person hold the lock at a time | 14:59 |
| rick_h_ | so the importer starts up, and it's taking over 30min (keep having to kill/restart his import) | 14:59 |
| rick_h_ | and it grabs the lock | 14:59 |
| rick_h_ | and the website says "hey, I need that..." | 14:59 |
| brousch | So the web site add would throw it's stuff into celery and it would be processed in the queue with the import | 14:59 |
| rick_h_ | but the web site doesn't toss a single new bookmark to celery, you want to have that stored now | 15:00 |
| brousch | Sounds like maybe it should | 15:00 |
| rick_h_ | the proper thing to do would be to catch the whoosh index lock exception, and file a background celery process for later | 15:00 |
| rick_h_ | or move fulltext off to something that works with multiple users/locks | 15:00 |
| brousch | That sounds like extra code. Why not just always do it in celery? | 15:01 |
| rick_h_ | most imports take 2min | 15:01 |
| jrwren | i was afk until now. congrats rick_h_ | 15:01 |
| rick_h_ | and most people aren't saving new bookmarks every 2min so it's not been an issue | 15:01 |
| jrwren | excellent work. | 15:01 |
| brousch | But as you scale | 15:01 |
| rick_h_ | jrwren: heh, well it's blowing up in places :) | 15:01 |
| rick_h_ | brousch: yea, as I scalethe right thing to do is move fulltext off of whoosh | 15:01 |
| brousch | Onto what? | 15:02 |
| rick_h_ | every db actoin can't have to go through celery | 15:02 |
| rick_h_ | brousch: elasticsearch, lucense/blah | 15:02 |
| brousch | Why not? I thought that was the new hotness | 15:02 |
| rick_h_ | heh | 15:02 |
| brousch | async all the things! | 15:06 |
| rick_h_ | dammit, something is up. I don't get what. | 15:08 |
| jrwren | bmark.us feels faster. | 15:09 |
| jcastro | rick_h_: where you hosting this btw? | 15:10 |
| rick_h_ | jcastro: pair os ec2 instances | 15:11 |
| rick_h_ | /os/of | 15:11 |
| rick_h_ | I think the add new bookmark issue isn't all the whoosh as well. I shut down everything, restarted services, and still getting an issue. | 15:16 |
| rick_h_ | heh, guess I need an account delete button | 15:21 |
| rick_h_ | someone is going through and deleting all their imported bookmarks one by one | 15:21 |
| rick_h_ | yea, whoosh index is melted | 15:28 |
| rick_h_ | there goes that | 15:28 |
| rick_h_ | but adding should work again | 15:28 |
| rick_h_ | snap-l: brousch ^ | 15:32 |
| snap-l | rick_h_: Will check. Thanks! | 17:25 |
| snap-l | "Xubuntu is not a refugee camp" | 17:36 |
| snap-l | Oh, this is going to be a good blog post. | 17:36 |
| rick_h_ | .lol | 17:37 |
| snap-l | http://open.knome.fi/2012/08/19/xubuntu-is-not-a-refugee-camp/ | 17:37 |
| snap-l | Evening (at least, in 35 minutes) | 21:26 |
| rick_h_ | party | 21:53 |
| brousch | Ridiculous http://twitter.com/N165Mars | 23:32 |
| Blazeix | brousch++ | 23:42 |
| brousch | It's kind of cool to send Shawn Powers a personal invite to BarCampGR | 23:45 |
| rick_h_ | lol | 23:58 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!