| derekv | anyone know if there is a general non-dbms-specific database channel? | 00:11 |
|---|---|---|
| widox | rick_h_: hm, possibly. when and where? | 00:16 |
| rick_h_ | widox: depends on what cmaloney says | 00:28 |
| rick_h_ | widox: going to try to visit a couple of coffee shops in the RO/birmingham area as potential places for CHC | 00:28 |
| rick_h_ | derekv: what's up? | 00:28 |
| derekv | here's what i got, i'd preffer a simple schema, eg, some users, some items owned by that user, some metadata for those items (eg, tags), and some log of events pertaining to those items | 00:28 |
| derekv | i'm pretty sql vs nosql agnostic on this one | 00:29 |
| derekv | but i'd like something which can scale out easily in the future | 00:29 |
| derekv | perfect transational consistancy is probably not super important | 00:29 |
| rick_h_ | I'm a believer in "Use pgsql until there's a good reason not to" person | 00:29 |
| widox | rick_h_: ah, I think Birmingham pretty much sucks for cofeeshops -- I know there is a Starbucks there ... | 00:30 |
| rick_h_ | so maybe my input isn't the best on this one | 00:30 |
| rick_h_ | widox: yea, cmaloney said he had a couple of 'forgotten' places to check out | 00:30 |
| rick_h_ | widox: so curious what these places are | 00:30 |
| derekv | the only downside of pgsql is that i couldn't just easily scale it out afaik | 00:30 |
| rick_h_ | pgsql scales a LOOOONG way out | 00:30 |
| widox | there is an independant one, but I think it closes real early like. never actually gone there | 00:31 |
| rick_h_ | it's a rare thing for most places to outgrow it imo | 00:31 |
| widox | heh, my company outgrew it | 00:31 |
| derekv | I thought it scaled a long way up? | 00:31 |
| widox | well, supposedly | 00:31 |
| derekv | but not out | 00:31 |
| rick_h_ | widox: oic, out vs up you mean? well you can out with r/o slaves and such | 00:31 |
| rick_h_ | it doesn't do master-mater yet, but I think that's rare | 00:32 |
| derekv | without changing the schema for shards or something... forgive me i'm a newbie regarding this stuff | 00:32 |
| rick_h_ | widox: really? I'd be curious to see what they were doing that they outgrew it | 00:32 |
| widox | rick_h_: raping databases | 00:32 |
| widox | rick_h_: don't worry, we solved it with ORACLE POWER | 00:32 |
| derekv | lol | 00:32 |
| rick_h_ | widox: lol, ok. | 00:32 |
| * widox weeps | 00:32 | |
| rick_h_ | wow, ok. | 00:33 |
| rick_h_ | derekv: scaling out in a read-only sense is easy. It's a matter of caching, replicating to servers in different DCs, etc. Writes is harder, but even writes can scale up a long way ime | 00:34 |
| derekv | so, i could make 4 tables in pgsql and i bet it'd handle a whole lot of bandwidth but, since I don't think I need ACID on the bulk of writes I wondered if there wasn't something i could choose early on to make scaling easier later if needed | 00:34 |
| rick_h_ | derekv: the issue is that you're talking metadata/logging which lends itself to random sql-ish queries | 00:35 |
| rick_h_ | and making arbitrary queries up on nosql situations is expensive and doesn't scale well | 00:35 |
| rick_h_ | that's why there's crap like map/reduce and hadoop to basically crawl data and build an index before you can run your query so it won't run forevery locking things all the while | 00:36 |
| widox | rsync'ing sqlite dbs? | 00:36 |
| rick_h_ | I mean, didn't mongo just recently get edit locks vs 'I'm looking at it' locks? | 00:36 |
| widox | hm, no bites on that eh | 00:36 |
| rick_h_ | widox: :P | 00:37 |
| derekv | I have no idea what mongo recently got. I couldn't tell you the difference between mongo , couch, cassandra, riak, ... | 00:37 |
| rick_h_ | derekv: I mean instagram was postgres on ubuntu on ec2 | 00:37 |
| rick_h_ | not sure if it still is in FB or what | 00:37 |
| rick_h_ | heh, ok. Mongodb, the old scool kid. Couch doesn't scale up well but the sync and html api is awesome for small apps, cassandra is the pretty lady these days. Go cluster go. http://docs.basho.com/riak/1.2.0/references/appendices/comparisons/Riak-Compared-to-Cassandra/ | 00:39 |
| rick_h_ | I hate mongodb with a burning bashing, I'd check out cassandra if I needed it and riak doesn't have the network of folks I know to trust it for anything long term atm | 00:40 |
| rick_h_ | but, all that is after I've grown the app to work for years on pgsql and know what my scaling needs are if/when it comes time to replace it | 00:40 |
| derekv | I'm certainly not trying to say I can't use pgsql and make it work, just curious if there was some shiny thing that could be better and I could tell everyone about how I was using it and feel really cool at social gatherings etc | 00:41 |
| rick_h_ | and with the json support, materialized views, server side functions for things that are required to be super fast, it's got the stuff you need. | 00:41 |
| rick_h_ | heh, well depends on your circle then :P | 00:41 |
| derekv | lol | 00:41 |
| rick_h_ | and I'll make fun of whatever circle it is because they won't know enough technically to justify their choice. :) | 00:42 |
| rick_h_ | https://www.braintreepayments.com/braintrust/scaling-postgresql-at-braintree-four-years-of-evolution looks like a great read | 00:43 |
| derekv | checking it ou | 00:47 |
| derekv | oh redis, that was one i was trying to think of the name of the other day | 00:47 |
| rick_h_ | redis == memcache | 00:47 |
| rick_h_ | redis is great for queues, caching, etc. | 00:47 |
| rick_h_ | I use redis for bookie queue processing, the r.bmark.us data, and it's great as long as the data can fit in memory | 00:48 |
| rick_h_ | derekv: like that article, pgsql is great for the primary store of the app and as you find things you bring in other stuff for specific uses | 00:48 |
| derekv | hmm | 00:49 |
| rick_h_ | sourceforge, at one time, used mongodb for a single materialized view engine for mysql | 00:49 |
| rick_h_ | so you visited a project page and the data came from mongodb, but the data lived in a series of tables back on mysql that any edit ot the project caused a rebuild of the mongodb document | 00:49 |
| derekv | thats the kind of complexity, i'm trying to avoid | 00:50 |
| rick_h_ | welcome to real scaling | 00:50 |
| rick_h_ | scaling isn't "Use X and all your problems are solved" | 00:50 |
| rick_h_ | even oracle :P | 00:50 |
| widox | oracle loves all | 00:51 |
| * rick_h_ is afraid of oracle | 00:51 | |
| derekv | so what about the log table, i'll write to it but not really edit it or care if nodes are not in perfect sync | 00:51 |
| rick_h_ | I remember using the free oracle sql thing for my college db class. That was 1GB on my 20GB hard drive | 00:51 |
| widox | especially living on a box with 1TB of RAM, yummy | 00:51 |
| rick_h_ | derekv: right but you'll want to query it. | 00:51 |
| rick_h_ | wow, 1TB of memory? postgresql didn't perform well on that same hardware? | 00:52 |
| rick_h_ | I mean the whole DB must live just in pure memroy | 00:52 |
| derekv | hmm it'd still probably want to sync writes to the spinny disks, if it had them | 00:53 |
| derekv | so write transations would still have to be limited by IO | 00:53 |
| derekv | regardless of dbms | 00:53 |
| widox | rick_h_: PS was switched out before we got to this level | 00:53 |
| widox | but, we have ~20TB of data in there | 00:53 |
| rick_h_ | widox: wow, ok. Remind me to get more info on what you guys are doing. I missed a few levels of stuff it looks like | 00:54 |
| widox | though, we kinda use the db as an application server -- in that a shit ton of biz logic resides in functions/procs/packages and such | 00:54 |
| widox | it actually is a giant butt hurt | 00:55 |
| rick_h_ | lol | 00:55 |
| rick_h_ | derekv: do what you want, but if you're asking for real advice/questions it's a can of 'right tool for the job' and unless you understand the tool it's hard to use it well | 00:55 |
| * rick_h_ puts away cranky old greybeard outfit and goes back to cleaning up the dishes. | 00:59 | |
| rick_h_ | widox: I'll ping when I hear from cmaloney | 00:59 |
| derekv | rick_h_: I appricate your advice | 00:59 |
| derekv | especially that I could use multiple technologies | 00:59 |
| rick_h_ | derekv: I'm part bitter. I've been handed work projects on mongodb that are on mongodb for NO good reason and the lack of good tools for migrations, etc are nuts | 01:00 |
| rick_h_ | derekv: mongodb, for instance, isn't really data safe until you've got a few machines at least. Most nosql require 3+N instances to be safe | 01:00 |
| derekv | i'm just starting out and, probably i'll prototype this thing in whatever looks easiest to prototype it in (likely something traditional like pgsql)... | 01:01 |
| rick_h_ | derekv: so take all that with a grain of salt, but a nugget of experience as well | 01:01 |
| widox | rick_h_: k | 01:01 |
| derekv | rick_h_: good to know... i'd like to scale from 1 to N not 3 to N | 01:01 |
| rick_h_ | unless you go oracle then you scale by how much income you can fork over :P | 01:01 |
| derekv | I guess I feel like, relational, ACID databases are awesome for that need them, but what if you don't need that level of transational consistancy? is there something simpler to use in that case? | 01:02 |
| derekv | yea i'd like to start with "free" since I don't even have an income model figured out yet | 01:03 |
| rick_h_ | derekv: the thing is that all the new stuff is young, missing tooling, still finding niches, etc. And they've all grown out of scaling past specific issues in ACID/sql | 01:03 |
| rick_h_ | so you have to figure out which one is the right one to solve the issue in RDS you're going to hit | 01:03 |
| rick_h_ | if you don't need all that I'd say couchdb is really cool, easy, fun to work with | 01:04 |
| rick_h_ | but it's not for large scale, so while it's great for prototyping and smallish apps it sounds like you're already going well past that | 01:04 |
| widox | yeah, I'd say. going relational is pretty damn fast, given the precieved overhead of consistancy and what-not -- they have good tools to let you know what can be tweaked | 01:05 |
| derekv | yep I need to read up. the users table seems like a classic rdms use case, the log and items table (basiclly a higher level log) are the interesting bits | 01:05 |
| derekv | i suppose, my app should shard really well | 01:08 |
| derekv | i mean, an item pertains to a user, log events pertain to items, metadata pertains to items | 01:08 |
| rick_h_ | don't forget you can shard by date and such as well | 01:09 |
| rick_h_ | at traffic engine we did request logging and aggregateing on pgsql partitioned tables by month which tends to line up with query usage | 01:09 |
| widox | heh, TE | 01:10 |
| widox | we partition as well. works pretty well | 01:10 |
| derekv | if only, i know how to calculate the cost of operating a machine to support X number of users so I could predict when i'd have to actually start thinking about scaling | 01:12 |
| derekv | if only, diatomic wasn't a closed source jvm frankenstien | 01:19 |
| derekv | datomic | 01:19 |
| derekv | asdfart | 01:19 |
| cmaloney | rick_h_: Sorry, was AFK for a bit | 01:20 |
| cmaloney | We're taking Pixel to the vet from 11:30am-12:30pm, so if you wanted to meet up we could do that | 01:21 |
| cmaloney | When are they heading to the zoo? | 01:21 |
| derekv | i need to just prototype it in the easiest thing because lets be honest i have no idea the requirements yet | 01:25 |
| derekv | or sit and think for a year like rich hickey | 01:26 |
| rick_h_ | cmaloney: think they're heading down around 10am ish | 01:27 |
| rick_h_ | cmaloney: so figured I'd go down, hit up a place, get some lunch, hit up a second place and head back in the afternoon | 01:27 |
| cmaloney | rick_h_: Ah OK. | 01:27 |
| rick_h_ | derekv: smart move :) | 01:27 |
| cmaloney | Hrum. | 01:27 |
| rick_h_ | derekv: there's a reason bookie dev/default install is sqlite | 01:27 |
| cmaloney | If you wanted to meet up for lunch we might be able to do something together if you're interested. | 01:28 |
| rick_h_ | cmaloney: coolio | 01:28 |
| rick_h_ | cmaloney: can you shoot me a couple of places you were thinking and why don't you ping when you're done with the vet and we can meet up for lunchables | 01:28 |
| cmaloney | But yeah, Pixel appt. is a fixed point in time | 01:28 |
| cmaloney | kk | 01:28 |
| rick_h_ | cmaloney: right, and my free time is going to be fixed. If only were were single without any responsibilities :) | 01:29 |
| cmaloney | yeah, no kidding | 01:32 |
| cmaloney | OK, list sent. | 01:32 |
| cmaloney | Will text when Pixel's appt is done | 01:32 |
| cmaloney | (fixed point in time is a Doctor Who reference. ;) ) | 01:32 |
| rick_h_ | cmaloney: ah, that buzzed over my head | 01:33 |
| cmaloney | Yeah, it's a non-changing event in time | 01:33 |
| cmaloney | which is shorthand for "we can't just wave this one away" | 01:33 |
| rick_h_ | cmaloney: coolio, got email. Thanks | 01:36 |
| cmaloney | np | 01:41 |
| derekv | man sound like trying to upgrade your car to a truck while drift racing | 01:43 |
| derekv | i know what i want... one database has everything of interest to real time operation of app, and a log which is stored in possibly a different place or technology which can be used for any offline anaylsis or migration i might want to do | 01:53 |
| rick_h_ | widox: so going to try to find "The Grumpy Barista" in royal oak. Google maps doesn't know about it, but a google search pops up an address in yelp/4square | 02:25 |
| rick_h_ | widox: will try to get down there 10-10:30am | 02:26 |
| widox | rick_h_: wah?! gmaps doesn't know about it | 02:46 |
| rick_h_ | widox: yea, confusing. | 02:53 |
| widox | according to street view, its no coffee shop | 02:54 |
| rick_h_ | so https://foursquare.com/v/the-grumpy-barista/4bf2c5576ba2b713cf078977 it looks like it's at the same address as the cocao cafe | 02:55 |
| rick_h_ | widox: hmm, so yea street view has it a small corner place there at the cafe | 02:59 |
| rick_h_ | widox: I don't think that'll work so plan B | 02:59 |
| derekv | https://www.fpcomplete.com | 03:00 |
| widox | ah, "3 check-ins in the last 60 days" could be promising then | 03:00 |
| derekv | your going to check it out? | 03:05 |
| rick_h_ | no, it looks tiny on streetview. Trying to find the next place on street view. Cafe Du Marquis | 03:06 |
| rick_h_ | and can't seem to find that one | 03:09 |
| derekv | welp | 03:11 |
| widox | rick_h_: haha yelp reviewer "I guess the hookah smell is a bit strong and the hippie vibe clashes a bit with the house music, but it's sweet and cozy" | 03:11 |
| widox | for cafe du marquis | 03:11 |
| derekv | i have no business model but here goes "launching a startup on haskell" | 03:11 |
| rick_h_ | widox: yea, stopping that one | 03:12 |
| derekv | at least i can get a cool talk out of it maybe | 03:12 |
| rick_h_ | derekv: hah, awesome | 03:12 |
| derekv | thouh i think someone already did that talk | 03:12 |
| derekv | there was this coffee place with hookas in RO that closed years ago | 03:13 |
| derekv | it was huge. they served cake in huge slices. coffee there was horrible. | 03:13 |
| widox | hm, maybe their hookah sales were better then coffee sales | 03:15 |
| widox | rick_h_: are you hating on | 03:15 |
| widox | Bean and Leaf | 03:15 |
| widox | ? | 03:15 |
| rick_h_ | widox: yea, it's on the list. Was just looking it up | 03:15 |
| widox | why newline, I dunno. paste fail | 03:15 |
| rick_h_ | it seems a bit out of the way, rochester? | 03:15 |
| widox | its got small tables -- but there are some bigger guys in the back | 03:15 |
| widox | no, downtown RO | 03:16 |
| widox | 106 S Main St | 03:16 |
| widox | http://www.mybeanandleaf.com/ | 03:16 |
| widox | hah, I guess they do have a Rochester location | 03:16 |
| * rick_h_ looks for another one | 03:16 | |
| rick_h_ | widox: ok, bean and leaf it is! | 03:17 |
| widox | rick_h_: ok, ~10:30 ? | 03:19 |
| rick_h_ | widox: yep | 03:20 |
| widox | rick_h_: k. I'll see ya there then | 03:20 |
| derekv | in case anyone was thinking otherwise, seems like actually there's no shortage of web frameworks in haskell | 03:27 |
| rick_h_ | derekv: heh yea I looked at a couple when I was trying to pick up haskell | 03:28 |
| derekv | I was thinking of making a strictly json API (possibly with accept html but like no time on aesthetics), and a seperate javascript client | 03:40 |
| derekv | so the nice looking dynamic site is hosted possibly eslewhere and is a different codebase/project | 03:41 |
| derekv | https://bmark.us/redirect/c1d42b8e57e967 | 03:54 |
| cmaloney | I think you're think of Brazil | 04:03 |
| cmaloney | That was the first coffee shop I'd ever been to | 04:04 |
| cmaloney | I think it's where Mr. B's is. | 04:04 |
| rick_h_ | derekv: that's kind of how bookie has been for a while. html is client side driven over the api | 04:04 |
| rick_h_ | I'll say that while it's been good experience and helped the api, if I had to do it again I'd split it up a bit and do more server side rendering. | 04:04 |
| derekv | finding some good urls | 04:21 |
| derekv | domain names rather | 04:21 |
| derekv | virthualenv [sic] | 05:15 |
| derekv | should do some sort of alias git --reset hard -> git stash save | 05:32 |
| cmaloney | morning | 14:05 |
| rick_h_ | morning | 14:17 |
| jrwren | morning | 15:20 |
| rick_h_ | woot, bean and leaf with the mifi power | 15:43 |
| rick_h_ | http://www.speedtest.net/result/3105158479.png | 15:43 |
| cmaloney | rick_h_: Wow. | 15:52 |
| cmaloney | That's some serious speed there. | 15:52 |
| cmaloney | I would love to have that upload speed. | 15:52 |
| cmaloney | OK, time to bundle a cat. | 15:53 |
| rick_h_ | cmaloney: yea, so an almond milk latte is kind of crap | 15:54 |
| jrwren | i'd show my speedtest results, but i'm on wifi, its the limiting factor for me :( | 16:05 |
| jrwren | http://www.speedtest.net/my-result/3105202152 -- wifi | 16:06 |
| jrwren | I shall run a wired one later today, it will probably show 60mbit down :) | 16:06 |
| jrwren | even still, that is some FAST LTE rick_h_ | 16:06 |
| rick_h_ | jrwren: yea, faster on mifi than my at home, 10x on the upload :/ | 16:07 |
| cmaloney | rick_h_: yeah, I don't like almond milk much | 16:24 |
| cmaloney | tastes strange | 16:24 |
| jrwren | what is the command to install build deps? | 16:28 |
| jrwren | dpkg-checkbuilddeps complains, i swear there was a command to install build deps | 16:28 |
| rick_h_ | greg-g: pushing up 0.1a5 with a fix for that hangup I think and icon status support. Make sure to go into preferences and do a "Sync" button press. | 23:17 |
| rick_h_ | hmmm, not showing it in https://addons.mozilla.org/en-US/firefox/addon/bookie/ yet :/ | 23:18 |
| rick_h_ | not sure if https://addons.mozilla.org/en-US/developers/addon/bookie/versions/1500376 works for you or not | 23:19 |
| rick_h_ | widox: ^^ | 23:20 |
| rick_h_ | there we go, just took a few min | 23:25 |
| greg-g | ah, as I was requesting my password (forgot that one) | 23:25 |
| greg-g | :) | 23:25 |
| rick_h_ | woot, working here | 23:25 |
| rick_h_ | so it'll show the + on pages you've bookmarked, green for success, red for error, etc | 23:26 |
| rick_h_ | looks like it'll take 2-4wks to get reviewed so not expecting auto updating to work until then | 23:26 |
| greg-g | rick_h_: yay! | 23:27 |
| greg-g | works | 23:27 |
| rick_h_ | woot | 23:31 |
| greg-g | thanks man! | 23:32 |
| rick_h_ | greg-g: np, thanks for testing. More work to go, but widox has been going crazy with it and it's approaching usable :) | 23:34 |
| greg-g | :) | 23:35 |
| greg-g | cmaloney: ever run into this issue on your squeezebox? it seems to have indexed it ok (shows up in interface), just wondering if I should be worried otherwise :) | 23:46 |
| greg-g | oh, link... | 23:46 |
| greg-g | cmaloney: http://thread.gmane.org/gmane.music.equipment.slimdevices.squeezecenter/41459 | 23:46 |
| * greg-g is testing before he buys | 23:49 | |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
