derekvanyone know if there is a general non-dbms-specific database channel?00:11
widoxrick_h_: hm, possibly. when and where?00:16
rick_h_widox: depends on what cmaloney says00:28
rick_h_widox: going to try to visit a couple of coffee shops in the RO/birmingham area as potential places for CHC00:28
rick_h_derekv: what's up?00:28
derekvhere's what i got, i'd preffer a simple schema, eg, some users, some items owned by that user, some metadata for those items (eg, tags), and some log of events pertaining to those items00:28
derekvi'm pretty sql vs nosql agnostic on this one00:29
derekvbut i'd like something which can scale out easily in the future00:29
derekvperfect transational consistancy is probably not super important00:29
rick_h_I'm a believer in "Use pgsql until there's a good reason not to" person00:29
widoxrick_h_: ah, I think Birmingham pretty much sucks for cofeeshops -- I know there is a Starbucks there ...00:30
rick_h_so maybe my input isn't the best on this one00:30
rick_h_widox: yea, cmaloney said he had a couple of 'forgotten' places to check out00:30
rick_h_widox: so curious what these places are00:30
derekvthe only downside of pgsql is that i couldn't just easily scale it out afaik00:30
rick_h_pgsql scales a LOOOONG way out00:30
widoxthere is an independant one, but I think it closes real early like. never actually gone there00:31
rick_h_it's a rare thing for most places to outgrow it imo00:31
widoxheh, my company outgrew it00:31
derekvI thought it scaled a long way up?00:31
widoxwell, supposedly00:31
derekvbut not out00:31
rick_h_widox: oic, out vs up you mean? well you can out with r/o slaves and such00:31
rick_h_it doesn't do master-mater yet, but I think that's rare00:32
derekvwithout changing the schema for shards or something... forgive me i'm a newbie regarding this stuff00:32
rick_h_widox: really? I'd be curious to see what they were doing that they outgrew it00:32
widoxrick_h_: raping databases00:32
widoxrick_h_: don't worry, we solved it with ORACLE POWER00:32
rick_h_widox: lol, ok.00:32
* widox weeps00:32
rick_h_wow, ok.00:33
rick_h_derekv: scaling out in a read-only sense is easy. It's a matter of caching, replicating to servers in different DCs, etc. Writes is harder, but even writes can scale up a long way ime00:34
derekvso, i could make 4 tables in pgsql and i bet it'd handle a whole lot of bandwidth but, since I don't think I need ACID on the bulk of writes I wondered if there wasn't something i could choose early on to make scaling easier later if needed00:34
rick_h_derekv: the issue is that you're talking metadata/logging which lends itself to random sql-ish queries00:35
rick_h_and making arbitrary queries up on nosql situations is expensive and doesn't scale well00:35
rick_h_that's why there's crap like map/reduce and hadoop to basically crawl data and build an index before you can run your query so it won't run forevery locking things all the while00:36
widoxrsync'ing sqlite dbs?00:36
rick_h_I mean, didn't mongo just recently get edit locks vs 'I'm looking at it' locks?00:36
widoxhm, no bites on that eh00:36
rick_h_widox: :P00:37
derekvI have no idea what mongo recently got.  I couldn't tell you the difference between mongo , couch, cassandra, riak, ...00:37
rick_h_derekv: I mean instagram was postgres on ubuntu on ec200:37
rick_h_not sure if it still is in FB or what00:37
rick_h_heh, ok. Mongodb, the old scool kid. Couch doesn't scale up well but the sync and html api is awesome for small apps, cassandra is the pretty lady these days. Go cluster go. http://docs.basho.com/riak/1.2.0/references/appendices/comparisons/Riak-Compared-to-Cassandra/00:39
rick_h_I hate mongodb with a burning bashing, I'd check out cassandra if I needed it and riak doesn't have the network of folks I know to trust it for anything long term atm00:40
rick_h_but, all that is after I've grown the app to work for years on pgsql and know what my scaling needs are if/when it comes time to replace it00:40
derekvI'm certainly not trying to say I can't use pgsql and make it work, just curious if there was some shiny thing that could be better and I could tell everyone about how I was using it and feel really cool at social gatherings etc00:41
rick_h_and with the json support, materialized views, server side functions for things that are required to be super fast, it's got  the stuff you need.00:41
rick_h_heh, well depends on your circle then :P00:41
rick_h_and I'll make fun of whatever circle it is because they won't know enough technically to justify their choice. :)00:42
rick_h_https://www.braintreepayments.com/braintrust/scaling-postgresql-at-braintree-four-years-of-evolution looks like a great read00:43
derekvchecking it ou00:47
derekvoh redis, that was one i was trying to think of the name of the other day00:47
rick_h_redis == memcache00:47
rick_h_redis is great for queues, caching, etc.00:47
rick_h_I use redis for bookie queue processing, the r.bmark.us data, and it's great as long as the data can fit in memory00:48
rick_h_derekv: like that article, pgsql is great for the primary store of the app and as you find things you bring in other stuff for specific uses00:48
rick_h_sourceforge, at one time, used mongodb for a single materialized view engine for mysql00:49
rick_h_so you visited a project page and the data came from mongodb, but the data lived in a series of tables back on mysql that any edit ot the project caused a rebuild of the mongodb document00:49
derekvthats the kind of complexity, i'm trying to avoid00:50
rick_h_welcome to real scaling00:50
rick_h_scaling isn't "Use X and all your problems are solved"00:50
rick_h_even oracle :P00:50
widoxoracle loves all00:51
* rick_h_ is afraid of oracle00:51
derekvso what about the log table, i'll write to it but not really edit it or care if nodes are not in perfect sync00:51
rick_h_I remember using the free oracle sql thing for my college db class. That was 1GB on my 20GB hard drive00:51
widoxespecially living on a box with 1TB of RAM, yummy00:51
rick_h_derekv: right but you'll want to query it.00:51
rick_h_wow, 1TB of memory? postgresql didn't perform well on that same hardware?00:52
rick_h_I mean the whole DB must live just in pure memroy00:52
derekvhmm it'd still probably want to sync writes to the spinny disks, if it had them00:53
derekvso write transations would still have to be limited by IO00:53
derekvregardless of dbms00:53
widoxrick_h_: PS was switched out before we got to this level00:53
widoxbut, we have ~20TB of data in there00:53
rick_h_widox: wow, ok. Remind me to get more info on what you guys are doing. I missed a few levels of stuff it looks like00:54
widoxthough, we kinda use the db as an application server -- in that a shit ton of biz logic resides in functions/procs/packages and such00:54
widoxit actually is a giant butt hurt00:55
rick_h_derekv: do what you want, but if you're asking for real advice/questions it's a can of 'right tool for the job' and unless you understand the tool it's hard to use it well00:55
* rick_h_ puts away cranky old greybeard outfit and goes back to cleaning up the dishes. 00:59
rick_h_widox: I'll ping when I hear from cmaloney00:59
derekvrick_h_: I appricate your advice00:59
derekvespecially that I could use multiple technologies00:59
rick_h_derekv: I'm part bitter. I've been handed work projects on mongodb that are on mongodb for NO good reason and the lack of good tools for migrations, etc are nuts01:00
rick_h_derekv: mongodb, for instance, isn't really data safe until you've got a few machines at least. Most nosql require 3+N instances to be safe01:00
derekvi'm just starting out and, probably i'll prototype this thing in whatever looks easiest to prototype it in (likely something traditional like pgsql)...01:01
rick_h_derekv: so take all that with a grain of salt, but a nugget of experience as well01:01
widoxrick_h_: k01:01
derekvrick_h_: good to know... i'd like to scale from 1 to N  not 3 to N01:01
rick_h_unless you go oracle then you scale by how much income you can fork over :P01:01
derekvI guess I feel like, relational, ACID databases are awesome for that need them, but what if you don't need that level of transational consistancy? is there something simpler to use in that case?01:02
derekvyea i'd like to start with "free" since I don't even have an income model figured out yet01:03
rick_h_derekv: the thing is that all the new stuff is young, missing tooling, still finding niches, etc. And they've all grown out of scaling past specific issues in ACID/sql01:03
rick_h_so you have to figure out which one is the right one to solve the issue in RDS you're going to hit01:03
rick_h_if you don't need all that I'd say couchdb is really cool, easy, fun to work with01:04
rick_h_but it's not for large scale, so while it's great for prototyping and smallish apps it sounds like you're already going well past that01:04
widoxyeah, I'd say. going relational is pretty damn fast, given the precieved overhead of consistancy and what-not -- they have good tools to let you know what can be tweaked01:05
derekvyep I need to read up.  the users table seems like a classic rdms use case, the log and items table (basiclly a higher level log) are the interesting bits01:05
derekvi suppose, my app should shard really well01:08
derekvi mean, an item pertains to a user, log events pertain to items, metadata pertains to items01:08
rick_h_don't forget you can shard by date and such as well01:09
rick_h_at traffic engine we did request logging and aggregateing on pgsql partitioned tables by month which tends to line up with query usage01:09
widoxheh, TE01:10
widoxwe partition as well. works pretty well01:10
derekvif only, i know how to calculate the cost of operating a machine to support X number of users so I could predict when i'd have to actually start thinking about scaling01:12
derekvif only, diatomic wasn't a closed source jvm frankenstien01:19
cmaloneyrick_h_: Sorry, was AFK for a bit01:20
cmaloneyWe're taking Pixel to the vet from 11:30am-12:30pm, so if you wanted to meet up we could do that01:21
cmaloneyWhen are they heading to the zoo?01:21
derekvi need to just prototype it in the easiest thing because lets be honest i have no idea the requirements yet01:25
derekvor sit and think for a year like rich hickey01:26
rick_h_cmaloney: think they're heading down around 10am ish01:27
rick_h_cmaloney: so figured I'd go down, hit up a place, get some lunch, hit up a second place and head back in the afternoon01:27
cmaloneyrick_h_: Ah OK.01:27
rick_h_derekv: smart move :)01:27
rick_h_derekv: there's a reason bookie dev/default install is sqlite01:27
cmaloneyIf you wanted to meet up for lunch we might be able to do something together if you're interested.01:28
rick_h_cmaloney: coolio01:28
rick_h_cmaloney: can you shoot me a couple of places you were thinking and why don't you ping when you're done with the vet and we can meet up for lunchables01:28
cmaloneyBut yeah, Pixel appt. is a fixed point in time01:28
rick_h_cmaloney: right, and my free time is going to be fixed. If only were were single without any responsibilities :)01:29
cmaloneyyeah, no kidding01:32
cmaloneyOK, list sent.01:32
cmaloneyWill text when Pixel's appt is done01:32
cmaloney(fixed point in time is a Doctor Who reference. ;) )01:32
rick_h_cmaloney: ah, that buzzed over my head01:33
cmaloneyYeah, it's a non-changing event in time01:33
cmaloneywhich is shorthand for "we can't just wave this one away"01:33
rick_h_cmaloney: coolio, got email. Thanks01:36
derekvman sound like trying to upgrade your car to a truck while drift racing01:43
derekvi know what i want... one database has everything of interest to real time operation of app, and a log which is stored in possibly a different place or technology which can be used for any offline anaylsis or migration i might want to do01:53
rick_h_widox: so going to try to find "The Grumpy Barista" in royal oak. Google maps doesn't know about it, but a google search pops up an address in yelp/4square02:25
rick_h_widox: will try to get down there 10-10:30am02:26
widoxrick_h_: wah?! gmaps doesn't know about it02:46
rick_h_widox: yea, confusing.02:53
widoxaccording to street view, its no coffee shop02:54
rick_h_so https://foursquare.com/v/the-grumpy-barista/4bf2c5576ba2b713cf078977 it looks like it's at the same address as the cocao cafe02:55
rick_h_widox: hmm, so yea street view has it a small corner place there at the cafe02:59
rick_h_widox: I don't think that'll work so plan B02:59
widoxah, "3 check-ins in the last 60 days" could be promising then03:00
derekvyour going to check it out?03:05
rick_h_no, it looks tiny on streetview. Trying to find the next place on street view. Cafe Du Marquis03:06
rick_h_and can't seem to find that one03:09
widoxrick_h_:  haha yelp reviewer "I guess the hookah smell is a bit strong and the hippie vibe clashes a bit with the house music, but it's sweet and cozy"03:11
widoxfor cafe du marquis03:11
derekvi have no business model but here goes "launching a startup on haskell"03:11
rick_h_widox: yea, stopping that one03:12
derekvat least i can get a cool talk out of it maybe03:12
rick_h_derekv: hah, awesome03:12
derekvthouh i think someone already did that talk03:12
derekvthere was this coffee place with hookas in RO that closed years ago03:13
derekvit was huge.  they served cake in huge slices.  coffee there was horrible.03:13
widoxhm, maybe their hookah sales were better then coffee sales03:15
widoxrick_h_: are you hating on03:15
widoxBean and Leaf03:15
rick_h_widox: yea, it's on the list. Was just looking it up03:15
widoxwhy newline, I dunno. paste fail03:15
rick_h_it seems a bit out of the way, rochester?03:15
widoxits got small tables -- but there are some bigger guys in the back03:15
widoxno, downtown RO03:16
widox106 S Main St03:16
widoxhah, I guess they do have a Rochester location03:16
* rick_h_ looks for another one 03:16
rick_h_widox: ok, bean and leaf it is!03:17
widoxrick_h_: ok, ~10:30 ?03:19
rick_h_widox: yep03:20
widoxrick_h_: k. I'll see ya there then03:20
derekvin case anyone was thinking otherwise, seems like actually there's no shortage of web frameworks in haskell03:27
rick_h_derekv: heh yea I looked at a couple when I was trying to pick up haskell03:28
derekvI was thinking of making a strictly json API (possibly with accept html but like no time on aesthetics), and a seperate javascript client03:40
derekvso the nice looking dynamic site is hosted possibly eslewhere and is a different codebase/project03:41
cmaloneyI think you're think of Brazil04:03
cmaloneyThat was the first coffee shop I'd ever been to04:04
cmaloneyI think it's where Mr. B's is.04:04
rick_h_derekv: that's kind of how bookie has been for a while. html is client side driven over the api04:04
rick_h_I'll say that while it's been good experience and helped the api, if I had to do it again I'd split it up a bit and do more server side rendering.04:04
derekvfinding some good urls04:21
derekvdomain names rather04:21
derekvvirthualenv [sic]05:15
derekvshould do some sort of alias git --reset hard -> git stash save05:32
rick_h_woot, bean and leaf with the mifi power15:43
cmaloneyrick_h_: Wow.15:52
cmaloneyThat's some serious speed there.15:52
cmaloneyI would love to have that upload speed.15:52
cmaloneyOK, time to bundle a cat.15:53
rick_h_cmaloney: yea, so an almond milk latte is kind of crap15:54
jrwreni'd show my speedtest results, but i'm on wifi, its the limiting factor for me :(16:05
jrwrenhttp://www.speedtest.net/my-result/3105202152  -- wifi16:06
jrwrenI shall run a wired one later today, it will probably show 60mbit down :)16:06
jrwreneven still, that is some FAST LTE rick_h_16:06
rick_h_jrwren: yea, faster on mifi than my at home, 10x on the upload :/16:07
cmaloneyrick_h_: yeah, I don't like almond milk much16:24
cmaloneytastes strange16:24
jrwrenwhat is the command to install build deps?16:28
jrwrendpkg-checkbuilddeps complains, i swear there was a command to install build deps16:28
rick_h_greg-g: pushing up 0.1a5 with a fix for that hangup I think and icon status support. Make sure to go into preferences and do a "Sync" button press.23:17
rick_h_hmmm, not showing it in https://addons.mozilla.org/en-US/firefox/addon/bookie/ yet :/23:18
rick_h_not sure if https://addons.mozilla.org/en-US/developers/addon/bookie/versions/1500376 works for you or not23:19
rick_h_widox: ^^23:20
rick_h_there we go, just took a few min23:25
greg-gah, as I was requesting my password (forgot that one)23:25
rick_h_woot, working here23:25
rick_h_so it'll show the + on pages you've bookmarked, green for success, red for error, etc23:26
rick_h_looks like it'll take 2-4wks to get reviewed so not expecting auto updating to work until then23:26
greg-grick_h_: yay!23:27
greg-gthanks man!23:32
rick_h_greg-g: np, thanks for testing. More work to go, but widox has been going crazy with it and it's approaching usable :)23:34
greg-gcmaloney: ever run into this issue on your squeezebox? it seems to have indexed it ok (shows up in interface), just wondering if I should be worried otherwise :)23:46
greg-goh, link...23:46
greg-gcmaloney: http://thread.gmane.org/gmane.music.equipment.slimdevices.squeezecenter/4145923:46
* greg-g is testing before he buys23:49

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!