[00:21] s3 [00:21] yea [00:22] hazmat: s3, gonna switch to akamia ( rackspace cloudfiles ) real soon ( probably have the update pushed by uds ) was just talking to Joey and marcoceppi earlier about it [00:22] real cdn, faster, same price [00:23] lots of JS optim comming too [00:23] now that the server side is manageable [00:24] imbrandon, cool, although frankly just having more points of presence then s3 eu would be a big win (ie any cdn would be a big improv) [00:24] hazmat: how many pops does s3 eu have ? [00:24] lifeless, doesn't really matter when i'm not in the eu ;-) [00:25] heh [00:25] lifeless: not sure but s3 is only served from the one its stored in [00:25] unless you use cloudfiles service [00:25] or cloudfront [00:25] then its more like a cdn , origin pull that uses s3 as origin [00:25] err yea cloudfront [00:26] bah [00:26] names all so close :) [00:26] imbrandon, mix and match clouds is a confusing biz ;-) [00:26] heh yea [00:26] i think thats how it may be with omg tho untill we grow [00:27] OSAPI supprt [00:27] rack for cdn aws for deploy + s3 db dump backups [00:27] leaste thats the tentive plan i came up with today [00:27] gotta sell it to the others [00:28] but it seems like the right thing for themoment [00:28] then when juju can move so can we, ver very easy [00:30] but yea we've squeesed pretty much what we're gona get out of pure httpd enhancements out of it, maybe a bit more, but the rest will mostly be code and front end, right now 80% of the page req 0 to done is front end rendering anyhow [00:30] thus we need to work on the JS badly [00:32] and thats another place where newrelic shines , man i love that company [00:32] i'm almost as much a newrelic fanboy as i am apple and github HAHHAHA [00:33] monitor.us sup has gotten much better and closer to the newrelic level of service herer reciently too but not had a chance to try it myself yet [00:34] "Unlimited Application Insight at Light Speed!" wtf [00:34] oh and if you ever consider exceptionhub for a project, dont, not even worth the $9 a quarter it costs [00:35] imbrandon: have you used tracelytics.com ? [00:35] its so bad i actually started a self hosted opensource version and have it 80% complete, just to spite them [00:35] lifeless: nah, not heard of them [00:35] getexceptional and exceptinohub [00:36] imbrandon: js exceptions - I need that glued into LP OOPS someday [00:36] are the only two that i found quickly that had php api's and js apis [00:36] Just haven't gotten around to it [00:36] already have analysis console of course ;) [00:36] lifeless: i can get ya the glue, its very easy [00:37] i am not great on the py part for lp you smooth the edge on that and i'll snag my bits out and givem to ya, its very simple idea these places do [00:37] once its caputured its just jsond and compressed then posted to server [00:37] server does all the work [00:38] so, basically I need a dict sent to a web service [00:38] yup [00:38] http://bazaar.launchpad.net/~canonical-launchpad-branches/python-oops/trunk/view/head:/oops/config.py#L54 [00:38] describes the keys we use today [00:38] kk [00:38] they are all optional but the more the better [00:38] oh thats not many [00:39] i was expecting 54 [00:39] lol [00:39] :) [00:39] we use bson for the rabbitmq glue / disk store - (but json would be fine too, obviously can't do binary in that case) [00:39] yea, well there might be a lib but it would add weight to the thing [00:39] not sure if the diff is worth it [00:39] I'd use json to start with [00:40] lingua franca of the open web [00:40] wouldent be hard to try / untry once in plkace anyhow [00:40] right [00:40] ok, time to put on the hacking music and put my head down for a bit. [00:40] yea every place i seen just gzip encodes a json req, no matter how the lang glue captures the exceptions [00:41] then posts it, server gunzip's ties to an accounts sent with the api key [00:41] and stuffed into a db for analytics later [00:42] right [00:43] so id is a guid or other random thing [00:43] * imbrandon really needs to bytes the bullet and get comfy with python, i mean i know the basics , honestly more than some novices i know but still .... mid-newyears resolution ? heh [00:43] reporter should be configurable, e.g. LP would use 'Launchpad-js-prod' [00:43] right [00:44] topic would probably be also configurable (e.g. LP might supply the topic in the page body, for correlation purposes) [00:44] ditto branch_nick and revno, LP would inject that [00:44] https://github.com/bholtsclaw/exceptional-php/blob/master/exceptional/remote.php [00:44] is the send bits i would convert to yer need [00:45] to give ya an idea [00:45] ok, well I have one other thing I *have* to get done today [00:45] kk [00:45] I might whip up the web service end of this after that, can't promise it tho. we'll see. [00:45] yea i'll much with this inbetween my charms , i never speed too much at one moment , task flip [00:46] sure thing [00:46] spend* [00:46] it is very very nice once its even half way in place tho [00:47] its like, man why did i never do this 5 years ago [00:47] yeah [00:47] the zope and django versions are invaluable, key tools in LP problem diagnosis [00:47] hrm [00:48] zope /me runnnnnnnnnns [00:48] lol jk [00:48] https://errors.ubuntu.com/ uses a common substrate [00:48] isnt LP based on zope heavily, or was to begin with ? [00:48] yes, LP was one of the first zope3 things built [00:48] ohhhh nice [00:48] ( the ui ) [00:49] I wouldn't advise folk to get into zope3 tho [00:49] it meets a very niche set of requirements [00:49] heh nah, that enews i just converted , well reimplmented was zope something, bnever looked at the code only the business req , into drupal [00:49] fun fun [00:49] was solid and working for 10+ years tho [00:50] prior to them contracting me to do that [00:50] prob zope2 then [00:50] different beast [00:50] if you can tell from output its http://enews.penton.com [00:50] many things inherited from, but many different and no necessarily better :P) [00:50] mine is http://enewspro.penton.com ;) [00:51] Server: Zope/(Zope 2.7.4-0, python 2.3.5, linux2) ZServer/1.1 [00:51] :) [00:51] the pro one wants a pasword :> [00:51] yea i never even had to open the code on that one, year long project with 6 guys under me full time [00:52] still never touched it [00:52] only the brd [00:52] i was happy [00:52] oh yea , prod i dont have a pass for [00:52] you could hit http://dev.enews-pro.gotpantheon.com [00:52] and use "admin" and "admin1" [00:52] :) super secure [00:52] dev server [00:53] Content Encoding Error [00:53] [00:53] [00:53] [00:53] [00:53] [00:53] [00:54] [00:54] The page you are trying to view cannot be shown because it uses an invalid or unsupported form of compression. [00:54] and no i dident pick the pink and blue, it was handed to me, i only was tech engagement lead , e.g. head coder with ass on line [00:54] lololol [00:54] NIce [00:54] what browser ? [00:54] ff [00:54] just a norm one ? [00:54] Accept-Encoding:gzip, deflate [00:54] hrm [00:54] yeah, normal [00:54] wow [00:54] strange [00:54] Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 [00:55] never had that from anyone else but the dev is on totaly diff hardware / software than stage and prod too [00:55] Content-Encoding:gzipContent-Length:2522 [00:55] dev == some funky outsourced nginx from india with mongodb cache [00:55] heh [00:55] ctrl-shift-refresh fixed it [00:55] stage and prod was in house over there [00:56] ahh prob tried to send gz headers without telling ya [00:56] first time [00:56] or somethn [00:56] anyhow its a pretty simple app, build out newsletters that go out daily for 100+ subcompanies [00:57] something like 1.3 mil email a month from that thing [00:57] and its really justa wysiwyg editor for a farm of F5's blasting constantly [00:57] well only prod, dev wont actually send anything [00:58] legs cut out from it on back end [00:58] it thinks it does tho :) [00:58] but yea all UI choices on that were not me at all [00:58] i mean i put theme there but was handed them [00:58] and not input on changing them [00:58] none [00:58] grrr [00:59] but yea pretty much all functional code there i either wrote or designed and hand picked the dev to complete it [00:59] then did the review for merge [00:59] along with anothre trusted senior dev [00:59] to check me own arse [01:00] considering how much that thing sends tho, and then archives forever ( in an archive db but same ui) [01:00] is impressive to me even having done it [01:01] the real ones probably been in production full blast since mid january and i bet arround 5 to 8 mill nodes already are in the archives [01:02] but i finished the proj and opted to find another late feb [01:02] so not 100% on that one [01:02] dev is on my farmed out server btw, your not hackin the gibson :) [01:03] i need to take the code down really [01:05] yea so if you ever get an "Industry" news letter of some type , like trucker magazine or Win SQLPro [01:06] look at the bottom for "Penton Media" its those freaks [01:07] they own like every industry mag possible and are borderline spammers, legit but only barely thus me choosing to move on once the projcet was in a state i did not mind handing it off [01:10] mmmm i do like that ui alot ( not meaning the standard ubuntu branding parts, altho those are bad ) but what chart widge/lib/whatever is thaty [01:10] is it open ? or avail at least ? [01:11] would be kinda nice to grab some of the new relic info and snaitize it then use that to represent it via their api [01:11] instead of the embed charts the giv [01:11] was gonna use google charts but that looks much slicker and dont make my cpu fan kick on [01:11] lol [01:12] imbrandon: yes, all open [01:12] nice i'll have to add that to my note pad to check out later then [01:13] * imbrandon keeps a oldschool pen and paper otherise shiney would overule work [01:13] for todo's later in the week [01:13] SpamapS: land your patch [01:13] SpamapS: for txaws [01:14] wow wait a min, LP is fully open now ? [01:14] wow i really need to get out of my little hole in the sand and look around a bit [01:15] LP has been fully open for a few years [01:15] SpamapS: is awsome even deployable, i cant find any actual code or instuctions to piece existing code togather if i wanted to use it [01:16] lifeless: yea, just now noticing, how dumb of me [01:16] heh [01:16] * imbrandon rembers when some ppl would sign nda's to help with it prior [01:16] infact william grant, isnt he on the LP team now [01:17] bah, JS code [01:17] yes [01:17] he is [01:18] cool yea he was an execlent contrib when we used to be on the same bits of ubuntu working [01:18] glad for him [01:18] very young too iirc ( well then , likely a full man now ) [01:19] lifeless: ohh this is much easier than i thought it would be [01:20] i _might_ have this ready in just a few [01:20] def not long tho [01:32] $json_ret = $u->url_get_contents($apiurl); [01:32] bah [02:08] lifeless: i got to run for a bit, promised someone id help them IRL for a few , ive got a good chunk of this ripped out , its pretty isolated to begin with as i builts it from 2 or 3 others ideas i had evaluated, anyhow i'll jot ya an email if your not around when i get back [02:08] probably only an hour or so === Furao_ is now known as Furao === hspencer is now known as hspencer[afk] [05:10] lifeless: I thought I did [05:15] lifeless: I don't see any pending reviews for txaws anyway [05:17] SpamapS: can you have a look at https://code.launchpad.net/~bkerensa/charms/precise/locker/trunk [05:17] I have a bug open for it [05:22] bkerensa: you might not have noticed this, but precise's nodejs is only one patch level behind upstream [05:22] SpamapS: oh [05:22] bkerensa: I bet you can use the stock nodejs/npm from precise [05:23] SpamapS: Yeah I will go ahead and fix that [05:23] bkerensa: also you open port 8042 twice. The one after 'start' is far more appropriate :) [05:24] :D [05:25] bkerensa: other than that it looks pretty clean. :) You may want to explain in the README that scaling out is not possible. [05:26] anyway, about to pass out.. so tired [05:35] SpamapS: the indicators one ? [05:35] SpamapS: if so, cool === lynxman- is now known as lynxman === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away === daker__ is now known as daker [13:36] lifeless: yeah, in trunk aws-status is an indicator now :) [14:29] https://bugs.launchpad.net/charms/+bug/991980 [14:29] doh [14:29] <_mup_> Bug #991980: Oneiric official branches are all locked < https://launchpad.net/bugs/991980 > [14:50] SpamapS: did you guys, for lack of better word, push the precise charms to the correct repo? [14:51] with m_3 [14:59] senior7515: yes, the lp:charms/hadoop is in the right place... and no, we haven't deprecated lp:charms/hadoop-{master,slave,mapreduce} yet [15:00] senior7515: those were the ones you were interested in right? I recommend using lp:charms/hadoop on precise [15:01] senior7515: that combo passed basic tests this morning [15:01] m_3: thanks a lot. [15:01] yeah [15:01] about to do some testing on that. [15:01] thanks [15:03] senior7515: cool... lemme know how it goes. love to know more about your project sometime too if it's not proprietary [15:04] m_3: basically just trying to compute averages of a large mongo collection on the fly.. [15:04] averages per day, hour, month, year, graphs, etc. [15:05] senior7515: oh cool... are you planning to fork the lp:charms/hadoop to work with mongo? it's pure hdfs atm [15:06] well, not sure how this works yet.. :) But basically there is this mongo-hadoop lib [15:07] you drop it in the jar directory of hadoop [15:07] senior7515: shouldn't be too much work... just thinking offhand... mongo's pretty easy to work with in general, and the mongo charm manages replsets pretty well [15:07] and it gives you a streaming api [15:07] awesome!... yeah, then that shouldn't be hard to change the charm to do then [15:08] ohh i see, is a charm some sort of config file ? [15:08] senior7515: it might even make sense to make that a config option in the primary hadoop charm... have to think about it [15:08] true [15:09] m_3: any idea why juju ssh doesn't work ? [15:09] senior7515: yeah, poke around in the charm... the key places to look are the hook/install and hook/datanode-relation-changed [15:10] no clue... there are several situations where 'juju ssh' can be broken depending on your provider [15:10] this is more a problem with the bare metal or openstack providers [15:10] i'm on ec2 [15:10] ec2 and lxc providers shouldn't have that problem [15:10] ah, then you should be good [15:10] it doesn't work though :) [15:11] make sure your client version is up to date and matches what your environments.yaml says for 'juju-origin' [15:12] i.e., both should be 'distro' or both should be 'ppa' [15:12] ohh k cheking [15:14] m_3: sooo question what does it mean that the client matches juju-origin. I mean I only have one env set up 'prod' and it has juju-origin:distro [15:14] what else should be there ? [15:14] i just did an update on the system and didn't update juju… so I assumed is up to date [15:14] ah, so on your client machine you can install juju from the universe archives (just apt-get install juju) [15:15] or from a ppa (a more recent version of juju) [15:15] I did… my client is a server on ec2 cuz the mac juju is broken .. I reported the bug [15:15] but besides the point [15:15] yeah [15:15] is updated from the ppa [15:16] the client machine is 10.04.4 LTS soo I had to use the ppa [15:16] ah, ok... then if you're client machine is installed from the ppa, then your environment.yaml file should have 'juju-origin: ppa' [15:16] this makes sure that the version on your client matches the version installed on the instances [15:17] shouldn't matter too much, but that might explain an ssh breakage [15:17] oh...hmmm [15:17] if your client is on ec2, then I recommend precise [15:17] 10.04 is pretty old for juju [15:17] hmm i see... [15:18] safer route to take... especially if you spun it up just for this... definitely use precise [15:18] ok got you.. soo basically spawn up precise new instance. install juju there… [15:19] will it reattach to the hadoop-master/0 [15:19] if I do juju bootstrap ? [15:22] k so changing to ppa didn't work, spawning new instance for juju [15:23] so basically juju is a client, and it also spawns a server on ec2 or whatev… with juju software installed. so the client sends commands to the juju server that it spawns and then it does the dirty work of installing and configuring instances etc ? [15:23] senior7515: safest, if you can do it, is just drop it all... then spin up a precise client... then bring up a new stack of services based on the lp:charms/hadoop charm.. and test from there [15:23] ok... [15:23] will do. then... [15:23] senior7515: yes, great description [15:26] ok off to killing and spawning [15:27] that's totally my life lately... waiting on ec2 :) === TheMue_ is now known as TheMue [16:13] m_3: soo ok I finally terminated everything and i'm up and running with juju server and client installed on precise [16:13] do I just do juju deploy hadoop-master ? [16:13] or I have to clone the charms repo [16:13] and install from a local dir ? [16:15] senior7515: sorry.. in a meeting atm... clone lp:charms/hadoop, not hadoop-master [16:15] the one hadoop charm itself is the one to use [16:16] the README file in lp:charms/hadoop has some great walkthroughts [16:16] * m_3 finding link [16:17] https://code.launchpad.net/~charmers/charms/precise/hadoop/trunk [16:17] senior7515: ^^ [16:18] senior7515: and http://jujucharms.com/charms/precise/hadoop [16:18] the last link shows the README with the examples [16:18] * m_3 gotta run [16:19] m_3: thanks a lot! === avalanch_ is now known as avalanche123|w [17:36] arosales: https://blueprints.launchpad.net/sprints/uds-q?searchtext=juju [17:36] arosales: ok so I made all the juju sessions people wanted [17:36] but then I realized that I have super launchpad powers so they become autoapproved. [17:36] jcastro: thanks man! [17:36] arosales: what's supposed to happen is you need to approve the ones under servercloud-q-juju [17:37] * arosales is looking [17:37] so I think to make up for that you can just tell me which juju ones you think you'd like to see consolidated, etc. [17:37] or if you think they're good to go [17:37] the only one I'm confused about is this intelligent brain thing [17:37] 17 for servercloud track . . . [17:38] m_3: ^^^ more PhD research? :) [17:38] jcastro: does that fit into the schedule ok? [17:38] yep, it does [17:38] jcastro: can you kill the first versions "/juju" of the two blueprints that were already there? [17:38] I think I'd rather overbook a little bit because we can always say "we don't need this, remove it." on the fly [17:38] jcastro: if it fits into the schedule ok, its ok with me. [17:38] whereas "oh no we need to find room for 5 more sessions this week" would be tough [17:38] jcastro: that stuff's just about autoscaling tools [17:38] we can bump it to next series if you want [17:39] people have been asking for it [17:39] m_3: and SpamapS probably have a better pulse on things that can consolidated though [17:39] m_3: done [17:39] danke [17:39] m_3: ok I can fix the description so that's more obvious [17:40] by all means [17:40] are these dups? [17:40] https://blueprints.launchpad.net/juju/+spec/servercloud-q-juju-intelligent-infrastructure [17:40] https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-juju-intelligent-infrastructure [17:40] the first one is for the wrong project, I just declined it [17:40] so we're good [17:41] jcastro: same with https://blueprints.launchpad.net/juju/+spec/servercloud-q-juju-integration [17:41] arosales: yeah, I'd mistakenly put them under /juju instead of /ubuntu [17:41] and https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-juju-integration [17:41] yep, killed that one too [17:41] ah ok [17:41] I just refiled them under ubuntu that's why they're there, though I wonder why LP doesn't drop them from the list [17:42] jcastro: thanks for documenting those blueprints [17:42] hulk should be happier now [17:42] pro tip for you guys: [17:42] always schedule early, even if the bp is empty. [17:42] he's totally gotta be watching for that name now :) [17:42] because when the slots fill you are doomed [17:42] so I like to claim my space early [17:42] jcastro: good to know [17:42] then go back and fill it with content, agenda, etc. [17:43] jcastro: thanks for the info, that makes sense [17:43] m_3: when these schedule in about 15 minutes, what you'll want to do [17:43] is go into each session from the schedule [17:43] and like, layout an agenda and stuff [17:43] jcastro: if we need to we can shorten the mysql ultils session to just one hour. [17:44] m_3: that way you can just walk into the session and know what to do, as I always forget and UDS is a brain shuffling spaz fest, so I put my notes in the etherpad before I get on the plane. [17:44] jcastro: if servercloud spots become crampe [17:44] jcastro: you're critical in a couple of those too [17:44] yeah [17:45] i've criticalled you guys on some of mine as well [17:45] arosales: http://summit.ubuntu.com/uds-q/2012-05-07/display [17:45] looks like only about 1/3 of the rooms have been booked per hour [17:45] so we should be good [17:45] jcastro: ok, thanks. [17:46] arosales: I'm going to try to make Thursday as juju empty as I can so we can have time for contest/demo planning, etc. [17:46] yeah, we totally have a lot of overlap [17:46] I'm just waiting an hour for the schedule to settle, then I'll make ours nice and smooth [17:46] charm release process -vs- charmstore maintenance [17:47] juju best practices -vs- charm workflow [17:47] cool [17:47] I want workflow to be more about fixing the 10 step process [17:47] than the actual best practices for authors [17:48] * m_3 just wants to go to desktop icon sessions all week [17:48] jcastro: cool, thanks. [17:48] jcastro: also it looks like http://summit.ubuntu.com/uds-q/meeting/20357/servercloud-q-cloud-imgages/ [17:48] is having some issues [17:48] I think utlemming filed a bug [17:49] its linking to the wrong blueprints, and thus not scheduling with the correct folks who have limited availabilit [17:49] availability [17:49] I think the misspelling threw it off at first, I'll check [17:50] jcastro: thanks for taking a look. [17:50] jcastro: if need be, can we manually move it to monday or tuesday [17:50] k [17:51] jcastro: thanks for all help, and documenting the juju blueprints [17:54] arosales: tuesday, 16:15 [17:55] I think that works, I'll confirm with utlemming in ubuntu-server [18:01] jcastro: there were two on charm testing... James had a charm-testing in general [18:01] in addition to the unit-tests [18:01] m_3: oh those I combined, you need them seperate? [18:02] no, combined is fine with me [18:02] also juju-upstart-integration was william's... [18:02] don't know if James added himself on that or you did [18:05] jcastro: Barton's asking for you [18:05] ok, hopping on [18:05] you need the #? [18:11] how can we customize the installed packages on all systems ? to use a configuration engine like puppet or chef, I need it pre-installed on all system... [18:13] flepied: it's easy to add to the top of the install hook... there're discussions of adding some packages to metadata, but it's easy to do in the install hook [18:14] flepied: you can add ppas or other package sources there too.. it's just shell script [18:15] m_3, I don't want to do apt-get in the install hook, I want to use puppet for example... [18:15] flepied: right, I'm saying you can bootstrap puppet in the install hook [18:16] that's easy with puppet... chef's a little more work, but it's easy to add too [18:17] jimbaker: i saw something in my scrollback re that relation-get bug i hit last week. has that landed yet? [18:17] m_3, in which install hook ? [18:18] flepied: actually, that might make a great subordinate charm... [18:19] that can be deployed alongside of any charm [18:19] flepied: but note that juju does the "service orchestration" or coordination... best integration so far is puppet in masterless or chef in solo mode [18:19] flepied: this is a topic for the ubuntu developer summit next week [18:20] flepied: (juju integration in general) [18:20] capistrano's in there too [18:20] adam_g, not yet [18:21] however, you can try out that branch and see that it works for you or not - i just need to add appropriate testing before it can land [18:21] jimbaker: ah, ok. sounds good [18:22] adam_g, you can try out a version of the change in lp:~jimbaker/juju/debug-relation-hook-context, just set juju-origin as usual [18:23] m_3, I don't think it will work to use a subordinate as it will be too late to be able to use puppet to install what is needed [18:25]