[00:02] <xavierbick> Hello everyone, I saw this bug on launchpad (https://bugs.launchpad.net/juju-core/+bug/1336473) and wanted to submit a fix to github, but I'm not quite sure what the normal procedure is for contributing a bug fix to juju.  Can I just assign myself to the bug and mark it as in progress?
[00:02] <mup> Bug #1336473: Support new t2 instance types on AWS <ec2-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1336473>
[00:03] <davecheney> xavierbick: sure
[00:04] <xavierbick> great, thanks davecheney
[00:05] <perrito666> davecheney: I need to make the api server able to restart jujud with a given result from a FacadeCall, much like what happens for workers but in this case the call would answer to the client forwarding the result and then restart, yet I am not very sure where to add this
[00:07] <davecheney> perrito666: i think someone else asked this question recently
[00:07] <davecheney> menn0-afk: did you ask this recently
[00:12] <perrito666> davecheney: I send a similar question to the mailing list some time ago
[00:12] <perrito666> before understanding the actual problem fully
[00:17] <davecheney> ahh
[00:50] <perrito666> davecheney: that means you dont have an answerfor me right?
[00:50] <perrito666> :p
[00:53] <davecheney> perrito666: i'm not even sure I understand the question
[00:53] <perrito666> I most likely suck at explaining it
[00:54] <davecheney> perrito666: you want an api method to restart the apiserver, but not just exit the process ?
[00:54] <perrito666> Ill try to explain better, bear with me Its late :)
[00:57] <perrito666> I created a Client API method for Restore, which basically... restores :)
[00:58] <perrito666> now restore its a rather destructive thing which shuffles things around so, after finishing it I need to restart jujud
[00:59] <perrito666> so It was suggested by william that I should handle the return value of said call and restart jujud on a given result
[00:59] <perrito666> now, even if restarting jujud, I need to answer to the client first so it acknowledges that the command finished
[01:04] <davecheney> perrito666: i dunno
[01:04] <davecheney> i'd just do exit(0)
[01:08] <perrito666> davecheney: what I am not sure is where to :) there is a blurry area between state/api/client and state/apiserver/client :p which is not all that clear to me, even though is properly documented
[01:08] <perrito666> anyway, 31C at 11PM is too much for my brain I am going to sleep under the AC until the head clears off
[01:09] <perrito666> jam: if you read the abovewhen you arrive and have an answer please drop it here and Ill read it in the morning
[01:09] <davecheney> 31c, fuk that noise
[01:09] <perrito666> noise?
[01:10] <perrito666> 31°C <-- temp, fun fact, we are in the middle of winter
[01:10] <davecheney> bzzt
[01:40] <wallyworld> davecheney: that status bug has been fixed in trunk, i just backported to 1.20
[01:40] <wallyworld> you did the fix for trunk a little while back
[01:41] <davecheney> it's sort of a fix
[01:41] <davecheney> it's still shit
[01:41] <davecheney> we should just stop trying to carry on if api.GetStatus screws up
[01:58] <wallyworld> sure, but at least the nil pointer is fixed and 1.20 is consistent with master
[02:00] <wallyworld> axw: just otp, there soon
[02:00] <axw> wallyworld: nps
[02:05] <wallyworld> axw: ready now
[02:36] <katco> wallyworld: i need to take care of some things, but i think i addressed all the issues your brought up in https://github.com/juju/juju/pull/584
[02:37] <wallyworld> katco: awesome, thank you, will look
[02:37] <katco> wallyworld: no, thank you :)
[02:38]  * katco away for a bit, maybe for the night.
[02:42] <axw> wallyworld: just putting my desk back together, will bbs to review
[02:42] <wallyworld> sure
[03:26] <jam> perrito666: state/api/client is the actual user/client/not-the-server side of the code, and state/apiserver/client is the stuff running inside the API server. "client" in this case is that it is the set of "client facing" APIs (Status, Deploy, Upgrade, etc), not Agent facing APIs.
[03:27] <jam> perrito666: essentially everything in state/apiserver should be mirrored as state/api, the former being the one producing the API, and the latter being the one consuming it.
[03:28] <jam> perrito666: as for restarting gracefully, the worker/Upgrader code does this by raising UpgradeReadyError
[03:28] <jam> which signals up the worker/runner stack that we want to restart our process.
[03:28] <jam> However, that is being run in a *worker* not during an API call.
[03:29] <jam> I don't have an immediate "this is clearly how to do it" but we should probably take this to an on list discussion, so that fwereade et al can get in on the conversation
[03:52] <wallyworld> axw: manual provisioning question - in manual/provisioner.go, there's a ProvisionMachine() method; know the one i mean?
[03:55] <axw> wallyworld: yup?
[03:55] <wallyworld> in that method, we call args.Client.ProvisioningScript
[03:55]  * axw nods
[03:55] <wallyworld> and DisablePackageCommands will be false
[03:55] <katco> wallyworld: i have to go to bed, but i just pushed up more changes. thanks for the review.
[03:55] <wallyworld> katco: np, still going through it :-)
[03:55] <axw> wallyworld: orly? /me looks
[03:56] <wallyworld> katco: thanks for the effort :-)
[03:56]  * katco hands wallyworld a fine-toothed comb
[03:56] <wallyworld> lol
[03:56] <wallyworld> i don't have nits :-P
[03:56] <katco> wallyworld: yeah well, this has drug out long enough. i wanted to take advantage of you being online ;)
[03:56] <axw> wallyworld: DisablePackageCommands isn't set, so yes, it'll be false
[03:57] <wallyworld> axw: so that means we will run upgrade/update on manual machines, no?
[03:57] <katco> ok, sorry, off to bed. thanks again wallyworld, axw.
[03:57] <axw> wallyworld: ProvisioningSCript calls cloudcfg.SetAptUpgrade(false)
[03:57] <wallyworld> katco: night
[03:57] <axw> katco: night night
[03:58] <axw> wallyworld: the idea was that manual should only install what it needs, and not muck about with the machine otherwise
[03:58] <wallyworld> axw: oh, so it does, so it ignores the DisablePackageCommands
[03:59] <wallyworld> whereas now, we want to default to the current behaviour, but allow it to be overridden
[03:59] <axw> wallyworld: IIRC, DisablePackageCommands is set on the MachineConfig in the API server code
[04:00] <axw> wallyworld: probably do that in the API server code then, I think.
[04:00] <axw> apiserver/client
[04:00] <wallyworld> axw: it's not set all for the ProvisioningScript call
[04:00] <wallyworld> it's only set on machineconfig by the lxc container manager
[04:01] <axw> wallyworld: it is. look at state/apiserver/client/client.go
[04:01] <axw> there's a Client.ProvisioningScript method that sets up the MachienConfig, then calls manual.ProvisioningSCript
[04:03] <wallyworld> yes, i meant that nothing sets its value to true
[04:03] <axw> wallyworld: kapil's external code does :)
[04:03] <wallyworld> right
[04:03] <wallyworld> nothing in juju
[04:04] <wallyworld> so instead in juju we hard code disableaptupgrade to false
[04:04] <wallyworld> so now in the new branch, that is configurable
[04:04] <axw> wallyworld: upgrade is *always* false for manual, regardless of what you do. you can *also* disable update&install by setting DisablePackageCommands
[04:04] <axw> ok
[04:04] <axw> mmm
[04:05] <axw> yeah, that sounds fine
[04:05] <wallyworld> ie it defaults to disabled like now
[04:05] <wallyworld> but the user can choose to have their box upgraded if they want
[04:05] <axw> as long as it doesn't override DisablePackageCommands
[04:06] <wallyworld> that's the hard bit i'm pondering about - how to stay backwards compatible
[04:06] <wallyworld> since we don't know if DisablePackageCommands was explicitly set or not
[04:07] <wallyworld> if think if it is false, we use the new config settings
[04:07] <wallyworld> if true, we disable everything regardless of the settings
[04:08] <axw> wallyworld: +1
[04:08] <axw> wallyworld: the client side doesn't need to change at all
[04:08] <wallyworld> it's not "perfect", bu i think the best we can do
[04:08] <axw> just state/apiserver/client/client.go
[04:08] <wallyworld> yep
[04:09] <axw> manual.ProvisioningScript shouldn't force upgrade=false anymore, that needs to move into apiserver
[04:09] <axw> which has access to config
[04:11] <wallyworld> yep
[04:21] <menn0> wallyworld, thanks for dealing with bug 1359435
[04:21] <mup> Bug #1359435: Next version selection for upgrades is no longer correct <juju-core:Triaged> <juju-core 1.20:In Progress by wallyworld> <https://launchpad.net/bugs/1359435>
[04:22] <wallyworld> menn0: no problem, thanks for noticing the issue
[05:17] <wallyworld> axw: a small one https://github.com/juju/juju/pull/591
[05:18] <axw> looking
[05:20] <wallyworld> axw: thanks
[05:59] <davecheney> wallyworld: could you please kill http://juju-ci.vapour.ws:8080/job/github-merge-juju/390/console
[05:59] <davecheney> it's not going to pass and there is a large backlog behind it
[05:59] <wallyworld> davecheney: it will pass, why won't it?
[06:00] <wallyworld> that uniter failure happens all the time
[06:00] <wallyworld> passes 2nd time through
[06:01] <davecheney> wallyworld: ok,
[06:01] <wallyworld> davecheney: did i mention i hate our tests?
[06:14] <davecheney> wallyworld: to tell
[06:15] <wallyworld> of late, most runs require the fallback -p 2 to pass
[06:15] <wallyworld> i will be cataloging the failures and trying to get people to fix them
[06:18] <davecheney> wallyworld: +1 to turning off the retry
[06:19] <davecheney> tests pass or GTFO
[06:19] <wallyworld> i think so, we'll try and get an itemised list of failures first
[06:26] <davecheney> you'll have no shortage of canidates then
[06:47] <davecheney> wallyworld: i'll close with this graph, http://juju-ci.vapour.ws:8080/job/github-merge-juju/buildTimeTrend
[06:47] <davecheney> things clearly went wrong, around this commit waigani endpoint-on-bootstrap-take3
[06:47] <davecheney> not saying it was this commit
[06:48] <wallyworld> davecheney: yeah, early after the cutover, we were building in 18 or so minutes
[06:49] <wallyworld> we had hoped to get it down to 10 or 11 with the use of lxc or nail up instances
[08:02] <mattyw> morning folks
[08:11] <TheMue> morning
[08:14] <voidspace> TheMue: morning
[08:15] <voidspace> TheMue: I updated this PR to use errors.Annotate everywhere
[08:15] <voidspace> TheMue: https://github.com/juju/juju/pull/561
[08:15] <TheMue> voidspace: great, will take a look
[08:15] <voidspace> TheMue: I would appreciate a quick look for the sake of sanity when you have a moment
[08:15] <voidspace> thanks!
[08:16] <voidspace> TheMue: as I was touching the file I did all of the ones that were appropriate to change
[08:16] <voidspace> (in some cases there isn't an existing error to annotate of course, so those remain Errorf)
[08:16] <TheMue> voidspace: I’m still fighting with changing an existing API part into a new version. it’s a rattail :/
[08:16] <voidspace> fun
[08:17] <TheMue> voidspace: the errors package IMHO has useful funcs there too
[08:17] <voidspace> TheMue: ok
[08:18] <TheMue> voidspace: you’ve got a LGTM
[08:19] <voidspace> TheMue: awesome, thanks
[08:19] <voidspace> TheMue: and are you happy(ish) for me to merge the other PR without an additional test?
[08:19] <voidspace> TheMue: given that we concluded that we *do* test (in provider specific ways) that only the requested ports are open
[08:20] <TheMue> voidspace: here’s the errors package http://godoc.org/github.com/juju/errors
[08:20] <TheMue> voidspace: have to take a deeper look myself
[08:20] <voidspace> TheMue: thanks
[08:20] <voidspace> I will peruse
[08:20] <voidspace> I wonder how good the documentation is
[08:22] <TheMue> voidspace: and regarding the other PR I can live w/o another test, yes. it only have been more general thoughts about how to ensure that only the wanted ports are open. but that would be a huge change :)
[08:22] <voidspace> TheMue: and we have to open some ports in a provider specific way - as we have to specify some ports when we create the machine for some providers I believe
[08:22] <voidspace> as we need some ports immediately
[08:23] <voidspace> so a *general change* is maybe not even possible
[08:23] <voidspace> although we don't need the state port immediately just to bootstrap
[08:23] <TheMue> voidspace: yep, you’re right
[08:57] <Beret> wallyworld, nice update, thanks
[08:58] <Beret> wallyworld, we'll do all we can to get you more information when we hit those LXC issues
[08:58] <wallyworld> Beret: no problem and thank you
[10:30] <perrito666> morning all
[10:50] <voidspace> perrito666: morning
[10:50] <voidspace> dimitern: TheMue: standup?
[10:51] <TheMue> dimitern, voidspace: standup?
[10:51] <TheMue> voidspace: oh, just typed the same
[10:51] <voidspace> hah
[10:51] <TheMue> ;)
[10:51] <dimitern> voidspace, TheMue, sorry, brt
[13:00] <dimitern> voidspace, TheMue, others? networker refactoring branch - https://github.com/juju/juju/pull/593 , I'd appreciate a review when you have some time
[13:05] <wwitzel3> dimitern: I will take a look, I'm one of the on-call reviewers today :)
[13:06] <TheMue> dimitern: will look
[13:26] <dimitern> wwitzel3, TheMue, cheers guys!
[13:27] <perrito666> natefinch: 1:1?
[13:28] <perrito666> lol just saw the email
[13:35] <wwitzel3> man, trying to use our existing providers as a baseline for reviewing a new provider is turning out to be confusing.
[13:36] <wwitzel3> I feel like all our existing providers implement the same behavior as each other, just in a slightly different way.
[13:36] <wwitzel3> which provider should be considered the "gold standard"?
[13:36] <katco> wwitzel3: +1
[13:36] <katco> wwitzel3: i just did some work on providers, and i can't answer that.
[13:36] <mgz> wwitzel3: well, ah
[13:37] <mgz> they're all bad in different ways :)
[13:37] <katco> wwitzel3: if i had to guess, i'd say ec2 since it's probably been around the longest?
[13:37] <TheMue> wwitzel3: always the next one, which will make it all better
[13:37] <katco> TheMue: lol!
[13:38] <wwitzel3> I can distill these answers to "Yes, good luck"
[13:38] <wwitzel3> :D
[13:38] <TheMue> wwitzel3: +1
[13:39] <katco> wwitzel3: haha... well what are you trying to review? there are probably some concepts that in their pure form remain the same
[13:39] <wwitzel3> katco: the CloudSigma provider set of PRs
[13:39] <katco> wwitzel3: link to one?
[13:40] <wwitzel3> katco: https://github.com/juju/juju/pull/173
[13:41] <wwitzel3> katco: it is also the fact that it is just 2,000+ new lines. So there is no reference built in to the diff.
[13:41] <katco> wwitzel3: yeah, i see that! tough review...
[13:41] <wwitzel3> katco: that's why I'm trying to look to our existing providers as a reference / guide
[13:42] <katco> wwitzel3: yeah, i think i'd be doing what you are. and probably bouncing between ec2 which is probably the oldest, and maybe local which is one of the newer ones. or azure.
[13:43]  * katco dusts hands off. glad i could provide absolutely no value for you. ;)
[13:44] <wwitzel3> sanity checks are always helpful
[13:45] <perrito666> katco: well if you manage to make a career out of that you can become an argentinian real stateagent
[13:45] <wwitzel3> hah
[13:45] <katco> perrito666: i don't have it in me, unfortunately. it would drive me crazy.
[13:46] <perrito666> katco: there is good money in it and you can code while ignoring your customers
[13:47] <perrito666> fwereade: are you around?
[13:47] <fwereade> perrito666, yeah
[13:47] <katco> perrito666: lol
[13:47] <fwereade> perrito666, meant to ping yu this morning
[13:48] <fwereade> perrito666, you were asking last night about restarting the state server from within an api call?
[13:48] <perrito666> fwereade: heh yeah, our discussion the other day end up including more people and opinions and I am not entirely sure how to implement that :p
[13:48] <fwereade> perrito666, so, I have been 100% focused on leader-election things for the last day or two, I am probably missing context
[13:49] <perrito666> fwereade: Well I triedto implement that according to what we spoke, and suddenly I noticed that there was a gray area on state where I am not that familiar
[13:49] <fwereade> perrito666, are there concerns about the special-error approach?
[13:49] <fwereade> perrito666, ah, go on
[13:49] <fwereade> perrito666, I'd expect state to be out of the loop on that sort of question
[13:50] <perrito666> fwereade: the deal is, I am not entirely sure where is that I should be catching the error to ensure the client gets the response and trigger the restart (I might not be as good explaining myself as I think I am on this particular matter)
[13:53] <fwereade> perrito666, well -- somewhere after the function returns, and before it gets packaged up and sent back over the API
[13:53] <fwereade> perrito666, how many choices do you have there?
[13:54] <fwereade> perrito666, presumably it needs to be a layer that has, or can easily and sanely be given, access to the apiserver worker's tomb
[13:54] <fwereade> perrito666, so I would guess that the apiserver worker itself would be the right one to actually handle sending on the error to its runner
[13:54] <fwereade> perrito666, but I can well believe that we need another layer in there somewhere
[13:55] <fwereade> perrito666, I don't immediately recall what's actually in play there
[13:56] <perrito666> fwereade: as I get it th call propagates like this: apiclient->rpc layer->(X)->apiserver
[13:56] <perrito666> and I believe I should be standing in (X)
[13:56] <fwereade> perrito666, hmm, that sounds wrong
[13:57] <fwereade> perrito666, the apiserver needs to know that it should stop
[13:58] <fwereade> perrito666, and I think it has to finish the call, passing a "everything's good, restarting now!" response down via rpc, while arranging its own tidy suicide separately
[13:59] <perrito666> fwereade: I see, makes sense
[13:59] <fwereade> perrito666, possible approach: return result *and* error in that situation
[14:00] <fwereade> perrito666, in apiserver, define the error (type) that indicates "return these results and then terminate the api server with ErrStateRestored(?)"
[14:01] <fwereade> perrito666, except
[14:01] <fwereade> perrito666, can you really actually *do* the restore while everything's running?
[14:02] <fwereade> perrito666, what's the actual sequence of events?
[14:02] <perrito666> fwereade: yes, writing
[14:03] <natefinch> perrito666: standup?
[14:04] <perrito666> natefinch: getting there, I was looking for a webcam
[14:04] <perrito666> :p
[14:07] <TheMue> dimitern: great work, but I would like a second to take a look too
[14:08] <perrito666> fwereade: so this is how it works: [Bootstrap in safe mode][Upload File]<stop mongo><restore mongo><restore conf files for juju> <start mongo><restart jujud?>
[14:09] <fwereade> perrito666, then I don't quite understand where the API server comes in
[14:10] <fwereade> perrito666, ah: so immediately after you've bootstrapped, you tell the API server to do all that stuff?
[14:10] <perrito666> yup
[14:10] <perrito666> fwereade: so once I have a working api server I ask for it to perform the restore
[14:10] <fwereade> perrito666, and is the API server still responding to requests while this is going on?
[14:12] <perrito666> fwereade: mm I see where you are going, I could make API server deny requests, much like what upgrade does
[14:12] <perrito666> update
[14:12] <sinzui> natefinch, jam: We have a new regression. Can you find an engineer to fix or revert the problem? https://bugs.launchpad.net/juju-core/+bug/1360286
[14:12] <mup> Bug #1360286: ppc64el and i386 unittests cannot find tools <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1360286>
[14:12] <fwereade> perrito666, I'm really rather nervous about having the api server running at all while this is going on
[14:14] <fwereade> perrito666, in particular I am nervous about what firewaller, networker, etc might do while running against incorrect state
[14:14] <fwereade> perrito666, the provisioner might take down machines so that's the obvious risk
[14:14] <perrito666> fwereade: actually the state is down (and API will not connect to it until restarted)
[14:15] <fwereade> perrito666, how do you ensure it's down? starting jujud will start all sorts of workers, surely, and you won't know to shut them down until you get this api request
[14:21] <perrito666> fwereade: hold, I think we could be not on the same page here, let me finish this standup and then I can give you my full attention (I would really love tho have a white board between us but that is too much asking)
[14:22] <fwereade> perrito666, a whiteboard in one's webcam's FOV is a useful thing, I never got round to setting it up though
[14:22] <fwereade> perrito666, could be
[14:22] <fwereade> perrito666, natefinch: brb, but will join yu in your standup in 5
[14:22] <perrito666> fwereade: we are no longer there, just ping me when you are back
[14:29]  * perrito666 suddenly hits his head on an open window... brb
[14:29] <fwereade> perrito666, natefinch: will be in the hangout when you're ready
[14:29] <perrito666> back
[14:30] <ericsnow> fwereade, perrito666: mind if I join?
[14:30] <fwereade> ericsnow, ofc not
[14:30] <perrito666> ericsnow: you are welcome to join
[14:30] <ericsnow> fwereade, perrito666: you guys are the best! :)
[15:43] <perrito666> aghh my ears are aching
[15:44] <natefinch> anyone have thoughts on why a machine after upgrading from 1.18 to 1.20 would says "missing API Hosts"?
[15:45] <wwitzel3> it's lonely?

[15:45] <perrito666> natefinch: agent.conf corrupted?
[15:46] <natefinch> perrito666: interesting you should say that, he did say he had to update the agent.conf by hand to add apiaddresses
[15:46] <natefinch> or at least, he thought that was the problem, but adding it didn't help
[15:50] <natefinch> can someone else give me a hand? I may have to leave soonish.  On #juju in canonical irc, guy's nick is ivoks, he's onsite with a customer right now.
[15:55]  * fwereade bbs
[15:56] <perrito666> natefinch: sorry was buying foood
[16:20] <katco> how do i got about upgrading an environment to my dev version? "juju upgrade-juju --dry-run --version 1.21-alpha1" gives me "ERROR no matching tools available"
[16:23] <natefinch> --upload-tools should work I think
[16:24] <natefinch> can anyone else help ivoks in canonical irc #juju?  I have to go
[16:25] <natefinch> sinzui, alexisb, mgz, wwitzel3 ^
[16:25] <natefinch> perrito666 was looking at iot but had to go too
[16:25] <katco> natefinch: ah i see... it doesn't like --upload-tools with --dry-run
[16:25] <katco> natefinch: that's what was tripping me up
[16:26] <sinzui> katco, interesting. tip is definitely broken for ppc64el and i386, what arch you testing?
[16:26] <sinzui> see bug 1360286
[16:26] <mup> Bug #1360286: ppc64el and i386 unittests cannot find tools <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1360286>
[16:26] <katco> sinzui: amd64, i think it was just the combination of --dry-run
[16:27] <katco> sinzui: if i do --upload-tools alone, it gives me a list of available tools, and best version
[16:27] <sinzui> katco, oh, we have one or two related bugs where --upload-tools doesn't in other combinations
[16:27]  * sinzui looks
[16:30] <sinzui> katco, These are the upload-tools bugs https://bugs.launchpad.net/juju-core/+bugs/?field.tag=upload-tools and bug 1307643 looks like the issue
[16:30] <mup> Bug #1307643: juju upgrade-juju --upload-tools does not honor the arch <upgrade-juju> <upload-tools> <juju-core:Triaged> <https://launchpad.net/bugs/1307643>
[16:32] <katco> sinzui: interesting, thank you so much. it looks like it might? have worked?
[16:32] <katco> sinzui: i'm noticing it incrementing the agent-version
[16:33] <sinzui> katco, status shows the new version?
[16:33] <katco> sinzui: yeah: 1.21-alpha1.x
[16:33] <katco> sinzui: where x is incrementing...
[16:34] <sinzui> katco, upload-tool will always do that
[16:35] <sinzui> katco, that is to distinguish a version taken from the client instead of the net
[16:35] <katco> sinzui: so how do i know that it's actually updated? state is started
[16:41] <sinzui> katco, I trust the log 99% of the time. When I doubt, I ssh to the machine and run "jujud version"
[16:42] <sinzui> katco, I have never seen it differ from status. Once, did this and found th reason for a failed upgrade was bad symlink. There were to jujuds on the machine. I deleted the symlink and juju correctly upgraded itself a minute later
[16:43] <katco> sinzui: lol ok, so rule of thumb: trust agent-status
[18:17] <perrito666> ericsnow: you are a cruel man, such lengthy email on a friday afternoon
[18:19] <perrito666> ericsnow: why would backup allow upload?
[18:20] <perrito666> ericsnow: I have not yet looked at the metadata but I do have a request
[18:21] <perrito666> ericsnow: lemme know when we can have a call
[18:56] <perrito666> ericsnow: ?
[19:09] <ericsnow> perrito666: hey just got back
[19:10] <perrito666> hey, read above :)
[19:10] <ericsnow> we should hang out
[19:10] <ericsnow> moonstone?
[19:10] <perrito666> ericsnow: sure
[21:57] <katco> rick_h__: you still around?