/srv/irclogs.ubuntu.com/2017/02/16/#juju.txt

Budgie^Smorewell it has been one of those days!02:16
lazyPowerBudgie^Smore how so?03:01
Budgie^Smoreended up reinstalling the work laptop ... windows to ubuntu03:02
lazyPowerah the great nuking and repaving day03:03
Budgie^Smoreworking on a Vagrantfile to create the different VMs that I need... came across an old git repo of someone doing that for MaaS03:03
Budgie^Smoreoh and the window to ubuntu change was cause I couldn't get ssh to work nicely so I could ssh and start a VM headless from the CLI03:07
kjackalGood morning juju world!07:34
=== jamespag` is now known as jamespage
kklimondawhen a hook is failing, I can run juju debug-hooks unit to see what's happening08:17
kklimondahow can I change options being passed to the hook?08:17
kjackalhi kklimonda I am not sure you can change the parameters passed to the hook. However since you are in debugging mode you can alter the values in tha parameters you already get, right?08:29
=== frankban|afk is now known as frankban
kklimondayes, I can definitely do that08:46
kklimondaor I could if this wasn't used in a dozen different places.. I have to figure out where to set it in hooks09:13
ZiclazyPower: ping back09:16
Zic(but you're probably sleeping :p)09:16
kklimondaif bundle deployment fails on one machine, can I retry just this one machine?09:44
BlackDexi'm working on a shared server with someone, and we both use juju but with different envirionments10:24
BlackDexi mean models10:25
BlackDexare there envirionment variables i can use so that juju selects that model instead?10:25
BlackDexbecause if i, or someone else does juju switch, its selecting that model10:26
BlackDexJUJU_ENV isn't working10:27
anrahwould JUJU_MODEL work?10:28
BlackDexno idea10:29
BlackDexlets check10:29
BlackDexyes10:30
BlackDexthat sounds like a nice feature10:30
BlackDex:)10:30
BlackDexA pitty it isn't documented10:30
BlackDexhmm it is on github i see10:32
anrahBlackDex: https://jujucharms.com/docs/stable/reference-environment-variables10:32
BlackDexhow could i have missed that10:33
BlackDexi search for environment in the docs10:33
BlackDex:S10:33
BlackDexprobably some typo10:33
BlackDexthx!10:33
anrahnp! :)10:33
anrahDoes bundle deployment work with manual cloud?10:42
anrahI have manually added couple servers to juju and I would like to use bundle-file to deploy my apps10:42
anrahI get errors for each machine:10:43
anrahplacement "0" refers to a machine not defined in this bundle10:43
anrahwhen setting machine-part on the bundle, i get error:11:12
anrahERROR cannot deploy bundle: cannot create machine for holding my-charm unit: cannot add a new machine: use "juju add-machine ssh:[user@]<host>" to provision machines11:12
=== Guest80064 is now known as CyberJacob
=== frankban is now known as frankban|afk
alexlist@jcastro: in LP#1662172 I mentioned that conjure-up didn't copy .kube/config and kubectl to the controlling host, and just noticed the workaround is documented here: https://github.com/juju-solutions/bundle-canonical-kubernetes/tree/master/fragments/k8s/core - however to streamline things, I suggest to amend the docs to copy kubectl to ~/bin which should be in people's path if they use the default .profile from /etc/skel12:16
jcastroI did this a bunch of time yesterday and at the end it prompted and copied the binary over12:29
jcastrooh, and also, if not obvious, for 1.5.x we stopped bundling the elastic bundle by default, though you can still deploy after the fact12:30
jcastrooh ok, I see we put that workaround in the docs anyways: https://kubernetes.io/docs/getting-started-guides/ubuntu/installation/12:31
jcastroalexlist: any other feedback on that page before I start a PR?12:34
alexlistjcastro: Not yet, will probably redo the whole thing once more in a VM to verify...12:35
jcastrodid you deploy the bundle manually or via conjure?12:35
alexlistconjure12:36
jcastrook, at the end conjure should prompt you to copy the creds and binary over automatically12:36
alexlistLemme try this once more...12:36
jcastroif that doesn't happen lmk the version of conjure and we can have stokachu take a look12:36
alexlistok12:36
jcastrohttps://github.com/kubernetes/kubernetes.github.io/pull/255612:45
jcastroa review here would be lovely!12:45
stokachualexlist, yea lemme know as it's supposed to copy to ~/bin automatically in the steps view12:51
jcastrooh ok so conjure already copies to ~/bin?12:51
jcastrook so really the manual steps were the ones that needed to be fixed then12:51
jcastrothen when rye has the readmes use the upstream markdown instead of the bundle markdown it should all generate from one source of truth instead of the two we have now12:53
ahasenackmarcoceppi: hi, around?13:13
ahasenackdoes anybody know if there is a python3?-libchamstore package for trusty somewhere? It used to be in ppa:juju/devel, but the latest build there failed13:13
marcoceppiahasenack: I'm fixing that today13:14
ahasenackthe trusty build?13:14
=== rvba` is now known as rvba
ahasenackI actually don't know what is pulling libcharmstore in13:14
marcoceppicharm-tools13:14
ahasenackwhere will it be uploaded to?13:15
marcoceppijuju/stable13:15
SimonKLBhey marcoceppi! could you tell me what the current best practice is for exposing charms deployed in lxd containers on aws? when im running locally with nested lxd containers i create some simple NAT rules in iptables, but when im running in a public cloud i also need to get the proper security rules in place and those doesnt seem to be created when executing `juju expose X` on the containerized application13:24
SimonKLB(sorry, that was a long question) :D13:25
rick_hSimonKLB: since you can't get at the containers from the outside you need something to help proxy things.13:26
rick_hSimonKLB: usually i setup a HAProxy on the root of the host13:26
marcoceppirick_h: does the network setting changes in 2.1 help address this?13:28
rick_hmarcoceppi: not atm. You still can't get multiple addresses/mac addresses so the containers are internet addressable13:28
SimonKLBrick_h: right! it would be useful to have some mechanism to expose containerized applications though, for example in the openstack bundle where lots of components are put in lxd containers instead of directly on the host13:28
rick_hSimonKLB: yes, agreed. So the team is working to enable that when there's something that allows it. The network changes in 2.1 are a move in that direction and I know that by 2.2 the idea is to have that on places like manual provider, openstack, places where you might be able to get dhcp to the containers for root level ip addresses13:29
rick_hSimonKLB: but AWS does some things in their SDN that only allows the one mac address on hosts so it's harder to get containers exposed13:30
SimonKLBrick_h: even without dhcp access, NAT:ing could be an option, you just need to first open the ports in the security rules13:30
rick_hSimonKLB: right, but because it has to be one mac the host can't NAT multiple things inside and tell who it goes to is my understanding. It could only do something based on one container per port perhaps.13:31
rick_hSimonKLB: but yea, atm you have to handle that via a proxy and config and such but the team's actively working on it across many of the provders13:32
SimonKLBrick_h: yea, that is what im doing right now, adding NAT rules with destination port as the match, so if you expose an application that is running a service on port 80 you add the iptables NAT rule and then expose the application like you would if it was running on the host13:33
SimonKLBfor example exposing keystone in the openstack-base bundle: iptables -t nat -A PREROUTING -p tcp --dport 5000 -j DNAT --to [private ip]:500013:35
SimonKLBthis works great when you're running on localhost13:36
alexliststokachu: it did indeed copy the files to ~/.kube/config and ~/bin/kubectl, but the last steps in conjure-up still threw errors13:44
stokachualexlist, what error?13:44
alexliststokachu: http://pastebin.ubuntu.com/24007303/13:45
ZicI had the same error today alexlist / stokachu13:45
Zicconjure-up tries to run kubectl get nodes / get pods before the cluster is ready during the deployment13:45
alexliststokachu: most likely a race condition - now it tried to copy the files even though the deploy isn't finished yet13:45
alexlistwhat Zic said...13:46
Zic(I just ignored the error, and the installation finished correctly btw)13:46
stokachuwhat rev are you guys on?13:46
Ziclet me check13:46
stokachusnap list conjure-up13:46
ZicI don't use the snap13:46
Zicit's the PPA version13:46
stokachuZic, ah, should migrate to the snap when you can13:47
Zic2.1.0-0~201701041302~ubuntu13:47
Zicstokachu: noted :)13:47
alexlist2.1.0-0~201701041302~ubuntu16.10.113:47
ZicI'm on Ubuntu 16.0413:47
stokachuyea you guys should be using the snap version13:47
Zicok13:47
stokachumy test runners haven't seen this error yet if you do a `sudo snap install conjure-up --classic --candidate`; `sudo apt-get remove conjure-up juju-2.0`13:48
alexliststokachu: I just followed https://jujucharms.com/canonical-kubernetes/ which tells me to use the PPA...13:48
stokachuconjure-up provides everything you need13:48
stokachualexlist, yea ive got a PR to get that changed13:48
stokachualexlist, hasn't landed yet13:48
alexlist\o/13:48
stokachuwith snaps you can deploy on trusty now too13:49
stokachulatest juju etc13:49
Zicstokachu: I don't have any preference between deb or snap, but https://jujucharms.com/canonical-kubernetes/ should be upgraded with the snap package installation I guess13:49
Zicit's the only reason of why I used the PPA :)13:50
stokachuZic, yep soon as they land the PR and bup charm revs13:50
Ziccool13:50
=== frankban|afk is now known as frankban
BlackDexIs there someone here who can help me with some problems with the nrpe charm? It doesn't install/announce the disk/mem/cpu checks? stub, blahdeblah, hloeung, pjdc_ or someone else?13:51
magicaltroutthere's a bug and a fix open for that I think BlackDex13:51
stokachuZic, alexlist once Juju 2.1 GA ive got deb updates that will point you to the snap install version13:52
alexliststokachu: ok.13:52
magicaltroutcause i used it a bunch of times13:52
BlackDexmagicaltrout: if you mean bug 1605733, i couldn't find that file anymore in the new charms13:52
mupBug #1605733: Nagios charm does not add default host checks to nagios <canonical-bootstack> <family> <nagios> <nrpe> <unknown> <nagios (Juju Charms Collection):New> <https://launchpad.net/bugs/1605733>13:52
alexliststokachu: Do you think this will work on a plain Debian as well, or are there too many things missing? Just asking, as I have to deal with a managed hosting provider who prefer Debian ...13:53
stokachualexlist, i think there are efforts to get snapd running on debian13:53
magicaltroutyeah BlackDex well that was my issue 3 weeks ago13:53
stokachui think it already does on the latest13:53
magicaltrouti've not checked it since13:53
magicaltroutbut it was around like that for ages13:53
stokachualexlist, you should join #snappy and see if one of those guys know more13:54
stokachualexlist, but yea, the theory is wherever snappy can run you'll be able to use conjure-up13:54
stokachufedora, arch etc13:54
BlackDexi will check it again by using the charm-tools to download the charm13:54
magicaltrouti just juju ssh into the unit and hack around the code, but whatever floats your boat :)13:55
BlackDexthat is also an option, but not if you want to deploy it to a lot of instances13:55
magicaltrouthttp://bazaar.launchpad.net/~charmers/charms/precise/nagios/trunk/view/head:/hooks/common.py13:58
magicaltrouti don't think that has been updated BlackDex13:58
magicaltroutso I think that bug is still valid13:58
BlackDexman, i think i'm looking in the wrong charm now :p14:00
BlackDexi need to check the nagios charm, and not the nrpe14:00
magicaltroutyup14:01
BlackDexdoh14:01
magicaltroutthats why I just hacked the code :)14:01
BlackDexlets see if that is the case for the xenial version also14:01
BlackDexthat makes a bit more sense14:01
BlackDexoke14:03
BlackDexlets see what that does :)14:03
alexliststokachu: ok, with the snap versions everything works as it should14:10
stokachualexlist, great! thanks for testing14:11
BlackDexmagicaltrout: Thx for clearup, i now installed the latest nrpe charm with the manual patched nagios, that seems to work14:19
magicaltroutno problem BlackDex14:33
magicaltroutin other news.... someone just brought a parrot into our kitchen at work....14:33
BlackDexparrot wants a cookie14:38
lazyPowerZic  :D14:53
ZiclazyPower: the production cluster is all fine, I have just one little trouble: the kube-dns pod restart sometime with no reason other than 21h    3m    24    {kubelet mth-k8svitess-02}    spec.containers{dnsmasq}    Warning    Unhealthy    Liveness probe failed: HTTP probe failed with statuscode: 50314:57
ZicI mitigated this with scaling the kube-dns deployment to 5 replicas instead of 114:57
Zic(it keeps restarting sometime, ~1 every two hours, but at least, they are other kube-dns pod which not restarting in the same time that can handle request)14:58
magicaltroutwhen in doubt.... replicate!14:58
lazyPowerZic - fantastic, we were discussing on if we should scale that... i think there's a bug for this actually14:58
Zicmagicaltrout: already done :p14:58
lazyPowerZic - https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/18114:59
Zicthx14:59
lazyPowernot 1:1 the same, but if you coudl add your comments there it would add some weight14:59
lazyPowerand we can probably get that scheduled for post 1.6 release14:59
Zicother than that, all is fine, I'm running on 1 master for now but I will switch to nominal-3 when you're patch will be officially released regarding my bug :)14:59
lazyPowerZic - well i think we want to use teh autoscaler addon tbh15:01
lazyPoweras its using metrics to drive the scale15:01
lazyPowersurely it by defalt wants HA15:01
lazyPower*default15:01
Zics/you're/your/15:02
magicaltroutIm pushing for CDK to be used on a Darpa project here lazyPower, dunno if it'll win, but i'm tryin'15:02
lazyPowermagicaltrout - <3 <3 <315:02
lazyPowermagicaltrout - let me know if there's anything we can do to help support this effort15:02
magicaltroutslap some sysadmins?15:03
ZicI have just a landing page with a logo publicly available through this K8s cluster :D15:03
lazyPowerZic - no fancy mobile game servers? ;)15:03
Zicthe real start is planed for the end of the month15:03
ZiclazyPower: it's about video streaming with paid access :)15:03
Zic(no, it's not p*rn !!)15:04
magicaltrouti'll have to get SaMnCo 's GPU stuff in there we've got like 100+ GPUS at launch15:04
magicaltroutZic: you're streaming porn with CDK?!!! ;_15:04
Zicnope :p15:04
SaMnComagicaltrout: yooohoooo15:04
* magicaltrout tweets Zic 's revelation....15:05
SaMnCoWhat GPU?15:05
lazyPowerZic - schenanigans, sounds like p*rn to me15:06
magicaltrouti think most are titan X's SaMnCo but there are a range of different ones being brought in from other projects15:10
magicaltroutplus we got told off by Nvidia for using off the shelf hardware instead of the stupidly marked up "data processing GPUs"15:10
magicaltrout"how dare you use our chips for anything other than Games without us apply a 100% markup!"15:11
lazyPowerO_o15:11
lazyPowerobvious marketing effort is obvious15:11
magicaltroutyup15:11
ZiclazyPower: when I wrote "it's about video streaming with paid access" I immediately realized that it will be acknowledged as p*rn :p15:12
Zicso I precised it :D15:12
lazyPowerZic - or netflix, people like to conflat the two15:12
magicaltrouti like the fact no one can write porn15:12
Zicmagicaltrout: don't know if it's a banned word :p15:12
magicaltroutwell lazyPower has yet to tell me off15:12
Zicseems there is no bots in this chan15:12
lazyPoweri think its a fine word in the current context15:12
magicaltroutso i assume its acceptable, we are mostly adults after all :)15:12
jrwrenI'm going to imagine it is sports.15:12
magicaltrouthehe15:13
lazyPowerif you were like explicit i might have you remind you and i of the Code of Conduct15:13
Zic:D15:13
lazyPowerhowever, we're all being sensible15:13
magicaltrouthttp://www.darpa.mil/program/data-driven-discovery-of-models this is what we're working on lazyPower15:13
magicaltrout4 year programme about discovering data15:13
lazyPowermagicaltrout - wow this is a complex etl stack, its not really fully decomposed15:13
lazyPowerthis is the 10k foot diagram right?15:13
Zicplus, I talked about lazyPower's body some days ago, so I'm limiting myself about prohibited word :D15:13
* Zic left15:13
lazyPoweroh my15:13
lazyPoweryou had to bring that back up didnt you15:14
lazyPowerhawkwarddddd15:14
Zic:p15:14
magicaltrouthttp://www.darpa.mil/news-events/2016-06-1715:14
lazyPoweryeahhh ml for ml15:14
lazyPoweryour recursion is neat :D15:14
magicaltroutyeah lazyPower lots of crazy GPU powered machine learning to run over public datasets to try and automatically detect the models without users having to write the code15:14
lazyPowerand phase 1 of skynet will have been delivered15:15
lazyPowerthe fact CDK might possibly be empowering skynet, is kinda neat15:15
magicaltroutZic: https://irclogs.ubuntu.com/2017/02/16/%23juju.html you mean this publically accessible log ? ;)15:15
ZiclazyPower: I kept the meme's image you gave me as goodies :p15:16
magicaltroutZic 's love of lazyPower is forever preserved15:16
ZicxD15:16
lazyPower<315:16
Zicit's why I gave the Juju OpenStack project to one of my colleague, I only want to use Juju if there is "lazyPower parts"15:20
magicaltrouti wouldn't recommend that as a life choice ;)15:20
lazyPower^15:20
Zicfun fact: Canonical commercial's support answered us with a "About your OpenStack Kubernetes stack"15:20
ZicI only mentioned Kubernetes in my contact mail...15:20
Zic:>15:20
Zic(we're going to buy commercial's support at term, when it will really go prod)15:21
mbruzekawesome Zic15:22
=== tvansteenburgh1 is now known as tvansteenburgh
lazyPowerZic - i appreciate your contributions to my pizza budget15:25
Zic:D15:26
magicaltroutthat is the largest budget in Canonical15:26
magicaltroutmore than Marks private jet costs...15:27
Zicthe next Ubucon EU is at Paris15:27
Ziciirc15:27
Zicif you want your pizzas :p15:27
magicaltroutsod ubucon15:27
lazyPoweri'm a firm beleiver in paying it forward15:27
magicaltroutget lazyPower to sponsor you to juju charmer summit15:27
Zichttp://ubucon.org/en/15:27
lazyPowerZic - if you can enrich someone elses life with a pizza, pay it forward to them and i'll be happy15:27
Zic:D15:28
ZiclazyPower came to EU one time, he feared now15:28
Zic(because of me)15:28
magicaltroutna he got taken away by a weird british guy and his belgian friend.... he's not been the same since15:29
lazyPower^ true story15:29
lazyPowerWWI re-enactment actors15:29
lazyPoweri got a heck of a history lesson that night magicaltrout15:29
magicaltroutthats what you tell us lazyPower15:31
magicaltroutno one else was there to witness it15:31
lazyPowera gentleman never tells15:31
ZicxD15:31
lazyPowerbut what happened does *not* rhyme with zic's business model15:32
magicaltrouthehe15:32
SaMnCoZix who is your sales rep?N15:50
SaMnCo@Zic you is your sales rep?15:51
SaMnComissed the key :/15:51
SaMnCo@magicaltrout : so it is bare metal stuff?15:51
magicaltroutSaMnCo: we've got some baremetal stuff some openstack stuff15:52
magicaltroutdepends where stuff gets deployed15:52
ZicSaMnCo: we don't have any contact for now, we're just preparing it, but our contact is Mac Belonwu15:56
Zic(I don't answer him for now, as I need to do some COMEX discussion at our company before :/)15:57
ZicSaMnCo: or maybe I misunderstand your question: nope, I'm just a sysadmin at our company :)15:58
SaMnCoOK. I usually cover EU for pre-sales stuff around k8s, so any question or issue don't hesitate to involve me15:58
magicaltroutSaMnCo is trying to steal commission ! ;)16:00
SaMnCoahahah :D16:00
SaMnCoyou know sales / pre-sales, it's like Jehovah people16:00
SaMnCoalways go by 216:00
Zicwe're Paris-based, so maybe you're more involved in my request than your colleague16:01
magicaltroutthat then try and extor money from you with tales of woe! you're correct its identical16:01
magicaltroutsilly keyboard -that s/extor/extort16:02
SaMnCoZic: no no, I just wanted to understand where you were in the process to see if there was a need for tech support on your end. I contacted Mac, so we're all set.16:06
ZicSaMnCo: we just sent a request via ubuntu.com for now, we didn't really (he just had our sales rep on the phone) answer to him as we don't have all the elements16:18
SaMnCook16:41
=== plars_ is now known as plars
CynervaHey folks, if I git clone juju and run `snapcraft`, does it build from the local repo?17:29
CynervaI want to try something that's in the 2.1 branch but not in a release candidate yet17:29
CynervaI'm not familiar with golang or the juju repo, so I just want to make sure the resulting snap has whatever's in the tree :)17:31
=== externalreality_ is now known as externalreality
=== frankban is now known as frankban|afk
neiljerramIs there a way that I can have all units in a bundle deployed to machines in the same GCE zone?18:24
marcoceppineiljerram: you can, but it's typically not advised18:31
neiljerrammarcoceppi, I'm wondering if it might help with a problem I'm seeing with 'juju ssh'18:31
lazyPowerneiljerram - i can confirm i've been experiencing networking issues in google/us-central1 region18:32
lazyPowerand it started this morning, it was fine lastnight.18:32
=== Guest21765 is now known as tbc
neiljerramlazyPower, yes, that's where I've been seeing issues too.18:32
lazyPowerneiljerram - i moved to us-east1, its a bit slower, but it doesn't have the same connectivity issues18:33
neiljerramlazyPower, But I believe my issues are much more longstanding than just the last day or two.18:33
lazyPowerand more to the point of being obnoxious, its intermittant18:33
neiljerramWhat is the symptom that you see?18:34
lazyPowerneiljerram - i have issues connecting between untis in different az's and i have som external connectivity failures, specifically with their apt mirror.18:35
lazyPoweri had an etcd cluster tank during testing because AZ-a wasn't able to talk to AZ-c for whatever reason18:35
neiljerramInteresting.  The thing I notice first, in my case, is 'juju ssh ...' failing.  But it could also be that there are connectivity issues between the deployed units.18:36
lazyPowerthat would be consistent actually if your controller is in a different AZ than your unit18:37
lazyPoweri do believe that default behavior is to proxy through the controller to establish ssh, but that is configurable18:37
neiljerramAh, that sounds good, where is the switch for that?18:38
lazyPowerproxy-ssh                   default  false18:38
lazyPoweri see its defaulted to false here, so looks like i may be wrong18:38
lazyPowerneiljerram - fyi - juju model-config  or juju model-defaults18:39
neiljerramI have false as well, already - so guess that's good.18:40
neiljerramSo for getting all unit machines in the same zone, I think I just discovered the method for that:18:41
neiljerramfor n in `seq 1 10`; do juju add-machine zone=us-central1-c; done18:41
neiljerramBut I still have my controller in a different zone (us-central1-a)...18:42
neiljerramI guess that setting the zone for the controller would need to be something on the 'juju bootstrap# invocation.  Any ideas?18:43
lazyPowerneiljerram - there are bootstrap-constraints18:45
lazyPowerjuju bootstrap --help has an overview18:45
neiljerramI already had that help in my terminal, but hadn't seen --bootstrap-constraints.18:48
neiljerramSo would it be: juju bootstrap google/us-central1 bundle --config image-stream=daily --bootstrap-constraints zone=us-central1-c18:50
neiljerram(Just waiting for my existing controller to die so I can try myself.)18:50
neiljerramNo, ERROR unknown constraint "zone"18:51
lazyPowerhmmm19:02
lazyPoweri would have thought that would have mirrored the constraints you can pass to --constraints19:03
lazyPoweri admittedly have not attempted to pass a zone constraint on those constraints.19:03
neiljerramAh, it's --to instead of --bootstrap-constraints.19:09
neiljerram(Discovered from code reading!)19:09
neiljerramSo: juju bootstrap google/us-central1 bundle2 --config image-stream=daily --to zone=us-central1-c19:09
lazyPoweroh neat19:12
lazyPower#TIL19:12
=== mup_ is now known as mup
freyeshi marcoceppi , I noticed that ceph-proxy doesn't exist under https://bugs.launchpad.net/charms/ , so it's not possible to file bug against it ( https://jujucharms.com/ceph-proxy/xenial/0 ), could you it? or do you know who could do it?20:37
stormmorefinally gotten around to installing an IRC client20:44
stormmoreo/ juju world!20:44
=== externalreality_ is now known as externalreality
marcoceppio/ stormmore welcome back :)20:54
stormmorehate doing workstation reinstall but sometimes ya gotta do what ya do!20:59
lazyPowerstormmore - :) :) w21:00
lazyPower*wb21:00
=== Budgie^Smore is now known as stormmore
=== frankban|afk is now known as frankban
=== externalreality1 is now known as externalreality
Budgie^Smorehmm this is weird22:45
=== Budgie^Smore is now known as stormmore
=== Budgie^Smore is now known as stormmore
stormmoreOK this is anonying, didn't use to get ping timeouts :-/23:21

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!