| hazmat | what's the current dev milestone? 1.17.0 or 1.17.1 | 01:04 |
|---|---|---|
| axw | hazmat: 1.17.0 is not released yet AFAIK | 01:07 |
| axw | hmm | 01:07 |
| hazmat | axw, i see some commits landing for bugs against 1.17.1 its a bit unclear | 01:08 |
| axw | yeah | 01:08 |
| axw | better ask sinzui | 01:08 |
| axw | there's been no announcement, but I'm a bit confused why some 1.17.0 bugs are released | 01:08 |
| hazmat | most of those are external to the src | 01:09 |
| axw | wallyworld_: what are you up to? am I meant to be testing the entire backup/restore procedure again? | 01:10 |
| wallyworld_ | axw: hey, give me a minute and i'll let you know what we need to do today, just finishing a doc | 01:11 |
| axw | ok | 01:11 |
| bigjools | what does it mean when I get a "no CA certificate in environment configuration" ? | 01:20 |
| bigjools | I tried to copy an environment from another machine but I guess something is missing | 01:20 |
| hazmat | bigjools, you need to copy the jenv file the environments.yaml config isn't useful byitself (or even needed if you have the jenv) | 01:25 |
| bigjools | hazmat: I did | 01:25 |
| hazmat | bigjools, that's strange the jenv file has the certs | 01:25 |
| wallyworld_ | bigjools: did you copy the jenv file as well? | 01:25 |
| wallyworld_ | bah, too late | 01:26 |
| bigjools | yeah I see the ca cert in the jenv file | 01:26 |
| bigjools | oh hmmm I think I see a problem. I installed from the PPA but 1.13 is still in the path | 01:27 |
| bigjools | dafuq | 01:27 |
| bigjools | so, rm /usr/bin/juju and reinstalling the juju-core package fixed it. It's a package bug. | 01:29 |
| wallyworld_ | axw: hey, you see my email? i have 2 branches to land in 1.16. one is just about to finish in the bot, the other should start straight after | 01:37 |
| wallyworld_ | and then we can test | 01:37 |
| wallyworld_ | i'll test on ec2 | 01:37 |
| axw | wallyworld_: my net connection keeps dropping in and out, so maybe drop me an email in case I miss (or have missed) instructions | 01:39 |
| wallyworld_ | <wallyworld_> axw: hey, you see my email? i have 2 branches to land in 1.16. one is just about to finish in the bot, the other should start straight after | 01:39 |
| wallyworld_ | <wallyworld_> and then we can test | 01:39 |
| wallyworld_ | <wallyworld_> i'll test on ec2 | 01:39 |
| wallyworld_ | last branch in bot now | 01:40 |
| axw | ok | 01:40 |
| axw | it makes most sense for me to test on MAAS, but I don't like my chances with my shitting connection today | 01:40 |
| axw | shitty* | 01:40 |
| axw | wallyworld_: juju-core/1.16? | 01:41 |
| wallyworld_ | yeah | 01:41 |
| wallyworld_ | the set-env fix is landing | 01:41 |
| wallyworld_ | safe mode support just landed | 01:41 |
| axw | wallyworld_: are you testing the "fallback solution" on EC2? | 01:45 |
| wallyworld_ | yeah | 01:45 |
| bigjools | is it possible to add a new service unit with a different config to an existing one? | 01:46 |
| wallyworld_ | axw: do the additions to the document make sense? | 01:46 |
| axw | wallyworld_: the safe mode bit? | 01:46 |
| wallyworld_ | yeah | 01:46 |
| axw | yep | 01:46 |
| wallyworld_ | cool, hopefully will also make sense to folks on site | 01:46 |
| bigjools | if not, how do I deploy a service again with a different config? | 01:46 |
| wallyworld_ | bigjools: what sort of config? | 01:47 |
| bigjools | I have a tarmac charm, and I am deploying again with a different charm config for a different branch to land | 01:47 |
| bigjools | and if I try "deploy" again I get "ERROR cannot add service "tarmac": service already exists" | 01:47 |
| wallyworld_ | hmmm. not sure. i know you can add a service and give it a different name | 01:48 |
| bigjools | ok let's try that | 01:48 |
| bigjools | argh | 01:49 |
| bigjools | affects the charm config | 01:49 |
| wallyworld_ | :-( | 01:50 |
| wallyworld_ | i'm not sure. maybe davecheney knows? | 01:50 |
| davecheney | sup! | 01:50 |
| wallyworld_ | davecheney knows everrryyythiiiing | 01:50 |
| davecheney | davecheney !knows everything | 01:50 |
| wallyworld_ | bigjools has a question | 01:50 |
| wallyworld_ | see scrollback just above | 01:51 |
| davecheney | bigjools: juju deploy tarmac tarmac2 | 01:51 |
| bigjools | heh | 01:51 |
| davecheney | by default, fi you don't give a service name, we take the name from the charm | 01:51 |
| bigjools | davecheney: yeah I did that, had to change the charm config of course | 01:51 |
| bigjools | thanks both | 01:51 |
| davecheney | kk | 01:51 |
| wallyworld_ | bigjools: that's what i told you to try isn't it? | 01:51 |
| bigjools | wallyworld_: it is, and I just thanked you. | 01:51 |
| wallyworld_ | ah, np. i thought you said it didn't work | 01:52 |
| wallyworld_ | sorry, comprehension problm then | 01:52 |
| bigjools | you were leaping like a salmon to that conclusion :) | 01:52 |
| wallyworld_ | swish swish goes my tail | 01:52 |
| wallyworld_ | axw: ok, so both branches landed | 01:53 |
| axw | wallyworld_: cool, thanks, I'll get testing | 01:53 |
| wallyworld_ | me too | 01:54 |
| wallyworld_ | axw: how's it going? | 02:58 |
| axw | wallyworld_: doing the restore now | 02:58 |
| wallyworld_ | seems to work on ec2 | 02:58 |
| axw | nearly there | 02:58 |
| wallyworld_ | i tweak the doc a little | 02:58 |
| axw | wallyworld_: which bit? | 03:00 |
| wallyworld_ | here and there eg i added " around <instanceId> in the update db step | 03:01 |
| wallyworld_ | also, the restart rsyslog command at the end was messed up a bit (formatting) | 03:01 |
| axw | ok cool | 03:02 |
| wallyworld_ | axw: i'll send the email now since we have run out of time. but it looks like everything is ok | 03:12 |
| axw | wallyworld_: I've just fully restored | 03:13 |
| axw | safe-mode works | 03:13 |
| wallyworld_ | yay | 03:13 |
| axw | I'm just going to confirm that turning safe-mode off destroys the original instance, but htat's bonus points I think | 03:13 |
| wallyworld_ | ok, let me know | 03:13 |
| wallyworld_ | i forgot that bit, i'll try too | 03:13 |
| axw | wallyworld_: yep, did the trick | 03:15 |
| wallyworld_ | /o/ | 03:15 |
| axw | I'm surprised it immediately destroyed the instance tho? | 03:16 |
| wallyworld_ | \o/ even | 03:16 |
| wallyworld_ | yeah | 03:16 |
| wallyworld_ | it will trigger straight away | 03:16 |
| axw | ok nps, I thought it would only happen when you add/remove a machine | 03:16 |
| wallyworld_ | we wanted to not do that so added a hook | 03:16 |
| wallyworld_ | s/hook/trigger | 03:16 |
| wallyworld_ | whatever | 03:16 |
| wallyworld_ | :-) | 03:16 |
| axw | yep it's better than what I expected :) | 03:19 |
| jam | wallyworld_: if you're poking at the doc, try to make sure it is clear that "<foo>" is intended to be replaced with a real value | 04:57 |
| jam | rather than the actual text | 04:57 |
| wallyworld_ | sure. i thought that wouold be obvious | 04:57 |
| jam | wallyworld_: well, there are a lot of potential syntaxes, and these are people who know nothing about the line you're having them cut&paste | 04:57 |
| jam | so while it is a little obvious, I've tried to be *very* explicit | 04:58 |
| wallyworld_ | ok | 04:58 |
| wallyworld_ | actualy, the doc already says it | 04:58 |
| jam | if it is <newInstanceId> that is bits I added | 04:59 |
| axw | maybe change them to a different colour | 04:59 |
| jam | I see at least <name> and <admin-secret> that don't explicitly state they are variables to be replaced | 05:00 |
| jam | and <name> mabye not | 05:00 |
| jam | since it is just descriptive | 05:00 |
| wallyworld_ | well we should probably have a glossary then | 05:00 |
| wallyworld_ | eg <foo> means replace with actual value etc | 05:01 |
| wallyworld_ | that sort of thing should be done by tech write when the doc is productised | 05:01 |
| wallyworld_ | bbiab, school pickup time | 05:02 |
| axw | wallyworld_: https://streams.canonical.com/ just has 1.17.0 (which isn't released??) - am I looking in the wrong place? | 05:56 |
| axw | also, I thought it was /juju | 05:56 |
| wallyworld_ | axw: that was a test copy | 06:12 |
| wallyworld_ | it needs to be deleted | 06:13 |
| axw | wallyworld_: ok. | 06:13 |
| wallyworld_ | and yes, it's in the wrong directory :-) | 06:13 |
| axw | no worries, I guess that'll get sorted when 1.17 is released | 06:13 |
| wallyworld_ | yep, curtis is all over it | 06:14 |
| jam | wallyworld_: axw: from what he said yesterday, they're going to do a test release of 1.16.3 into streams.canonical.com, which should fix all that up, and then do 1.17.0 from there. | 06:26 |
| axw | jam: okey dokey | 06:27 |
| wallyworld_ | yeah, that's my undertsanding too | 06:27 |
| wallyworld_ | the 1.17 stuff that's there was from when we tested in SFO | 06:27 |
| wallyworld_ | to make sure the signing worked etc, when ben was aroud to do it for us | 06:28 |
| jam | hazmat: for 1.17.1 vs 1.17.0, we were going to release 1.17.0 last week, so we retargeted bugs, and made important bugs targeted to 1.17.1. However, since the release got delayed, we kept doing stuff. I retargeted the ones that landed back to 1.17.0 | 08:16 |
| rogpeppe | fwereade: well done for spotting the getBroker omission | 08:27 |
| mgz | morning! | 09:02 |
| jam | mgz: morning. I have to run to the grocery store, I'll try to be back in time for the team meeting, but don't hold it up for me. | 09:19 |
| mgz | jam: sure | 09:20 |
| === racedo` is now known as racedo | ||
| jam | fwereade: TheMue, fwereade, rogpeppe, dimitern: weekly team meeting ? | 10:02 |
| jam | https://plus.google.com/hangouts/_/calendar/bWFyay5yYW1tLWNocmlzdGVuc2VuQGNhbm9uaWNhbC5jb20.09gvki7lhmlucq76s2d0lns804?authuser=1 | 10:02 |
| rogpeppe | jam: ok, will leave the bridge hangout | 10:03 |
| jam | mramm: if you want to join us ^ | 10:03 |
| * dimitern lunch | 10:57 | |
| wallyworld_ | jam: i've updated the backup scripts branch if you get a chance to look. as well as doing the extra logs, it also restarts mongo if the dump fails | 10:57 |
| wallyworld_ | it also extracts environ config as json | 10:58 |
| jam | wallyworld_: just to make sure I understand, that script isn't intended for the NEC issue, but *is* useful as a general "backup your juju state server", right? | 10:59 |
| wallyworld_ | jam: yeah, i wasn't even going to commit to 1.16 branch | 11:00 |
| wallyworld_ | but we (fwereade and me) decided it is useful to have | 11:00 |
| wallyworld_ | and since you had commented on it previously, i thought i'd ask :-) | 11:00 |
| jam | mgz: so I think you should probably put together a patch to the juju-update-bootstrap plugin that allows dying things to be updated, you can even propose it, but we should pause for landing it until we sort out the release side of things. | 11:01 |
| jam | wallyworld_: I actually have it open in one of my tabs, I'll give it a loo | 11:01 |
| jam | look | 11:01 |
| wallyworld_ | thank :-) | 11:01 |
| mgz | jam: I have it | 11:01 |
| jam | wallyworld_: is there a reason we are backing up /etc/init/juju-machine-*.conf, but not a juju-unit-*.conf ? (there may not be one, I thought the unit agents were also run via upstart, but I could be off there) | 11:02 |
| wallyworld_ | jam: i think it was rules out as being needed, maybe by tim. not sure who said not to | 11:04 |
| wallyworld_ | i can easily add it | 11:05 |
| wallyworld_ | ah i think maybe we said we would handle units running on state server | 11:05 |
| jam | wallyworld_: so it doesn't help you back up your *state-server* (as that is never a unit) but it does back up *that machine* | 11:05 |
| wallyworld_ | because <some reason a can't recall> | 11:05 |
| wallyworld_ | s/would/wouldn't above | 11:06 |
| wallyworld_ | jam: my brain is fading, but i think if one considers what we do now with the manual process and a bucula backup, we sorta want to have enough info in the backup tar ball to restore a state server | 11:07 |
| wallyworld_ | assuming no other backup solution was used | 11:07 |
| wallyworld_ | and not supporting units | 11:08 |
| wallyworld_ | ah i recall now | 11:08 |
| jam | wallyworld_: sure. I think my point is if you did somethnig like deploy juju-gui to node 0 (which is what juju-quickstart does) then it won't come back up after you've restored your state server from a failure | 11:08 |
| wallyworld_ | it's no good backup up juju-unit-*.conf since we wil not have charms when we restore | 11:08 |
| wallyworld_ | yes you are right. i think because we cannot guarantee charm will be backed up, there's no point to restoring unit conf | 11:09 |
| jam | wallyworld_: k, "reasons". We may want to document it in a comment in the script, and we may come back to that later. But what you have is fine for me. Certainly as a "better than nothing, and we can iterate if we want to make things better" | 11:09 |
| wallyworld_ | yes, we will need to iterate for sure | 11:09 |
| wallyworld_ | but it's a good start | 11:10 |
| wallyworld_ | i'll add a comment | 11:10 |
| wallyworld_ | but will land tomorrow if you want to +1 before you eod | 11:11 |
| mgz | wallyworld_: remind me at some point to ask you to explain to me the tests you've got for the juju-metadata plugin | 11:12 |
| wallyworld_ | ok | 11:13 |
| wallyworld_ | which ones? i may not have a suitable answer right now | 11:14 |
| mgz | wallyworld_: particularly the magic in metadataplugin_test.go | 11:14 |
| wallyworld_ | mgz: it basically ensures that each metadata sub-command is properly registered | 11:18 |
| mgz | what's the exec doing in badrun... | 11:18 |
| mgz | wallyworld_: unrelated, did the packaging need updating when you added juju-bootstrap as a plugin? | 11:18 |
| wallyworld_ | mgz: hmmmm. i *think* it is smart enough to look for binaries called juju-* but am not 100% sure | 11:20 |
| jam | wallyworld_: I already LGTM'd | 11:20 |
| wallyworld_ | ok, thanks. | 11:21 |
| mgz | TheMue: can you please push lp:~gz/juju-core/devel-packaging over lp:~juju-qa/juju-core/devel-packaging | 12:33 |
| TheMue | mgz: eh, sure, only that I never have done that before :/ | 12:35 |
| TheMue | mgz: but I think you can help me here | 12:35 |
| rogpeppe | wallyworld_: I've just reviewed https://codereview.appspot.com/31960043/ | 12:40 |
| TheMue | mgz: is it branch qa/...; merge gz/...; push qa/... ? | 12:40 |
| wallyworld_ | thanks.' | 12:41 |
| wallyworld_ | rogpeppe: one quick comment - i don't like embedding scripts - harder to debug etc. so long as both scripts are in same directory it will just work | 12:41 |
| * TheMue would like to ask Curtis, but he will have stuffed turkey today :) | 12:41 | |
| rogpeppe | wallyworld_: why is it harder to debug? | 12:42 |
| wallyworld_ | cause you can't run the script stand alone | 12:42 |
| wallyworld_ | and syntax highlighting tc | 12:42 |
| TheMue | wallyworld_: +1 | 12:42 |
| wallyworld_ | and ide support | 12:42 |
| rogpeppe | wallyworld_: if it's a shell function, syntax highlighting should work fine | 12:42 |
| rogpeppe | wallyworld_: and it's trivial to run it standalone too | 12:43 |
| wallyworld_ | it is? | 12:43 |
| wallyworld_ | i thought jam had already +1 it actually | 12:43 |
| wallyworld_ | ah he did | 12:43 |
| rogpeppe | wallyworld_: yeah, just put remote_cmd "$@" just after the function definition | 12:43 |
| wallyworld_ | in the lp mp | 12:43 |
| rogpeppe | wallyworld_: ah, i didn't see his reply | 12:44 |
| wallyworld_ | i'll look at the $@ syntax, not familiar eith htat | 12:44 |
| wallyworld_ | bah my typing sucks | 12:45 |
| rogpeppe | wallyworld_: if you write shell scripts and don't know about "$@" you are inevitably writing buggy scripts, i'm afraid | 12:45 |
| wallyworld_ | it's a simple script | 12:45 |
| rogpeppe | wallyworld_: it's the *only* way to allow white space in arguments | 12:45 |
| wallyworld_ | lucky there's none required | 12:45 |
| rogpeppe | wallyworld_: still, it's best to be defensive | 12:46 |
| wallyworld_ | neither script takes args | 12:46 |
| rogpeppe | wallyworld_: in which case, just "remote_cmd" would do | 12:46 |
| rogpeppe | wallyworld_: for the record, "$@" (quotes necessary) is just the same as $* except that arguments with spaces in are kept as single arguments | 12:47 |
| wallyworld_ | ok | 12:47 |
| wallyworld_ | which function definition are you suggesting i put remote_cmd after ? | 12:49 |
| wallyworld_ | ah never mind, i see it in your comments | 12:51 |
| rogpeppe | wallyworld_: if you define remote_cmd as the first thing in juju-backup, then to test it, just put (just after) remote_cmd; exit $? | 12:51 |
| rogpeppe | wallyworld_: then you can test the shell function without the rest of the behaviour | 12:51 |
| wallyworld_ | rogpeppe: btw, those snippets for std err capture and working dir came fromstack exchange | 12:52 |
| rogpeppe | wallyworld_: FYI here's an illustration of the difference between "$@" and $* http://paste.ubuntu.com/6489365/ | 12:53 |
| wallyworld_ | people seemed to recommend those snippets from what i could see | 12:53 |
| rogpeppe | wallyworld_: it seems too complex - there are already quite a few levels of evaluation in sh scripts and $( eval 'sudo -n bash -c "(command)") doesn't fill me with delight | 12:55 |
| wallyworld_ | that last bit was to get the command chaining correct | 12:55 |
| rogpeppe | wallyworld_: sorry, i don't understand that | 12:56 |
| wallyworld_ | so that juju db would restart after any failure | 12:56 |
| rogpeppe | wallyworld_: i made an alternative suggestion for that - did you see that? | 12:56 |
| wallyworld_ | not yet | 12:56 |
| wallyworld_ | i'll have a proper look tomorrow, very tired now | 12:57 |
| rogpeppe | wallyworld_: np :- | 12:57 |
| rogpeppe | :-) | 12:57 |
| mgz | TheMue: you can litterall just `bzr push -d lp:~gz/juju-core/devel-packaging lp:~juju-qa/juju-core/devel-packaging` | 12:57 |
| wallyworld_ | thans for the review though, i'll look properly tomorrow | 12:57 |
| TheMue | mgz: that sound easy ;) | 13:01 |
| TheMue | mgz: could it be that the order is wrong? I get a lock error for your branch (readonly transport) | 13:05 |
| TheMue | mgz: ah, no, it's the one to pull from, so it's correct. hmmm ... | 13:12 |
| * dimitern is not feeling very well, so I'll lie down for a while | 13:13 | |
| === dimitern is now known as dimitern_afk | ||
| mgz | dimitern: hope you feel better soon | 13:13 |
| dimitern_afk | mgz, thanks | 13:13 |
| mgz | TheMue: you are in ~juju-qa so you should be able to push there if you have the same ssh key as in your launchpad account enabled | 13:14 |
| TheMue | mgz: the key is the same, but the error message mentions your branch | 13:15 |
| TheMue | mgz: bzr: ERROR: Cannot lock LockDir(chroot-83837968:///~gz/juju-core/devel-packaging/.bzr/branch/lock): Transport operation not possible: readonly transport | 13:16 |
| mgz | TheMue: just pull that branch down then, and push from local | 13:17 |
| rogpeppe | rebooting 'cos my graphics card has gone tits up again | 13:23 |
| TheMue | mgz: system says it's happy | 13:29 |
| TheMue | mgz: only wondering where I see in lp that I've just pushed it | 13:30 |
| mgz | TheMue: https://code.launchpad.net/~juju-qa/juju-core/devel-packaging | 13:32 |
| TheMue | mgz: that's where I looked | 13:34 |
| mgz | you should see the top revision is from me | 13:35 |
| TheMue | mgz: the 19? | 13:35 |
| mgz | TheMue: yup. ah, I see what's confusing you, I used --author so it doesn't say my name in the UI | 13:36 |
| TheMue | mgz: ah, found it, not the displayed name but the commiter | 13:36 |
| TheMue | mgz: exactly | 13:36 |
| * TheMue and the wonderful world of bazaar and launchpad | 13:37 | |
| rogpeppe | fwereade: have you tried mongorestore at all? Pierre is having some problems following the restore procedure here (can't connect to the database) and i'm wondering if there's something we've got wrong. | 13:52 |
| fwereade | rogpeppe, sorry, I'll pop in | 13:54 |
| jam | fwereade: rogpeppe: just passing by after dinner, is everything sorted out ? | 14:44 |
| rogpeppe | jam: nope, not really | 14:45 |
| fwereade | jam, caribou's reports with the build are encouraging, melmoth has had a bit more trouble -- the instance id definition wasunclear | 14:45 |
| jam | so... "juju destroy-environment local" seems to succeed but then "juju status -e local" is working... | 14:52 |
| jam | "juju destroy-environment --debug" claims to "removing service juju-db-jameinel-local, but it is still present in /etc/init | 14:53 |
| jam | well 'sudo juju destroy-environment' | 14:54 |
| jam | ah, I changed the name, that's probably why. strange it didn't *complain* that it couldn't remove the service, but it does complain that it can't remove /etc/rsyslog.d | 14:55 |
| rogpeppe | i'm stopping for lunch, and then i'm going to travelling for an hour or so, so will be incommunicado. will be back for a little while after then. | 15:33 |
| rogpeppe | fwereade: right, back online no | 17:28 |
| rogpeppe | w | 17:28 |
| fwereade | rogpeppe, heyhey | 17:29 |
| fwereade | rogpeppe, turns out the fricking provisioner pays no attention to the machine's address change if it happens after the provisioner starts | 17:30 |
| rogpeppe | fwereade: you mean the address updater? | 17:31 |
| fwereade | rogpeppe, yeah, the updater works but the provisioner doesn't care | 17:31 |
| fwereade | rogpeppe, the auth field is set up just once | 17:31 |
| fwereade | rogpeppe, it was meant to be consulted once per group of machines | 17:31 |
| fwereade | rogpeppe, hey ho | 17:32 |
| rogpeppe | fwereade: the auth field? | 17:32 |
| fwereade | rogpeppe, it's got a SetupAuthentication that returns state+api info for the new machine | 17:32 |
| fwereade | rogpeppe, including the addresses | 17:32 |
| fwereade | rogpeppe, which are looked up *once* when we start the provisioner | 17:32 |
| rogpeppe | fwereade: oh ffs | 17:32 |
| fwereade | rogpeppe, literally the only reason for startMachines was so that we could get up to date addresses | 17:33 |
| fwereade | rogpeppe, without that it's just a dumb loop | 17:33 |
| * fwereade goes to smoke a grumpy cigarette, brb | 17:33 | |
| rogpeppe | fwereade: BTW here's a the sketch for a restore command: http://paste.ubuntu.com/6490492/ | 17:34 |
| fwereade | rogpeppe, that looks good apart from the need to do the fresh bootstrap in safe mode | 17:44 |
| rogpeppe | fwereade: what do you suggest instead? | 17:45 |
| fwereade | rogpeppe, I'm not suggesting it's a bad approach, just a detail that's not explicitly addressed | 17:45 |
| rogpeppe | fwereade: i'm not sure i understand | 17:47 |
| fwereade | rogpeppe, well, if we bootstrap with a functioning provisioner, it'll take down all the existing nodes, right? | 17:47 |
| rogpeppe | fwereade: yes | 17:48 |
| fwereade | rogpeppe, so the re-bootstrap needs to either run in safe mode, or with the agent disabled -- am I just failing reading comprehension? | 17:48 |
| rogpeppe | fwereade: my sketch includes bootstrapping in safe mode, doesn't it? | 17:48 |
| rogpeppe | fwereade: i thought you were objecting to that, for some reason | 17:49 |
| fwereade | rogpeppe, yes, I'm just failing reading comprehension, sorry | 17:49 |
| rogpeppe | s/,// | 17:49 |
| rogpeppe | fwereade: ok, cool | 17:50 |
| rogpeppe | fwereade: sorry, i have to go, supper is on the table. i hope that's ok. | 18:20 |
| rogpeppe | g'night all | 18:22 |
| === gary_poster is now known as gary_poster|away | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!