| mup | PR snapd#5242 opened: tests: new test for joystick interface <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5242> | 01:25 |
|---|---|---|
| jamesh | zyga: if you wanted to test out the portals code you've been reviewing for me over the last several months, I put together some test instructions here: https://forum.snapcraft.io/t/desktop-portal-testing-notes-for-app-developers/5711 | 08:08 |
| zyga | brilliant, thank you | 08:08 |
| zyga | is mvo around today? | 08:33 |
| pedronis | no, still off afaiu | 08:37 |
| zyga | ah, ok | 08:37 |
| pedronis | mmh, the help for snap info --verbose is not correct since a while I suspect | 09:13 |
| Chipaca | pedronis: how so? | 09:17 |
| Chipaca | pedronis: incomplete perhaps | 09:18 |
| pedronis | Chipaca: well, incomplete means a bit not correct | 09:38 |
| Chipaca | pedronis: yes, incomplete is a flavour of incorrect | 09:38 |
| zyga | hm | 09:43 |
| mborzecki | travis jobs hitting timeouts again? | 09:53 |
| pedronis | Chipaca: I'm landing my branch and then will do a follow up to fix that (and couple other silly things related to help) | 09:56 |
| mup | PR snapd#5239 closed: client,cmd/snap,daemon,tests: expose base of a snap over API, show it in snap info --verbose <Created by pedronis> <Merged by pedronis> <https://github.com/snapcore/snapd/pull/5239> | 09:57 |
| thresh | to whoever thought it's a good idea to have backups of snaps in /var/lib/snapd/snaps, thank you | 10:18 |
| thresh | helps debugging a lot! | 10:18 |
| zyga | thresh: they are not really backups | 10:19 |
| zyga | they _are_ the snaps | 10:19 |
| zyga | but as for backups, Chipaca here is working on first part of backups (snapshots) | 10:19 |
| thresh | well yeah, but not removing the old ones | 10:19 |
| zyga | ah, I see what you mean now | 10:19 |
| thresh | I'm actually interested in a kinda timeline | 10:19 |
| thresh | e.g. "at that date we had snap rev 100 in stable" | 10:20 |
| thresh | it's partially doable looking at the metrics, though | 10:20 |
| mup | PR snapd#5244 opened: tests: shellchecks part 3 <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5244> | 10:31 |
| mup | PR snapd#5245 opened: cmd/snap-update-ns: add PathIterator <Created by zyga> <https://github.com/snapcore/snapd/pull/5245> | 10:54 |
| zyga | mborzecki, Chipaca: ^ perhaps interesting :) | 10:55 |
| zyga | mborzecki: review on shellcheck ready | 11:09 |
| cachio | pedronis, you have a vm in google, are you using it? | 11:21 |
| cachio | niemeyer, you too | 11:21 |
| pedronis | cachio: no, some left over from a killed run or something | 11:24 |
| cachio | pedronis, ok | 11:26 |
| cachio | pedronis, deleted, thanks | 11:27 |
| mborzecki | maybe we should consider switching fedora 28 to manual, at least for a week or so, their repos seem to be under some load right now | 11:33 |
| zyga | mborzecki: +1 | 11:33 |
| mup | PR snapd#5246 opened: spread: switch fedora 28 to manual <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5246> | 11:37 |
| mborzecki | zyga: ^^ | 11:41 |
| cachio | mborzecki, yes | 11:41 |
| cachio | it is giving timeout | 11:41 |
| cachio | I'll make the change | 11:41 |
| mborzecki | cachio: it's there already ^^ | 11:41 |
| cachio | mborzecki, awesome | 11:42 |
| cachio | this is fast | 11:42 |
| cachio | mborzecki, you should enable fedora-27 | 11:43 |
| mborzecki | cachio: but this would be the same infrastructure | 11:43 |
| cachio | mborzecki, well, it was not giving any timeout when we were using f27 | 11:44 |
| cachio | we started with the timeout errors when we moved to f28 | 11:44 |
| cachio | right? | 11:45 |
| === pstolowski is now known as pstolowski|lunch | ||
| mborzecki | cachio: pushed a commit enabling fedora 27 | 11:46 |
| cachio | mborzecki, tx | 11:47 |
| * cachio afk | 11:49 | |
| mup | PR snapd#5247 opened: cmd/snap: small help tweaks and fixes <Created by pedronis> <https://github.com/snapcore/snapd/pull/5247> | 12:01 |
| pedronis | Chipaca: ^ | 12:02 |
| Chipaca | pedronis: :-D awesome | 12:02 |
| jdstrand | zyga: hey, before you spend a lot of time on it, I suggest you read https://github.com/snapcore/snapd/pull/5241#issuecomment-393859855 | 12:02 |
| mup | PR #5241: interfaces/udev: call 'udevadm settle --timeout=10' after triggering events <Created by jdstrand> <https://github.com/snapcore/snapd/pull/5241> | 12:03 |
| zyga | thanks, reading | 12:03 |
| zyga | mmm, | 12:04 |
| zyga | I'd love to see the work towards us being able to "commit" certain things only once in a larger interface transaction | 12:05 |
| zyga | I think it's simple on our end but terribly complex in snapd in general because of hooks and some, perhaps, implicit assumptions on when the state has settled | 12:05 |
| jdstrand | zyga: yeah | 12:06 |
| jdstrand | zyga: it would definitely help with apparmor loads and speed up refreshes a lot | 12:08 |
| zyga | yes | 12:09 |
| jdstrand | zyga: core refreshes where interfaces change, etc | 12:09 |
| jdstrand | zyga: it would not be terribly helpful for udev with my upcoming PR | 12:09 |
| zyga | it might be able to inspect the rules and see if we need to trigger all or just some subsystems | 12:09 |
| jdstrand | zyga: that PR is pretty cool though-- I have a bunch of snaps installed and I udev monitor input events. I refresh and do things to cause snapd to call trigger and no input events are triggered, and no annoying pauses. then I install a snap that slots mir and the input subsystem is only triggered the one time. classic users aren't going to have mir, wayland or x11 slots snaps installed, and what I do for joystick doesn't cause a pause | 12:11 |
| jdstrand | neat! | 12:11 |
| jdstrand | zyga: ehh, we could do that but I think we would want to have the interfaces declare it. again, I don't think we should worry about '3' (only calling affected subsystems) until we see a need | 12:12 |
| === pstolowski|lunch is now known as pstolowski | ||
| jdstrand | and I don't think we'll see it. if we do, it won't be hard to expand on my '2' techniques | 12:13 |
| mborzecki | jdstrand: to add to udev triggered bugs, libinput and gsd seem to not play well with each other, and input settings are sometimes lost, and another one is pulse getting killed and no sound due to sound device changes | 12:14 |
| jdstrand | zyga: note, I also answered your questions in PR 5240 | 12:14 |
| mup | PR #5240: tests: add test to ensure /dev/input/event* for non-joysticks is denied <Created by jdstrand> <https://github.com/snapcore/snapd/pull/5240> | 12:14 |
| niemeyer | cachio: Thanks for the note.. I was using it, but probably won't come back to the problem in the next few hours.. so discarded it and will fire another one when I actually do | 12:15 |
| jdstrand | mborzecki: libinput/gsd would be worked around be my existing PR | 12:15 |
| jdstrand | mborzecki: I can submit that and then perhaps we could do a followup PR for sound | 12:15 |
| jdstrand | me hasn't seen that bug | 12:15 |
| mborzecki | i was able to trigger this by installing ohmygiraffe in a loop ;) | 12:16 |
| jdstrand | I find pulseaudio terribly annoying in general. it forgets stuff all the time when snapd didn't do anything | 12:16 |
| jdstrand | s/existing PR/upcoming PR/ | 12:17 |
| mborzecki | jdstrand: you can call spread shellcheck locally by issing `./spread-shellcheck <path-to-task.yaml>` | 12:21 |
| jdstrand | ah, handy | 12:21 |
| * jdstrand takes a note | 12:21 | |
| mborzecki | pedronis: what do you think about https://github.com/snapcore/snapd/pull/5243#discussion_r192378309 ? | 12:22 |
| mup | PR #5243: spread-shellcheck: silly fix & pep8 <Simple> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/5243> | 12:22 |
| zyga | mborzecki: can you split that so that pep8/shellcheck is in one PR and the ongoing discussion can evolve separately | 12:23 |
| jdstrand | zyga: another interesting idea for running apparmor or udev trigger only once is if we get to that point, we could run trigger first, then apparmor. this would avoid the race where apparmor opens up because it is relying on the device cgroup | 12:24 |
| mborzecki | zyga: i hope to have this resolved soon and avoid opening another pr | 12:24 |
| zyga | jdstrand: yeah, that could be easily done by arranging the backends in a specific order (or even making the two calls explicit so that backend order doesn't matter) | 12:25 |
| jdstrand | zyga: really, I want either profile composition in apparmor or inode labelling, then we can drop the device cgroup and have a udev helper that calls out to apparmor in a performant manner (ie, not a full profile reload) | 12:25 |
| zyga | what is profile composition? | 12:25 |
| zyga | and inode labelling? | 12:25 |
| zyga | note that udev backend is still very useful outside of ubuntu | 12:26 |
| pedronis | mborzecki: answered | 12:26 |
| jdstrand | zyga: profile composition is that idea that you parse your profile in different parts and stitch them togther. think of it as we have the entire profile that remains unchanged, we plugin a joystick and need to add a single rule. we compile that single rule, leaving all the rest of the policy alone and stitch it into a new profile to load | 12:27 |
| mborzecki | pedronis: thanks, pushing in a minute | 12:27 |
| zyga | ohh | 12:27 |
| zyga | jdstrand: that makes sense, is that on the roadmap? | 12:27 |
| jdstrand | zyga: the single rule parse is fast and the stitch is fast | 12:27 |
| jdstrand | zyga: yes | 12:27 |
| zyga | jdstrand: I hope we can mainline all of the current stuff first though, it's still a drag that we cannot use apparmor in debian and opensuse fully | 12:28 |
| jdstrand | zyga: inode labelling is where you have a rule that is like: 'file label=foo rw,' then you have a udev helper that adds 'foo' to the inode of the device. no profile recompiles | 12:29 |
| jdstrand | (that rule syntax is made up, but illustrates the point) | 12:29 |
| zyga | jdstrand: that looks like selinux | 12:29 |
| jdstrand | zyga: it does in that we are storing labeling information in the inode, yes. note that in kernel, this is all happening already with dynamic labeling | 12:30 |
| jdstrand | ie, all files have security labels in the kernel (that's a bit of a simplification of course) | 12:31 |
| zyga | my mind was blown a while ago when I read a bit more apparmor kernel code, the "label" is really a complex object, not just a string that I somehow implicitly expected | 12:31 |
| jdstrand | zyga: dynamic labelling is usually way more flexible (it is how we can do what we do) but there are times when inode labeling is really handy | 12:31 |
| jdstrand | zyga: yes | 12:31 |
| jdstrand | this is also planned. both composition and labelling have had (upstream) work done on them | 12:32 |
| jdstrand | zyga: you asked about upstream: this is all happening upstream and things are getting closer with each new upstream kernel | 12:32 |
| jdstrand | popey: I don't know if you saw my comment from 25 minutes ago: 07:11 < jdstrand> zyga: that PR is pretty cool though-- I have a bunch of snaps... | 12:37 |
| jdstrand | popey: I think you are going to like that :) | 12:37 |
| popey | Tease | 12:37 |
| popey | I didn't see it, no. | 12:37 |
| popey | Yay | 12:38 |
| jdstrand | popey: I have a hunch it will even avoid the xorg bug (since the error report indicated issues in the input event handling) | 12:39 |
| * zyga -> hungry | 12:39 | |
| popey | Good. I have just had another one :| | 12:39 |
| jdstrand | boo | 12:39 |
| jdstrand | popey: do you have anything in the logs at the time of the crash? | 12:39 |
| popey | which logs in particular? | 12:40 |
| popey | I have an xorg crash file as always | 12:40 |
| jdstrand | Xorg or syslog/journalctl | 12:40 |
| Chipaca | pedronis: wrt snapshot conflicts, should i change it to check task kinds instead of changes'? | 12:40 |
| mborzecki | cachio: fedora 27 looks bad too :/ i'll drop that last commit | 12:41 |
| pedronis | Chipaca: that is more typical, but honestly I want to get away from checking kinds (task or change) | 12:41 |
| Chipaca | pedronis: what would you rather do? | 12:41 |
| pedronis | Chipaca: just have a thing that consider if there's an op on the snap and have at most one op | 12:42 |
| pedronis | Chipaca: but it seems you want snapshot level granularity, is there a deep reasons, or is just nice? | 12:43 |
| Chipaca | pedronis: things get weird if you operate on a snapshot in conflicting ways | 12:44 |
| Chipaca | although if it were snap-level it would still catch them | 12:44 |
| pedronis | Chipaca: because snapshot and snaps are distinct concepts ? | 12:45 |
| pedronis | I mean snapshot has no corresponding snap? or does it? | 12:45 |
| popey | jdstrand: http://paste.ubuntu.com/p/jYCVpr8Z3d/ around line 3687. System went pop at 13:18 or so | 12:45 |
| Chipaca | pedronis: a 'snapshot-check' running at the same time as a 'forget' for example | 12:45 |
| pedronis | I understand | 12:45 |
| pedronis | I'm asking do this thing have always an associated snap or not? | 12:46 |
| pedronis | I haven't followed all the details | 12:47 |
| Chipaca | pedronis: but if a snapshot operation on a set of snapshots of snaps that include X is an operation on snap X, and thus conflicts with other snapshot operations that include snapshots of snap X, it'll probably work out | 12:47 |
| Chipaca | pedronis: a snapshot is of a snap | 12:47 |
| pedronis | ok, that's what I would like to go | 12:47 |
| pedronis | trying to understand if snapshots are special | 12:48 |
| Chipaca | pedronis: and some snapshot tasks that operate on a snap even add a snap-setup to help the snapstate side of conflicts | 12:48 |
| Chipaca | pedronis: (bah, just save and restore, check and forget don't but could easily) | 12:49 |
| pedronis | I mean we can have both snaps and snapshotes as high-level conflict sources | 12:49 |
| pedronis | what I think is getting fragile | 12:49 |
| pedronis | is looking at kinds | 12:49 |
| Chipaca | pedronis: can we refactor the whole thing after snapshots are landed? | 12:50 |
| pedronis | Chipaca: yes but the change kind thing is not here nor there | 12:50 |
| pedronis | doesn't mean is wrong but is yet another way to do things | 12:51 |
| pedronis | Chipaca: it's also going to be problematic when we start taking snapshots as part of other snap ops | 12:53 |
| Chipaca | pedronis: yeah | 12:54 |
| pedronis | (that's why in general we don't look at change kinds, they tend to be bag of things, except a few in devicestate) | 12:54 |
| Chipaca | pedronis: so i should at least change it to check task kind, not change kind | 12:54 |
| pedronis | yes | 12:54 |
| Chipaca | fair | 12:54 |
| jdstrand | popey: I'm seeing a lot of input devices being added and removed related to your nvidia hdmi/dp (and others). this makes me hopeful that the upcoming pr may help more than just the annoying input pauses during refresh | 12:56 |
| popey | Awesome | 12:56 |
| jdstrand | popey: how often do you see this? | 12:57 |
| popey | Wait. Input devices on display port? | 12:57 |
| popey | The crash or hangups? | 12:57 |
| * zyga just noticed that irc cloud does some magic to show popey's face next to the nickname | 12:57 | |
| jdstrand | popey: all kinds of things show up as evdev devices (/dev/input/event*) | 12:57 |
| popey | Ya. You can set it in your profile in the client zyga | 12:57 |
| popey | jdstrand: ok. How fun | 12:58 |
| jdstrand | popey: the crash. I know when the hangups will happen (and I can reproduce those, and have eliminated them in my poc) | 12:58 |
| popey | Ok | 12:59 |
| jdstrand | popey: I ask about the frequency of the crash cause I can give you a deb that can maybe alleviate some pain | 13:00 |
| popey | Depends how often I install snaps :) | 13:01 |
| popey | Happens at least once a week, twice this week | 13:01 |
| jdstrand | ok, let me be more direct | 13:01 |
| jdstrand | do you want a deb with my poc? | 13:01 |
| popey | I only worked 2 days this week though | 13:01 |
| popey | Sure | 13:01 |
| jdstrand | ok | 13:01 |
| popey | Thanks | 13:01 |
| jdstrand | hey, I can wormhole you this time! | 13:01 |
| niemeyer | cachio: Trouble connecting? | 13:02 |
| jdstrand | popey: wormhole is neat. note, this is snapd 'master' as of last night. if you want something based on release/2.33, I can build you another deb | 13:04 |
| cachio | niemeyer, trying | 13:04 |
| jdstrand | popey: you probably want to 'snap refresh core --stable' so that you continue using the deb | 13:04 |
| popey | Ok | 13:05 |
| jdstrand | popey: I plan on sending up the PR today, so hopefully next week it'll be in edge, if not 2.33 (we'll see) | 13:05 |
| popey | Awesome news. Much appreciated | 13:06 |
| jdstrand | popey: please give negative or positive feedback-- I think you'll be pleased with refreshes and installs | 13:06 |
| popey | Ok. Will beat it up a bit in a dark alley | 13:07 |
| mup | PR snapd#5246 closed: spread: switch fedora 28 to manual <Created by bboozzoo> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5246> | 13:26 |
| zyga | Power outage | 13:27 |
| niemeyer | jdstrand: QUick ping on https://forum.snapcraft.io/t/classic-confinement-request-communitheme-set-default/5146/17 | 13:39 |
| pedronis | mborzecki: thx about the heads up about merging master into my PR | 13:39 |
| jdstrand | niemeyer: a design for update alternatives is not going to be a quick call. I won't have time to prepare for said call by this afternoon. we could try for say, tuesday | 13:43 |
| niemeyer | jdstrand: That works as well, thanks! | 13:44 |
| zyga | Break for lunch | 13:45 |
| mup | PR snapd#4767 closed: interfaces: disconnect hooks <Critical> <Squash-merge> <Created by stolowski> <Closed by stolowski> <https://github.com/snapcore/snapd/pull/4767> | 13:50 |
| jdstrand | cachio: fyi, PR 5240 should be ready for another review | 14:03 |
| mup | PR #5240: tests: add test to ensure /dev/input/event* for non-joysticks is denied <Created by jdstrand> <https://github.com/snapcore/snapd/pull/5240> | 14:03 |
| cachio | jdstrand, sure, I'll do, tx | 14:04 |
| mup | PR snapd#5243 closed: spread-shellcheck: silly fix & pep8 <Simple> <Created by bboozzoo> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/5243> | 14:04 |
| pedronis | pstolowski: #5162 is red, haven't looked, might need a master merge | 14:05 |
| mup | PR #5162: overlord/hookstate: support undo for hooks <Critical> <Created by stolowski> <https://github.com/snapcore/snapd/pull/5162> | 14:06 |
| pstolowski | pedronis: The job exceeded the maximum time limit for jobs, and has been terminated - after starting fedora | 14:07 |
| pedronis | pstolowski: yea, need master (where I think fedora has been put to manual temporarely) | 14:08 |
| Chipaca | zyga: i just noticed opengl stopped working here | 14:11 |
| Chipaca | _just_ missed maciej | 14:11 |
| Chipaca | :-) | 14:11 |
| pstolowski | pedronis: ah, ok, doing | 14:17 |
| cachio | jdstrand, ready | 14:30 |
| mup | PR snapd#5240 closed: tests: add test to ensure /dev/input/event* for non-joysticks is denied <Created by jdstrand> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/5240> | 14:31 |
| jdstrand | cachio: thanks! :) | 14:34 |
| zyga | Still lunching | 14:38 |
| pedronis | mmh, didn't seem to help, still timeout | 14:47 |
| King_InuYasha | :/ | 14:48 |
| pstolowski | yeah something is not quite tight | 14:50 |
| pstolowski | *right | 14:50 |
| popey | jdstrand: your snap breaks my package manager, because ubuntu-core-launcher : Depends: snapd (= 2.32.8+18.04) but 2.33~rc1 is installed | 14:55 |
| popey | I'll live... | 14:55 |
| jdstrand | popey: remove ubuntu-core-launcher | 14:55 |
| popey | not needed? | 14:55 |
| jdstrand | popey: it is a transitional package and doesn't ship anything | 14:56 |
| mup | PR snapd#5248 opened: tests: add test to ensure /dev/input/event* for non-joysticks is denied - 2.33 <Created by jdstrand> <https://github.com/snapcore/snapd/pull/5248> | 15:02 |
| pedronis | pstolowski: yes, all fedora is manual but things are still timing out | 15:06 |
| zyga | 49 minutes :/ | 15:06 |
| pedronis | it's a bit hard to tell when it started | 15:08 |
| pedronis | lots of red runs also yesterday for master | 15:08 |
| cachio | niemeyer, the key name is kill_timeout_hs | 15:10 |
| cachio | is it ok? | 15:10 |
| cachio | do you prefer just timeout? | 15:10 |
| cachio | default to 2 | 15:12 |
| pedronis | not super clear what is slow, but for example snapd-snap tests say "running late" | 15:30 |
| cachio | zyga, we got a timeout now building 16.04 | 15:31 |
| cachio | it iw weird | 15:32 |
| cachio | https://api.travis-ci.org/v3/job/386694389/log.txt | 15:32 |
| pedronis | everything setupish seems to be running late | 15:32 |
| cachio | pedronis, perhaps another dependency is working slow | 15:33 |
| cachio | 8 build reds in a row | 15:34 |
| cachio | 26 <kill-timeout reached> on https://api.travis-ci.org/v3/job/386691952/log.txt | 15:36 |
| zyga | I will retry jobs at night | 15:45 |
| Chipaca | brb, need to reboot | 15:47 |
| niemeyer | cachio: This is "kill-timeout: 2h" in spread already.. we can just use the same syntax for both key and values | 16:00 |
| cachio | niemeyer, sure, kill-timeout will be | 16:00 |
| cachio | I already was testing the snap | 16:01 |
| cachio | it works properly | 16:01 |
| cachio | I'll make this change | 16:01 |
| pstolowski | zyga: are you going to kick all failed jobs at night? i'd be interested in landing #5162 (in any case i'll take a look at it tomorrow and retry it if necessary) | 16:08 |
| mup | PR #5162: overlord/hookstate: support undo for hooks <Critical> <Created by stolowski> <https://github.com/snapcore/snapd/pull/5162> | 16:08 |
| === pstolowski is now known as pstolowski|afk | ||
| niemeyer | cachio: Oops, minor tweak: this is actually halt-timeout | 16:46 |
| cachio | niemeyer, ok, np | 16:46 |
| cachio | I'll update it | 16:46 |
| niemeyer | cachio: kill-timeout is the time for individual tasks to be killed | 16:46 |
| niemeyer | cachio: warn-timeout for warns, and halt-timeout for system halting | 16:46 |
| cachio | niemeyer, we are gonna need an specific service account for this | 16:51 |
| cachio | or you prefer to use yours? | 16:51 |
| zyga | pstolowski|afk: yes, I will restart everything | 16:55 |
| pstolowski|afk | zyga: k, thx | 16:55 |
| cachio | niemeyer, I need to leave now, I'll update the gce-cleaner snap an leave an instance running in gce when I am back | 17:03 |
| cachio | niemeyer, I'll monitor this one for a time to see if it works properly and then you can take it | 17:04 |
| niemeyer | cachio: Nice, thanks! | 17:07 |
| * cachio afk | 17:08 | |
| mup | PR snapd#5249 opened: interfaces/home: remove redundant common interface assignment <Simple> <Created by jdstrand> <https://github.com/snapcore/snapd/pull/5249> | 17:20 |
| === tedg_ is now known as tedg | ||
| === sergiusens_ is now known as sergiusens | ||
| mup | PR snapd#5250 opened: interfaces/udev,misc: only trigger udev events on input subsystem as needed <Created by jdstrand> <https://github.com/snapcore/snapd/pull/5250> | 20:07 |
| jdstrand | popey: that one's for you buddy ^ | 20:07 |
| jdstrand | it may need some iterating | 20:08 |
| niemeyer | jdstrand: Replied on https://forum.snapcraft.io/t/auto-connection-of-gtk3-themes-icon-themes-and-sound-themes-interfaces/5118/17 | 20:12 |
| jdstrand | niemeyer: thanks, that sounds reasonable. when they respond, I'll issue it | 20:21 |
| niemeyer | jdstrand: Thanks! | 20:31 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!