/srv/irclogs.ubuntu.com/2019/09/04/#snappy.txt

jameshamurray: out of interest, was there some project in particular you wanted debug symbols for?04:32
amurrayjamesh: I have a couple reports of crashes of my emacs snap and I am stumped how to easily enable users to get good backtraces04:33
jameshwe definitely want debug symbols for the snaps seeded on the desktop04:37
jameshbut there's not much to be done when they want a full design for snapcraft+store+snapd before considering any changes04:39
amurrayjamesh: understood this is a "hard problem" ... but if folks want snaps to be first-class on the desktop there needs to be more focus on this IMO from all parties - it's not enough to have just you pushing on it (but I am glad you were)04:52
jameshamurray: sure.  We've discussed it multiple times at previous dev summits, but when I've got more work to do than time available it is hard to prioritise something that gets ignored.04:55
amurrayjamesh: i hear you04:56
jameshI have gone back to update the first PR (https://github.com/snapcore/snapcraft/pull/2229) a few times to save it from bitrot, but that's about it04:56
mupPR snapcraft#2229: elf: extract build ID and presence of debug info <Created by jhenstridge> <https://github.com/snapcore/snapcraft/pull/2229>04:56
mborzeckimorning05:13
mborzeckiwow, +11C outside :/05:13
zygaGood morning05:39
zygaWow, I have a few more so not that drastic yet05:40
zygaBrrrrrr05:40
mborzeckizyga: hey05:43
zygaDoing school walk, be in the office soon05:48
Mirvdo apps like signal-desktop, spotify, discord work for you on Ubuntu under Wayland? none of them start for me on SUSE under Wayland, and I'm wondering whether there's some low hanging fruit to grap.06:03
MirvFYI https://pastebin.ubuntu.com/p/bSjqrJDhtf/06:04
MirvI tried to search for LP bug reports at one point but it didn't sound they'd be broken for everyone06:05
MirvUbuntu defaults to X.org still so that makes it a bit different of course06:05
jameshMirv: is Xwayland listening on the abstract namespace socket?06:06
Mirvlet me parse that for some time (or tell me how to check). it's running as /usr/bin/Xwayland :0 -rootless -terminate -accessx -core -auth /run/user/1000/mutter/Xauthority -listen 4 -listen 5 -displayfd 606:07
jameshi.e. @/tmp/.X11-unix/X0 in addition to /tmp/.X11-unix/X006:07
jameshI think you can check with ss.  Let me work out the invocation06:07
jamesh"ss -xl | grep X11" might be the best way to check06:08
jameshI see two listening sockets locally06:09
mborzeckizyga: ^^ opensuse & wayland06:09
Mirvthere's @ too: https://pastebin.ubuntu.com/p/QVKJpjP26R/06:10
jameshI'm not sure then. The non-abstract socket is definitely blocked by snap confinement though06:11
Mirvok, let's wait for zyga then. I'm in no hurry, just starting my work day, happy to help debugging.06:11
zygaback in the office now06:23
zygaaha, I see the chatter about wayland and suse06:24
zygaI _guess_ I'm running tumbleweed under X11, let me recheck06:24
zygalogged back under wayland06:26
zygait fails on06:27
zyga(and now no copy-paste, fun)06:28
zygaon /run/user/1000/mutter/Xauthority06:28
zygaI recall seeing this in a PR recently06:28
zygahold on06:28
zygaMirv: confirmed, adding an extra rule for that makes it work06:31
zygaMirv: is there a bug report? I'll handle the rest06:31
* zyga needs to work on reviews today06:43
zygagood morning mvo06:44
mvohey zyga06:44
mvozyga: good morning! I also need to do reviews06:44
mborzeckimvo: hey06:45
mvohey mborzecki06:46
abeatomorning! question about the content interface, is it possible to share only a file, or it must be a full folder?06:48
zygaabeato: currently it must be a file but I have a patch to share single files stashed06:48
zygaer06:49
zygamust be a directory06:49
abeato:D06:49
abeatozyga, great, nice that we will have one file sharing soon, that will be useful if you want to share just a socker06:49
abeatoshocket06:49
zygasockets are more tricky06:50
zyganot sure it'd work for sockets06:50
zygaI have not tried06:50
abeatohm, right, well, but in the end it is a file, right? mount bind should work too?06:51
zygaish,06:51
zygasockets have a tricky apparmor side06:52
zygaanyway, it's a matter of checking06:52
abeatoright06:52
mupPR snapd#7403 opened: cmd: use libtool for the internal library <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7403>06:59
=== pstolowski|afk is now known as pstolowski
pstolowskimorning07:02
Mirvzyga: there was not, I now filed bug #184261507:02
mupBug #1842615 opened: Wayland apps not starting under openSUSE Tumbleweed <Snappy:New> <https://launchpad.net/bugs/1842615>07:02
mupBug #1842615: Wayland apps not starting under openSUSE Tumbleweed <Snappy:New> <https://launchpad.net/bugs/1842615>07:03
mborzeckipstolowski:  hey07:03
Mirvzyga: awesome :) meanwhile I checked the problem does not affect openSUSE Leap 15.0, so just Tumbleweed07:04
mborzeckizyga: can you take a look at https://github.com/snapcore/snapd/pull/7387 ?07:04
mupPR #7387: packaging/fedora, tests/lib/prepare-restore: helper tool for packing sources for RPM <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7387>07:04
zygaMirv: thank you07:17
zygamborzecki: yeah but I'll start with older PRs first07:17
zygaor I'll starve them again :)07:17
ackkzyga, hi, so I just ran into that issue again, and it doesn't have to do with snapcraft rewriting the prime dir: https://paste.ubuntu.com/p/BVDHmmCn48/07:19
zygaackk: that's interesting, can you mkdir /root/snap/maas/x1 yourself?07:20
zygaackk: or will you get a EROFS error?07:20
zygaMirv: I've updated the bug report and I'll be sending a PR to fix it later today07:20
ackkzyga, the dir already exists07:20
zygaackk: is it a mount point?07:20
zygaackk: can you write through it?07:20
ackk$ sudo ls -ld /root/snap/maas/x107:21
ackkdrwxr-xr-x 1 root root 12 Sep  4 06:59 /root/snap/maas/x107:21
mupBug #1842615 changed: Wayland apps not starting under openSUSE Tumbleweed <snapd:In Progress by zyga> <https://launchpad.net/bugs/1842615>07:21
ackkzyga, yeah I can write it as root07:21
ackkI mean, write a file in it07:21
ackkit's not a mountpoint07:21
zygalet me look again07:21
zygathat's odd07:21
zygaackk: can you please try this:07:22
zygasudo strace -o /tmp/strace.log SNAP_INSTANCE_NAME=maas07:23
zyga/snap/snapd/4605/usr/lib/snapd/snap-confine snap.maas.maas /bin/true07:23
zyganot sure if free of typos07:24
zygabut give it a try07:24
zygaI'd like to see strace run of snap-confine07:24
zygato see what was the error it failed on07:24
ackkzyga, that's all a single command right?07:25
zygacorrect07:25
ackkstrace: Can't stat 'SNAP_INSTANCE_NAME=maas': No such file or directory07:25
ackkthat can't go there, it seems07:25
ackkI guess before the strace?07:26
zygasorry, my bad07:26
zygayes07:26
zyga:)07:26
ackk$ sudo SNAP_INSTANCE_NAME=maas strace -o /tmp/strace.log /snap/snapd/4605/usr/lib/snapd/snap-confine snap.maas.maas /bin/true07:26
ackkstrace: exec: Permission denied07:26
Mirvthank you zy_ga!07:26
ackkzyga, ^07:27
zygaackk: _hmm_07:27
zygadid I get strace wrong somehow07:27
zygaackk: and it didn't leave any log, did it?07:28
ackkzyga, just http://paste.ubuntu.com/p/5VW47nhgv9/07:29
zygawhy would it do that?07:29
zygais that under lxd?07:29
zygaperhaps thats why :/07:29
zygacan you check on your host, if you got a denial in dmesg?07:30
Jonno_FTWhello07:30
zygaackk: one more idea07:30
Jonno_FTWI get this error: error: cannot add authorization: open /users/jonm/.snap/auth.json: permission denied07:31
Jonno_FTWwhen my $HOME is on NFS07:31
zygaackk: can you install an older snapd and check if this happens?07:31
zygaJonno_FTW: hey, can you tell us the command you ran?07:31
zygaJonno_FTW: and snap version, please07:31
ackkzyga, like, stable?07:31
zygayes07:32
Jonno_FTWsudo snap install redis-desktop-manager, snap version 2.39.2-1.fc3007:32
mvoondra: hey, do you want to push your "lk-next" changes to "lk"? it seems you adrdress issues found in the review from chipaca?07:32
zygaaha, fedora07:32
Jonno_FTWyep07:32
zygamborzecki: ^ can you render ideas on how to check denials for selinux + nfs?07:32
ondramvo I will, once I test them :)07:33
ondraI'm just flashing test image07:33
mborzeckizyga: ackk: snap run --strace='--raw -o foo.log' maas if you can hit ^C in time07:33
zygamborzecki: that likely wont help if lxd is in control on top and denies it already07:33
zygamborzecki: I meant the case above from Jonno_FTW07:33
mborzeckiJonno_FTW: can you try ausearch -m AVC and look for anything with recent time stamp07:34
ondramvo it is booting, so image prepare is working, I just want to run few core refresh tests07:34
ondramvo I already tested snapd snap update and that worked as well07:34
Jonno_FTWmborzecki: there's nothing recent07:35
Jonno_FTWcertainly nothing from around the time I started using snap (like 30m ago)07:35
ackkzyga, I can't switch to stable snapd as maas uses the system-usernames07:37
zygaackk: well, just for a test, it doesn't really have to work all the way07:38
zygajust to see if this is something in 2.4107:38
mborzeckiJonno_FTW: for the record, the error when you do snap login right?07:38
mborzeckis/error when/error pops up when/07:38
ackkzyga, mborzecki https://paste.ubuntu.com/p/D7Ddt4rZ6X/07:38
ackkzyga, well it doesn't work at all as I get https://paste.ubuntu.com/p/rJ4pWKvQkC/ from snapd07:39
zygawhat's /snap/snap/bin/maas/current?!07:39
zygaackk: try without /snap/bin/maas, just maas07:39
zygaI think that's snap run getting confused07:39
ackkzyga, that's the first try there07:39
zygaright, I just mean that /snap/snap is clearly wrong07:40
ackk$ snap run --strace='--raw -o foo.log' maas upgrade07:41
ackk/usr/bin/strace: exec: Permission denied07:41
ackkerror: exit status 107:41
zygaackk: on lxd host, can you run dmesg | grep DENIED07:41
zygaI'm sure it's lxd07:41
ackkzyga, https://paste.ubuntu.com/p/NPTRVQJtqQ/07:45
zygaackk: that's excellent07:45
zygaso that is confirmed07:45
zygaackk: to move forward we should figure out how to switch lxd's own profile on your host into permissive mode07:45
zygathen we can debug forther07:46
zyga*further07:46
zygaackk: do you think we can postpone until lxd developers are online?07:46
ackkzyga, sure07:46
ackkzyga, I don't have a real xenial host to test with unfortunately07:46
zygaackk: that's fine07:47
ackkzyga, also when I tried installing maas snap from edge I didn't hit the issue, so i'm not sure what triggers it. I can run "maas init" and maas works fine, then something gets messed up07:49
ackkbut it might only happen in "try" mode, IDK07:49
zygathe code where it fails should fail regardless of try mode07:49
zygaI'd like to get to the bottom of this later today07:49
ackkzyga, btw is there any chance that the snapfuse issue (taking 100%cpu in containers) could get fixed? that makes it impossible to install maas in a container07:49
ackk(install from a snap and not from "try')07:50
mborzeckizyga: pff the libtool patch fails on this https://paste.ubuntu.com/p/mGVnXnF5j6/ :/07:50
zygaackk: nobody on our team can fix that07:50
ackkzyga, is it a kernel or lxd thing?07:50
zygaackk: please escalate that to get help07:50
zygaackk: perhaps both07:50
zygait's a fuse filesystem07:50
zygamaybe the bottleneck is kernel side07:50
zygamaybe userspace07:51
ackkzyga, you're saying it's a bug in squashfuse?07:51
zygaI don't think any of us really read that code07:51
zygaackk: it's certainly not a bug in snapd, we just mount a filesystem07:51
zygamborzecki: yeah I ran into that a number of times07:51
zygamborzecki: load of garbage :/07:51
mborzeckiduh, obviously there's no way to have misspell skip files :/08:00
zygamborzecki: sed it08:01
mupPR snapd#7398 closed: boot, etc: simplify BootParticipant (etc) usage <Created by chipaca> <Merged by mvo5> <https://github.com/snapcore/snapd/pull/7398>08:18
zygamvo: I think we should all review from oldest to youngest08:19
zygait seems that the last two pages are like attic08:19
zygabrb, tea08:24
mborzeckizyga: added a workaround to #7403, other than that, all tests passed08:27
mupPR #7403: cmd: use libtool for the internal library <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7403>08:27
mvoa review for 7125 would be great, small, self-contained and open since jul08:30
mborzeckimvo: looking08:31
zygaback with tea08:45
mvoChipaca: good morning! if you could have a look at 7383 that would be great (should be simple, just a msg tweak)09:09
* Chipaca looks09:11
Chipacamvo: good morning! I've been working for the last hour but having to restart x a few times to sort out some things09:11
Chipacafun with nvidia :-/09:11
mvoChipaca: no worries09:11
zygaChipaca: drivers?09:11
mvoChipaca: and good luck with X, I kinda given up on nvidia for this reason :(09:11
zygamvo: my ubuntu experience with 1030 was very good FWIW09:11
zygamvo: not so much with 2080, LTS doesn't have the drivers for it yet,09:12
mvozyga: to be fair my experience is ~1-2y old so I may be totally off09:13
zygamvo: I think it depends on how recent the OS is, there have been some great strides lately09:14
zygamvo: not sure if you remember this but a few weeks ago nvidia has opened a large number of specs for their hardware to aid nouveau09:14
zygamvo: it looks like early days of amd shift, when the tactic was to open up, but it was still a long road towards benefits09:14
Chipacamvo: zyga: i broke things by trying to update the drivers from a ppa, to get some games running … undoing it was less than fun09:14
Chipacai'd forgotten how bad the quality of ppa debs was09:15
zygaChipaca: does your system work with 19.10?09:15
zygaChipaca: it's a good idea to just try that from a spare partition if it does09:15
Chipacazyga: I'm still waiting for confirmation it works with 18.04 :-/09:15
zygaChipaca: weekend evening project to check? :-)09:15
Chipacanu uh09:16
Chipacazyga: i've got way more fun weekend projects than updating a machine that works :-)09:16
zygathat's fair :-)09:16
zygaChipaca: I played one game recently, control, strongly recommend it if you have the 3 evenings to finish it09:16
zygaanyway, back to reviews :)09:17
mupPR snapd#7396 closed: tests: don't guess in is_classic_confinement_supported <Created by zyga> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/7396>09:18
mborzeckizyga: w8, 3 evenings? is that how long the game is?09:18
zygamborzecki: I think I played 6 hours each time09:19
zygawell into past 3AM :D09:19
zygamborzecki: I just finished the main plot, there's more to do but that's enough for me09:19
zygamborzecki: or I'm just pretty good at aiming09:20
zyga;-)09:20
mborzeckihahaha09:20
mborzeckimad quake skillz? :)09:20
zygamouse sensitivity set to maximum, field of view 120, those things ;)09:20
zygajust kidding though09:21
zygathe game is story driven09:21
zygathe combat is challenging but doesn't feel like grind09:21
zygaand unlike most shooters I can remember, it is very rewarding to use the very first revolver you get all the game09:21
zygaanyway09:21
zygasomething for evening :)09:22
mborzeckizyga: fwiw, double barrel works in through all of doom too :P09:23
zygamborzecki: indeed, but that's an exception, here the game gives you one gun, sharing ammo across all forms, and as you play you unlock more forms that are more suited for specific type of combat; what it doesn't do is to give you a much better gun that obsoletes the first one09:24
SaviqChipaca: hey, you seem to know about snapshots, is it expected that `snap restore` does not restart the snap? is there something that can be done about that?09:24
ChipacaSaviq: yes, it's expected; stop all services / apps before restore, start them after09:25
Saviqkk09:27
ChipacaSaviq: that is, if you need to09:27
ChipacaSaviq: you don't always need to -- same as you don't always need to stop for a 'save'09:27
SaviqChipaca: yeah, which is why  it feels like there should be hooks for this :)09:28
ChipacaSaviq: would a hook be able to know whether an app in the snap is running?09:29
Chipaca(in general the answer is 'no')09:29
SaviqChipaca: an app, no, but a service, yeah09:30
Chipaca(zyga's work wrt deferring refreshes while apps are running might let us fix that)09:30
Saviqthere's also https://forum.snapcraft.io/t/automatic-snapshot-opt-out/12131 - a `save` hook could reduce the size of the snapshot, but maybe it'd be better to have a whitelist for this particular usecase09:31
mborzeckipstolowski: can you take a look at https://github.com/snapcore/snapd/pull/7387 ?09:33
mupPR #7387: packaging/fedora, tests/lib/prepare-restore: helper tool for packing sources for RPM <Simple 😃> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7387>09:33
pstolowskimborzecki: sure09:34
mborzeckithanks!09:34
zygajamesh: I finally reviewed https://github.com/snapcore/snapd/pull/6767#pullrequestreview-28348331410:03
mupPR #6767: wrappers: allow snaps to install icon theme icons <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/6767>10:03
jameshzyga: thanks.  Looking10:03
zygajamesh: I think my only comment that would warrant blocking are related to validation10:04
zygajamesh: on non-compliant files in meta/gui/icons10:04
zygajamesh: as well as on file sizes in that place10:04
zygajamesh: everything else is something that is good for followup10:04
zygajamesh: I'm moving on to the next review, please ping me with your opinion10:05
zygajamesh: as I said, you have +2 and you can merge as is10:05
zygajamesh: but I'm worried about opening a hole10:05
Chipacazyga: FWIW we already don't do that sort of validation on icons10:05
Chipacazyga: so it's independent from that work10:05
zygaChipaca: yes but now we read them into memory10:05
zygaChipaca: in this PR10:05
Chipacawe do?10:05
Chipacawhere?10:05
zygaReadFile10:05
Chipacawhy do we ReadFile them?10:06
* Chipaca goes to look10:06
pedronisbecause Ensure* does that internally10:06
pedronisbut I made a proposal in one of my comments10:06
zygahttps://github.com/snapcore/snapd/pull/6767/files#diff-1a7d1cbe7825f23c89fbefcac29f347aR8210:06
zygahere10:06
zygaperhaps one way to solve this10:06
mupPR #6767: wrappers: allow snaps to install icon theme icons <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/6767>10:06
zygais to extend ensure10:06
zygato give it a handle that knows what the content is10:06
zygaand the size10:06
zygaso they can be stream like10:06
ograoh ! gpio-control ? who breeded that ? (thats beautiful ! )10:06
zygathat's a good idea separately but I would not like to just let this silently merge without pointing it out10:07
zygaogra: thank ondra10:07
* ogra thanks ondra 10:07
zygaI'll keep doing reviews to unblock people, I said my thing10:07
ograondra, beer for you in barcelona !10:07
zygapedronis: should I review https://github.com/snapcore/snapd/pull/6705 ?10:11
mupPR #6705: bootloader: little kernel support <Needs Samuele review> <Created by kubiko> <https://github.com/snapcore/snapd/pull/6705>10:11
pedroniszyga: I'm looking at it right now, going a bit slowly10:19
pedronisalso lunch10:19
zygaok, I'll pick up the next one10:19
zygamvo: hey, is https://github.com/snapcore/snapd/pull/6839 something that's ready for review?10:22
mupPR #6839: devicestate: allow remodel to different kernels  <Remodel :train:> <Created by mvo5> <https://github.com/snapcore/snapd/pull/6839>10:22
zygajamesh: this needs some comment feedback https://github.com/snapcore/snapd/pull/707310:24
mupPR #7073: interfaces: OpenGL/Vulkan drivers provided by a base should be usable <⛔ Blocked> <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/7073>10:25
jameshzyga: I think the way forward for that PR was to relax the AppArmor rules for snaps using a base other than core/core1810:26
zygajamesh: indeed, can you please comment on it so that it's clear where we stand?10:26
jameshon the assumption that non bootable bases should only contain files that are intended for use by apps10:26
zygayeah, I recall the conversation about that in Toronto10:27
mupPR snapd#7404 opened: cmd/snap: don't append / to snap name just because a dir exists <Simple 😃> <Created by chipaca> <https://github.com/snapcore/snapd/pull/7404>10:28
Chipacazyga: #7159 is an easy one10:29
mupPR #7159: tests: add functions to make an abstraction for the snaps <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7159>10:30
zygalooking quickly10:30
pstolowskipedronis: #7005 got 2 +1s and i adressed your earlier high-level feedback re having 'snap debug state..', would you like to take another look or can it land?10:32
mupPR #7005: debug: state-inspect debugging utility <Created by stolowski> <https://github.com/snapcore/snapd/pull/7005>10:33
pedronispstolowski: it might be easier to land it and I play with it a bit once is in edge10:50
pstolowskipedronis: ok, ty10:50
pstolowskipedronis: i'm thinking of adding some kind of high-level simplified potential-conflict-checking command for it in the future, something rudimentary that doesn't try to replicate existing logic (because that could simply hide an existing problem), but rather list candidates for closer inspection. not sure if/how this will work in practice and how valuable it will be, but sometimes just filtering out irrelevant data11:04
pstolowskifrom the state and setting initial focus is helpful11:04
zygajamesh: hey, question about https://github.com/snapcore/snapd/pull/7129#pullrequestreview-28353158211:06
mupPR #7129: userd: allow setting default-url-scheme-handler <Created by jwheare> <https://github.com/snapcore/snapd/pull/7129>11:06
zygaI know it's not your PR but do you know why we need to add a "get set sub-property API"?11:06
mvohm, tests are very red, shall we disable 19.10 for now?11:06
zygamvo: what is failing if I may ask?11:06
mvozyga: let me look again but pretty much everything is red11:07
zygalike all tests are failing?11:07
zygaI mean, there must be something that actually fails :)11:08
mvozyga: the samples I looked at this morning were 19.10 but let me re-look to see if this is actually still happening11:09
zygado you remember if a specific test was at fault?11:09
mvozyga: iirc desktop-portal but let me look again11:10
zygathat fails uniformly though11:10
zygait fails on me every day, or close11:10
* mvo nods11:10
zygaI'd be happier if we can disable the test instead11:10
zygaand look at it really hard until we really understand why11:11
mvohttps://travis-ci.org/snapcore/snapd/pull_requests <- looks sad11:11
pstolowskidegville: hey! i'm looking for the best place to add documentation for snap unset and snapctl unset (and equivalents with 'set' subcommands followed by exclamation mark). seems like they need to go to https://forum.snapcraft.io/t/configuration-in-snaps/510 and https://forum.snapcraft.io/t/supported-snap-hooks/3795 ; do you know any better spot?11:16
mupPR snapd#7405 opened: tests: disable interfaces-timeserver-control on 19.10 <Created by mvo5> <https://github.com/snapcore/snapd/pull/7405>11:19
jwhearezyga: re: 7129 my changes are almost entirely just repurposed existing code and style. i don't write go usually so feel free to make any stylistic changes you want11:23
jwhearethe reason i added GetSub was because i couldn't see a way in go to have multiple function heads with different numbers of arguments11:24
jwheare(unlike say erlang)11:24
jwhearewithout eliminating types and then doing dynamic type checks11:25
zygajwheare: hey, thank you for explaining that.11:28
zygaI will look at this with new information after lunch11:29
degvillepstolowski: hello - thanks for looking! I think you're absolutely right about those locations - especially Configuration in snaps. We can always shuffle things around if the words fit somewhere better, but I think they're the best place for now.11:34
pstolowskidegville: great, ty! ok if if i just edit it there (new paragraphs/subsections) and you take a look afterwards?11:35
degvillepstolowski: yep - sounds good!11:35
pstolowskiok11:36
ondraogra always ready for beer :P11:36
ondramvo sorry I was at dentist, back now and doing more testing11:38
ondramvo so far looks good11:38
ondramvo I will push changes from lk-next to PR branch11:38
mvoondra: sounds great, thank you11:38
ondramvo one more question, there is essential c header file, which should be implemented by corresponding bootloader, I was wondering where to host that header file11:39
ondramvo one option is to have it in bootloader/lkenv/snappy-boot.h11:40
Chipacahmm, is it always the same two 19.10 tests that are failing?11:40
ondramvo do you have any better idea where to host it?11:40
ondramvo OK there is problem with seeding with my test snap11:42
ondramvo I will debug, but it could be kernel seeding, when it tries to install bootimg11:43
Chipacaa pox on google:ubuntu-19.10-64:tests/main/interfaces-timeserver-control11:44
ondramvo interesting is that it seems to panic when applying core18 security profiles though11:55
ondramvo I will need to create image with cloud init to log in though, first boot crash is a bit pain to debug as there is no good way to log in. So I will try update path now11:57
ondramvo ignore me, i think it's kernel seeding, it just happens right after core11:58
pstolowskidegville: i updated the snap get/set document, but the one about hooks & snapctl needs some more work i think, it's a little bare when it comes to snapctl12:23
degvillepstolowski: thank you! I'll take a look at set/get.12:25
mupPR snapd#7406 opened: cmd/snap-confine: keep track of snap instance name and the snap name <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/7406>12:39
zygathanks!12:39
mborzecki+ '[' no = yes ']'12:42
zygatest failed12:42
zygathat's that ;)12:42
zygawhat printed that?12:42
mborzeckizyga: google:ubuntu-19.10-64:tests/main/interfaces-timeserver-control12:43
zygaour favourite test12:43
ijohnsonmborzecki: I saw the same thing on one of Chipaca's PR's last night12:44
ijohnsonI think it's a bug with eoan12:44
mborzeckimhm, mvo has a PR disabling it12:44
mborzeckifor 19.10 at least12:44
ijohnsonmborzecki: also thanks for the review, I responded12:44
mborzeckiijohnson: many dragons in snapstate handlers :) remember when i first dug into that12:45
ijohnsonyeah indeed!12:46
=== ricab is now known as ricab|lunch
ijohnsonand yeah also not really sure about the `saveSnapDisabledServices` name either, it feels awkward, but I also need to do the same thing multiple places so I wanted to eliminate code duplication12:46
pedronismvo: ondra: commented on the lk PR12:46
ondrapedronis thank you :)12:47
ondramvo found issue, we do not determeng run time mode correctly on device12:47
ondradetermine12:47
mupPR snapd#7405 closed: tests: disable interfaces-timeserver-control on 19.10 <Created by mvo5> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/7405>12:48
* ijohnson will miss SU again today, need to go with someone taking the citizenship oath :-)12:53
mvopedronis: thank you!12:56
zygamvo: https://github.com/snapcore/snapd/pull/7405/files#r320742967 ;-)12:57
mupPR #7405: tests: disable interfaces-timeserver-control on 19.10 <Created by mvo5> <Merged by chipaca> <https://github.com/snapcore/snapd/pull/7405>12:57
zygageee I wish I made coffee before the call12:58
zygajwheare: so, a quick question on Get/Set DBus API13:12
zygajwheare: can the "setting" attribute  be like a obj.attr?13:12
zygathat would allow us to have one get and handle everything with it13:13
zygajwheare: or perhaps / as I see the code already uses it13:13
mupPR snapcraft#2698 opened: dirs: check for existence of required data directories <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/2698>13:15
ondrapedronis I asked already mvo, but since you commented on it. I'd like to add c header file into lkenv dir for reference, otherwise I'm struggling to think about better place for it13:20
jwhearezyga: i tried not to change existing apis13:22
mupPR snapd#7403 closed: cmd: use libtool for the internal library <Created by bboozzoo> <Closed by bboozzoo> <https://github.com/snapcore/snapd/pull/7403>13:22
Chipacadegville, pedronis: https://twitter.com/jonworth/status/116918570582373580913:30
jwhearezyga: the / is used mainly just for logging strings tbh13:33
zygajwheare: I'm getting familiar with this area13:33
degvilleChipaca: ugh, thanks... if I squint, it looks like the sphinx. Riddle? Relic of lost civilisation?13:33
zygamy idea was to drop the Sub methods13:33
zygaand just allow more to be expressed by the property being interacted with13:33
mvozyga: re timeserver-control test - fun: https://paste.ubuntu.com/p/skNwxPPGGn/ - it looks like a version mismatch of the tools with the systemd version running13:37
jwheareif you'd rather the api just be e.g. Check("default-url-scheme-handler/irc", "ircclient.desktop") or Check("default-web-browser", "browser.desktop") that's your call13:38
pedronismvo: fun not13:38
mvopedronis: yeah, so the timdatectl from the snap is 229 and does not (apparently) understand what 241 tells it13:38
pedronisbig mess actually, we'll need to think13:39
jwhearei felt the more explicit separation of property/subproperty was a cleaner api, less magic, but i don't know the project13:39
Chipacadegville: right click, view image, zoom in -- or look for the post where jon links to the svg :)13:39
jwhearepersonally, i don't see more functions as a bad thing, in general13:40
pstolowskiChipaca: yay, that's very helpful, ty ;)13:40
jwhearebut i have no strong feelings. if you think encoding the subproperty within the property string makes sense go for it :)13:41
sergiusenspedronis: mvo sorry for insisting, but I wanted confirmation that the snapd snap builds fine with snapcraft from edge to fix  anything before another half complete release (I will start the release process with your green light)13:45
mvopstolowski: do you think you could double check if snapcraft from edge is fine with the snapd snap type?13:47
mvopstolowski: see the question from sergiusens13:47
sergiusensah, right, I did build pstolowski branch, if that is all you need, then we should be good to go (but ssome form of sanity test that could be run would be ideal)13:49
pstolowskimvo, sergiusens i will13:50
pstolowskimvo, sergiusens is it critical to check all build methods (lxd, multipass, destructive)?13:51
sergiusenspstolowski: in general no, unless you use hardcoded paths (I fixed on in a PR last week)... if you do try either multipass (or lxd) and --destructive-mode13:54
zygamvo: because of change in formatting?13:56
zygaI think I saw that13:56
mvozyga: yeah, I'm looking at it now13:58
mvozyga: hopefully not too hard to fix13:58
pstolowskisergiusens: ok, thanks, i'm going to check this in a moment (with my branch)14:00
=== ricab|lunch is now known as ricab
zygare14:11
zygamvo: any findings?14:15
mvozyga: in a meeting, but I see  [  503.301852] audit: type=1400 audit(1567605570.552:74): apparmor="DENIED" operation="capable" profile="snap.test-snapd-timedate-control-consumer.timedatectl-timeserver" pid=31856 comm="timedatectl.rea" capability=12  capname="net_admin"14:17
zygamissing permissions, it seems14:17
mvozyga: yeah, I look some more after the meeting14:19
zygathank you14:19
cachiomborzecki, Chipaca hey, could you please take a look to #740114:23
mupPR #7401: tests: add unstable stage for travis execution <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7401>14:23
Chipacacachio: yep14:23
cachiozyga, hey, yesterday I saw you were taking a look to a problem related to the snapd snap right?14:23
Chipacacachio: nice14:24
cachiozyga, it is the only concern I have before promoting this to candidate14:24
zygacachio: which specific problem, sorry - there were too many topics lately14:25
cachiozyga, the one related to maas14:26
zygaah14:26
zygasorry, yes14:26
zygait's not a regression IMO14:26
cachiozyga, the issue which you were discussing with ackk14:26
zygaand it's specific to specific development flow14:26
zygaI'd not block on it14:26
zygait's a UX issue, not a behavior change14:26
cachiozyga, perfect, thanks for the info14:26
zygaackk: ^ FYI14:28
mupPR snapd#7401 closed: tests: add unstable stage for travis execution <Created by sergiocazzolato> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7401>14:29
mupPR snapcraft#2699 opened: python plugin: add option to process-pipfile-lock for pipenv users <Created by cjp256> <https://github.com/snapcore/snapcraft/pull/2699>14:31
mupPR snapd#7407 opened: tests: run failing tests on ubuntu eoan due to is now set as unestable <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7407>14:33
jwhearezyga: do i interpret your latest comment to mean you're happy with Get and GetSub etc being separate?14:42
zygayes, that's right14:42
jwhearecool14:42
zygaI'm reviewing technical details now14:42
zygait's close I think14:42
mupPR snapd#7397 closed: tests: add --quiet switch to retry-tool <Simple 😃> <Created by zyga> <Merged by zyga> <https://github.com/snapcore/snapd/pull/7397>14:53
ackkzyga, so you know what's causing that issue?14:55
zygaackk: no, but if you are around we can work on exploring more14:56
zygaackk: we need to switch the host's lxd profile to permissive mode14:56
zygaackk: and try strace again14:56
cachiojamesh, hey, could you please take a look to the comment I left in #735114:56
ackkzyga, how do I do (and then revert) that?14:56
mupPR #7351: tests: retry checking until the written file on desktop-portal-filechooser <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7351>14:56
zygaackk: I think you can do that in-memory14:56
zygausing ...14:56
pstolowskisergiusens: snapd from my PR built fine with multipass (i took a gooooood while to download all deps). i'm going to check destructive now14:57
zygaackk: there's aa-complain14:57
zygaackk: but really we want to find the profile on disk14:57
sergiusenspstolowski: if what it built works,, then we are solid for moving forward :-)14:57
zygastgraber: where does lxd keep its apparmor profiles on disk?14:57
zygaackk: we need to edit one profile and add one word to make it advisory, not enforced14:57
ackkzyga, how do I find it? I'm not very familiar with aa14:58
pstolowskisergiusens: yeah i don't expect problem with destructive since that's already coverted by spread test in my PR.. but i can check manually for piece of mind14:58
zygaackk: unfortunately the kernel does not know so we just need to look around, hold on14:58
stgraberzyga: /var/snap/lxd/common/lxd/security/apparmor/14:59
zygastgraber: thank you so much!14:59
zygaackk: let's look at that path please14:59
zygaackk: we want to change the profile that was denied14:59
zygastgraber: perhaps there's an easier way to allow strace inside a container?14:59
zygaackk: can you please remind me the DENIED line that you found then?15:00
stgraberzyga: strace is allowed inside containers15:00
ackkzyga, https://paste.ubuntu.com/p/NPTRVQJtqQ/15:00
zygastgraber: we had a ptrace deny, perhaps it's more complex and two things together make it fail15:00
ackkstgraber, ^15:00
ackkstgraber, also, hi :)15:00
zygapeer="unconfined"15:00
zygaackk: when you ran snap-confine manually,that was inside the container, right?15:01
zygaackk: along with sudo and strace15:01
zygaall inside?15:01
ackkyes15:01
zygaok15:01
zygado I read that right, that snap-confine itself is denying the trace?15:02
zygaackk: can you show me how to get to this stage? I can explore this locally somewhat faster perhaps15:04
ackkzyga, well I'm not sure if you can get to that stage easily with the snap from the store15:04
stgraberso we do have a ptrace restriction which prevents cross-profile ptracing to avoid cross-container security issues, I wonder if this is somehow triggering there15:05
zygaackk: I'm happy to get some fresh stuff from git and play15:05
zygastgraber: perhaps, I can just disable everything quickly locally to get to the bottom of the error that is masked behind this15:05
stgraberackk: "lxc config set NAME raw.apparmor ptrace," should allow everything15:06
ackkzyga, so. you'd have to git clone lp:maas; build the snap and snap try it15:06
ackkstgraber, I assume I need to reboot?15:06
ackk(the container)15:06
stgraberyeah15:06
* ackk tries15:07
ackkthanks15:07
zygaackk: cloning15:07
zygaackk: so having maas cloned15:09
zygasnapcraft?15:09
ackkzyga, of course, after rebooting the container the issue went away :/15:09
zygaackk: oh?15:09
zygathat's interesting15:09
ackkzyga, yeah, I use --destructive-mode in the container15:09
zygacan you rebuild maas15:09
zygaand see if that breaks it again15:09
ackksure15:09
zygaand then, re-run the sudo snap-confine strace line15:09
* Chipaca needs caffeine15:10
* Chipaca looks for something intravenous15:10
mvoxnox: did timedatectl change in 19.10? when I run "timedatectl  set-ntp no" (or yes) the "timedatectl status" output does not change regardless of yes/no15:13
mvoxnox: this is GCE so maybe this is relevant15:22
mvoxnox: aha! so it looks like we have chronyc on GCE15:30
zygaooo15:30
zygamvo: so ntp is always on?15:30
zygaackk: any luck?15:31
mvozyga: sort of, yes15:32
mvozyga: anyway, I think I know what to do15:32
ackkzyga, no, I rebuilt and installed the snap (via snap try), but it seems to work so far15:32
ackkzyga, will try the strace if it happens again15:32
zygaackk: thank you15:33
ackkzyga, still, that snap-confine line fails with permission denied15:33
ackkalthough "sudo maas" currently works15:34
zygaackk: not with permission denied15:34
zygait's confusing15:34
ackk$ sudo SNAP_INSTANCE_NAME=maas strace -o /tmp/strace.log /snap/snapd/4605/usr/lib/snapd/snap-confine snap.maas.maas /bin/true15:34
zygaif you look at the source15:34
ackkstrace: exec: Permission denied15:34
zygait has failed already15:34
zygaah, sorry15:34
zygathat line does fail on strace and lxd15:34
Chipacahmm, i'm seeing repeated failures for google:ubuntu-{16,18}.04-64:tests/main/proxy-no-core15:35
zygathat's strace being unable to execute snap-confine15:35
ackk$ snap run --strace='--raw -o foo.log' maas upgrade15:35
ackk/usr/bin/strace: exec: Permission denied15:35
ackkerror: exit status 115:35
ackkzyga, ^ this one too15:35
zygayes, it's the same thing really15:35
xnoxmvo:  that.15:35
zyga(snap run call strace)15:35
xnoxmvo:  they asked and we changed default to chrony15:35
xnoxmvo:  on gce15:35
ackkzyga, ok15:35
JonelethIrenicuscan i change the default folder for snap configuration files?15:35
zygawhat does chrony do?15:35
zygaJonelethIrenicus: which files?15:36
zygabut most likely no, it's not very easy to do15:36
JonelethIrenicusin ubuntu 19.04 it has a snap folder in the home directory15:36
JonelethIrenicusit isn't a dot file15:36
zygathere's a bug report for that15:36
JonelethIrenicusahh ok15:36
zygait's extremely hard to change15:36
zygaI'm sorry15:36
JonelethIrenicusall good15:37
JonelethIrenicusjust was curious why it was in my home directory like that15:37
zygaJonelethIrenicus: this is bug https://bugs.launchpad.net/snapd/?orderby=-heat&start=015:38
zygaand I do intend to fix it15:38
zygab15:38
zygabut to fix it properly I need to build enough infra to be able to move it first15:38
JonelethIrenicusawesome thanks15:38
zygawithout breaking people15:38
zygaso it's a bit slow15:38
JonelethIrenicusright15:38
mvoxnox: thanks, I will fix our test then15:39
zygaI'm burned out with reviews today15:39
zygaI'll switch to coding15:39
* Chipaca hugs zyga 15:40
zygathanks Chipaca15:40
zygawriting is easier than reading15:40
zygayou know what you meant up front :)15:40
ondrapedronis are you OK for me to resolve comments which I already addressed? like change of error messages? Or are you gonna resolve those yourself?15:42
pedronisondra: I can do it15:42
pedronisChipaca: mborzecki: I proposed something in #7402 inspired by maciej ideas15:43
ondrapedronis thanks, just want to clean it a bit, so I can see which comments still need work15:43
mupPR #7402: daemon, client, cmd/snap: include architecture in 'snap version' <Needs Samuele review> <Created by chipaca> <https://github.com/snapcore/snapd/pull/7402>15:43
Chipacapedronis: we could think of (someday) adding cloud info there also15:44
Chipacanot sure if there's a uniform way of finding out what cloud you're on (maybe via cloud-init or sth)15:48
ondramborzecki do you have good example of gadget yaml to update to update assets15:49
ondramborzecki forum post seems to be missing good example15:49
mupPR snapd#7408 opened: tests: fix interfaces-timeserver-control on 19.10 <Created by mvo5> <https://github.com/snapcore/snapd/pull/7408>15:50
pedronisondra: done15:53
pedronisChipaca: we do get that info15:53
tomwardillhi! I'm trying to install snaps inside a xenial lxd, but getting "system does not fully support snapd: apparmor detected but insufficient permissions to use it". Is there a fix?15:53
ondrapedronis thanks15:53
pedronisChipaca: it's a bit too much info tough, people might want not to tell us15:53
ondrapedronis what do you think about option to add header file to snapd repo?15:54
ondrapedronis or what is best place to have that reference header file?15:55
zygaondra: would that header file ship with snapd binary15:59
zygaondra: or just in the source repo15:59
ondrazyga just in source15:59
ondrazyga it is reference which should be used by bootloader15:59
zygaI see15:59
Chipacapedronis: fair15:59
Chipacai'm off to the gym, will bbl15:59
ondrazyga so data structure used by snapd and bootloader is in sync15:59
zygaondra: put it in a manual page16:00
zygaondra: on the forum :)16:00
ondrazyga I want it somewhere from where it's easy wget16:00
zygaondra: why?16:00
pstolowskisergiusens, mvo: both multipass and destructive-mode builds worked with type:snapd + build-base:core 👌16:00
zygawould you expect people to wget it constantly?16:00
zygabecause it changes all the time16:00
ondrazyga as this is also needed at gadget build time, when device pre-populated env should be generated16:01
zygaondra: no, it's not16:01
zygaondra: what is needed is the structure or other type it defines16:01
zygaondra: hence my question16:01
zygaondra: is it expected to change often?16:01
zygaondra: what happens when it changes16:01
ondrazyga no it does not change, but file is used by around, and I want it to be in sync16:01
zygaondra: keeping it in easy-wgettable place doesn't seem to solve any issues16:01
zygaondra: you cannot have both :)16:01
zygaondra: I think that's more important than where it is16:02
zygaondra: what happens when, eventually, it must change16:02
ondrazyga if it changes, it will likely be new file, as it will have new version16:02
zygaondra: then you don't need to wget it16:02
zygait's fine to document as if it was a manual page16:02
zygabecause it's fixed forever in stone16:02
zygajust make sure to name it, v116:02
ondrazyga so new version will be new file, this file is essentially set in stone once snapd adopts it16:02
ondrazyga that's why I want single place to get it from16:03
zygaI think that's the essential information16:03
zygathank you16:03
ondrazyga OK where can I have it, so I can wget it?16:03
zygaondra: anywhere, because it's a constant now16:03
zygaondra: just make sure not to accidentally make go build it16:04
ondrazyga so can I add it to snapd repo? :P16:04
zygaondra: go is tricky for that16:04
zygaondra: include/ perhaps (we don't have it yet)16:04
ondrazyga if I only add it there, how do I make sure it's not go build?16:04
zygaondra: don't add any go files there16:04
ondrazyga I though to have it right next to go version, so bootloader/lkenv/16:04
zygathat's not godo16:05
zyga*good16:05
zygait might get pulled for cgo build16:05
ondrazyga hmm16:05
ondrazyga so creating include directory?16:05
zygaondra: yeah, it seems like that to me16:08
ondrazyga OK I will add it there16:08
ondrazyga goog idea16:08
zyganot a guarantee it will pass16:09
zygabut let's see16:09
ijohnsonyou could be tricky and add it to testdata16:09
zygaI hope my comments were not snarky in reception16:09
zygaI really did mean to point the light on the key issue of evolution16:09
zygabecause that's the hard part of this problem16:10
ijohnsongo ignores files in testdata, but it's not really test data so _shrugs_16:10
zygasharing an agreed structure is easy16:10
=== pstolowski is now known as pstolowski|afk
zygaI'm near EOD16:11
zygalooking at apparmor bits16:12
zygaspread is spreading16:14
* ijohnson lunches16:16
mupPR snapd#7409 opened: Allow a confined Wayland server running in a user session to work with Qt, GTK3 & SDL2 clients <Created by AlanGriffiths> <https://github.com/snapcore/snapd/pull/7409>16:43
diddledanso you can now run the gimp snap in Windows: https://usercontent.irccloud-cdn.com/file/SXwtQQGd/image.png16:45
zygadiddledan: is it stable? I had issues with the x server16:58
diddledanI'm using the x server in mobaxterm16:58
mvothanks pstolowski|afk17:00
ondraChipaca please re-review my changes, I believe I have address those I commented on17:10
Saviqsergiusens: is this error expected on a Pi? https://forum.snapcraft.io/t/building-for-core18-multipass-issue/8958/3517:51
Chipacaondra: will do (not right now tho)18:19
ondraChipaca thanks :)18:19
sergiusensSaviq: I think ijohnson got the gist of it. This is why the meeting this Friday is important :-)18:21
sergiusensSaviq: I am leaning to having that same image that is produced for multipass to be used for lxd on all architectures18:21
* ijohnson hopes the meeting on friday ends in snapcraft being able to make me a sandwich 18:21
ograSaviq, sergiusens, these images from friendlyarm are an awful hackish mess (and legal should really go after them for claiming this crap is based on UbuntuCore ... ) it is some self-debootstrapped rootfs with configs manually hacked up etc and somce home-brewed BSP kernel binary18:23
Chipacaijohnson: it can already make you a sandwich! the yaml is tricky though18:24
sergiusensogra: ah, interesting, but I will not go into any legal topic on a public forum :-)18:24
ograi wouldnt judge that an "error is expected on a pi" just because it fails on these hacked up vendor images18:24
ijohnsonI bet it uses those silly yaml anchors18:24
ograsergiusens, yeah, i shouldnt either it is just the third person on the forum in a month that has issues due to this "based on core" claim ... starts to really annoy me :)18:25
Chipacaijohnson: b̥̦̪̞ͅa̞̙̫ͅs̜e̫̬̱̠͈̬̬:̡̣̦ ̙͚̤͓̜̦̫{̷̟̥̱̺ ̝̣͉̦̞̥<҉̙̣̣̰<̪̟̪̘͡:̖̖̼ ͈̘̥̮*̛͈̜̙ͅs͎̺a̦n̙͔̫͓͡d̯w̧̪͖͍͈i̝͈̗̬̣c̷̤͔̻h̹̟͚ ͕̘̗̜̕}̷̳18:30
Chipacaor something like that18:31
* Chipaca is preparing dinner and might've gotten the zalgo wrong18:31
ijohnsonperfect18:32
zygajdstrand: hey, around?19:17
zygajdstrand: https://bugs.launchpad.net/snapd/+bug/1842615 came up today, the fix is one liner policy change19:17
mupBug #1842615: Wayland apps not starting under openSUSE Tumbleweed <snapd:In Progress by zyga> <https://launchpad.net/bugs/1842615>19:17
zygaowner /run/user/*/mutter/Xauthority r,19:17
zygaI was meaning to add it but wanted to run it by you for a quick look19:18
mupPR snapcraft#2700 opened: schema: build-base support for the kernel type <Created by sergiusens> <https://github.com/snapcore/snapcraft/pull/2700>19:19
mupPR snapd#7410 opened: tests: support fastly-global.cdn.snapcraft.io url on proxy-no-core test <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7410>20:21
cachiozyga, if you are there could tu please take a look to #741020:23
mupPR #7410: tests: support fastly-global.cdn.snapcraft.io url on proxy-no-core test <Simple 😃> <Created by sergiocazzolato> <https://github.com/snapcore/snapd/pull/7410>20:23
cachioit is blocking all the tests on travis20:23
zygadone20:24
cachiozyga, thanks!!!20:24
mupPR snapcraft#2687 closed: colcon plugin: add ability to ignore packages <Created by kyrofa> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2687>21:52
mupPR snapcraft#2694 closed: multipass: fix setup exception when multipass is not found in PATH  <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2694>21:52
mupPR snapcraft#2698 closed: dirs: check for existence of required data directories <Created by cjp256> <Merged by sergiusens> <https://github.com/snapcore/snapcraft/pull/2698>21:55
* Chipaca hugs cachio 22:05

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!