=== pbek_ is now known as pbek === pstolowski|afk is now known as pstolowski [07:18] pstolowski: good morning :) [07:23] mvo: hi, I'm going to merge #5502 today [07:23] PR #5502: many: streamline the generic conflict check mechanisms [07:28] pedronis: ok, I look at it now [07:50] pstolowski: interessting fix in the osutil/udev PR - is this a signed/unsigned issue? [07:50] s/is/was/ [07:50] mvo: yes [07:51] mvo: thx [07:51] pstolowski: interessting [07:51] mvo: it wouln't happen if my tests didn't use 0xff's to simulate invalid offset [07:52] pedronis: thank you, nice improvement, also nice to see how much cleaner it got [07:52] pstolowski: a, sweet [07:52] pstolowski: I was wondering how you get such a big offset that its a problem on 32bit [07:52] pstolowski: nice to know there was a test for it! [07:52] mvo: i'm not quite sure why it didn't panic on 64bits though. [07:53] mvo: but i added debug and it clearly showed negative number there [07:54] pstolowski: I assume its because on 64bit 0xff000000 will not wrap to a neagative number (space in the 64 bit int left). but I have not really looked at the details so I might be wrong [07:56] Chipaca: good morning [07:56] mvo: i think you're right. re your earlier question, the test parses crafted garbage data [07:56] mvo: it is! how're you today? [07:56] fun! the degraded test found that we have a sshgurad.service that is broken in our test images [07:56] pstolowski: tests ftw! nice job [07:57] Chipaca: I'm well, thank you! having fun reading code and poking at our boot stuff [07:57] i made a quickfix in the PR and also proposed it upstream [07:57] Chipaca: hope you are well too? [07:57] pstolowski: ta [07:58] tests ftw indeed [07:58] Chipaca: I'm also loosing hair over the question *why* the snap-run-hook test fails on core18 [07:58] mvo: yep! looking forward to the day [07:58] mvo: SCSI Chain overterminated [07:58] Chipaca: *cough* those were the days [07:59] * Chipaca thinks the bofh excuse database needs updating [07:59] nb, it's interesting Go chose to return int from len() [07:59] pstolowski: uint is a second-class citizen [08:00] pstolowski: fun, there is even a stack-overflow question about why go is doing that (and the answer even makes some sense) [08:09] mvo: do you have a link? [08:10] pstolowski: https://stackoverflow.com/questions/39088945/why-does-len-returned-a-signed-value [08:12] mvo: thanks, interesting explanations [08:14] pedronis: with 5502 landed, should I work on snapshotstate, or should i wait for a followup? [08:14] pedronis: and what about #5474 [08:14] PR #5474: many: finish sharing a single TaskRunner with all the the managers [08:18] hmm interesting failure of google:ubuntu-16.04-64:tests/main/interfaces-accounts-service [08:18] https://www.irccloud.com/pastebin/XyMk8wi7/ [08:19] quoth the raven, WAT [08:37] meh, I get quota CPUS exceeded [08:37] in my latest PR [08:50] mvo: have we alerted people about the sshguard service thing? [08:51] Chipaca: not yet, might be related to our image [08:51] Chipaca: I mean to our version of the cloud-image [08:51] mvo: ah, i thought we were using stock [08:51] Chipaca: I'm not sure, I wanted to check with cachio [08:51] Chipaca: its slightly strange because sshguard is universe [08:52] ah [08:58] https://forum.snapcraft.io/t/snap-refresh-fails-no-such-file-or-directory/6413 [08:58] If someone has a moment to look at that I'd appreciate it [09:02] popey: https://i.imgur.com/iJoG4Ks.jpg [09:02] IKR [09:02] (I have seen this before, but only now have captured it) [09:04] must be all these airplanes interfering with the cosmic rays at your place [09:04] popey: the most WAT bit is that after the Error, r319 is active [09:04] fun, right? [09:04] i.e the one it just said didn't exist [09:04] waiiit [09:04] this isn't even a manky machine that's been running forever with all kinds of crap on it [09:05] feck [09:05] it's a vm I woke up to test something unrelated [09:05] I mean, [09:05] it looks like we're adding the prerequisite N times [09:05] that can't be good [09:09] so this is a snapd bug? :) [09:09] 3 times [09:09] yes [09:10] \o/ [09:10] I mean, oh dear. [09:10] Thanks for looking at it. [09:10] popey: where's my emoji for person-blowing-a-raspberry [09:10] colon thorn [09:10] :þ [09:11] >:þ then [09:11] aka the angry lisping viking [09:11] shall I file a bug in snapd? [09:11] popey: let me dig a little bit more first please [09:11] okeydokey [09:12] my first dumb test of printing the name of the prerequisite when we decide to install it (after checking it wasn't due to be installed) shows that we're installing things that are already due to be installed [09:12] ooooh [09:12] this is probably because that snap is the default provider for a bunch of stuff [09:12] innit? [09:13] ¯\_(ツ)_/¯ [09:15] * Chipaca talks to itself [09:18] popey: yep, just repro'd it, on install [09:19] yay [09:19] still digging further [09:19] (there is a race, which is why you're not seeing it all the time) [09:21] so, yeah [09:21] we r dumb [09:22] popey: I'll work on a fix, but feel free to file a bug [09:22] kk [09:22] thanks [09:35] popey: interestingly the refresh succeeded, also [09:35] I mean, the change failed, but everything installed [09:35] yeah, when I have seen this in the past, I run "snap refresh" and it says it has nothing to do [09:36] yep [09:36] also also, in other news "snap refresh" and "snap install" now take a phenomenally long time to actually do anything [09:36] I feel like I'm staring at a broken computer when i run snap refresh or install these days [09:36] :-( [09:36] popey: I know, they used to be so zippy! :-( [09:36] interfaces code is _slow_ [09:37] 14 seconds to do nothing [09:37] i just refreshed then tried to refresh again, 14 seconds for the second run [09:37] popey: wait, it takes that long to say 'nothing to do'? [09:37] yes [09:37] that's not what I thought you meant [09:37] let me try [09:37] Chipaca: I will merge trunk in the follow ups, they will need review but you can start working on snapshot, you basically need to use AddAffectedSnapsByAttr [09:37] https://www.irccloud.com/pastebin/JadUttj8/ [09:37] pedronis: and drop the old kludgy conflicts stuff [09:38] yes [09:38] wtf is it doing for 14 seconds? [09:38] popey: snap list | wc -l [09:38] Chipaca: I alos need to update the wiki, there's quite a few kinds not mentioned there [09:38] popey: plz [09:38] pedronis: :-( thanks :-) [09:38] 205 [09:39] do you have SNAPD_DEBUG enabled? [09:39] god i hope not [09:39] how can I tell? [09:39] popey: nah it shouldn't slow things down, just log more [09:40] popey: (and we've asked canonicalers to have it enabled (and SNAPD_DEBUG_HTTP=3 at least iirc) so we can figure out one-off errors from the logs) [09:40] popey: “journalctl -u snapd” would have a lot of DEBUG lines [09:40] define " a lot" :) [09:40] popey: more than 0 [09:40] i haz 0 [09:41] so, not a lot [09:41] :-) [09:41] want another bug? [09:41] news to me btw that you've asked canonicalers to have debug enabled [09:41] popey: if you can (even momentarily) turn on debug, we can see where the bug is [09:41] ok [09:41] popey: I can forward you the email :-) [09:42] popey: subject «suggestion to work with debug enabled in snapd», from a year ago [09:43] unread :) [09:43] * Chipaca goes to the shed to get a 2x4 [09:44] I have enabled this before (to debug store slow download issues) and having debug switched on made the error go away [09:44] I am reluctant to enable these things as it makes my system behave differently than the people I talk to every day [09:44] normals don't enable this, and neither to devs [09:44] popey: the _only_ thing setting these debug flags does, afaik, is change the logging options [09:44] lies :) [09:44] popey: but I see your point [09:45] well, but turning it off and on is worse [09:45] it involves a restart [09:45] which is sure to change things [09:45] https://forum.snapcraft.io/t/extremely-slow-snap-downloads/4668/7?u=popey [09:45] given how snapd works [09:45] popey: snap downloads should be much faster since we switched to fastly [09:46] popey: but, snap refresh != snap downloads [09:46] i have seen user reports of slow downloads [09:46] popey: 14 seconds for snap refresh, with 200 snaps, is rather unique which is why i'd like to see logs of it to know who's being slow [09:46] https://askubuntu.com/questions/1056606/from-what-url-snaps-are-downloaded [09:46] for example [09:46] I am frustrated though, because the general response seems to be "ah well, can't do anything without debug data" [09:46] Chipaca: a review of #5515 would be appreciated [09:46] PR #5515: daemon: make sure most change generating handlers can produce errors with kinds [09:47] right, just enabled debug, now snap refresh is instant [09:47] popey: ok [09:47] popey: keep it enabled for a while and tell us if it gets slow [09:47] will do, thanks [09:47] popey: pretty please [09:48] popey: (this would be bad news, or good news, because it'd indicate a bug in a part of snapd we're itching to change) [09:48] Chipaca: ? is this a do nothing refresh [09:48] pedronis: yes [09:48] Chipaca: do we have plans to change that? [09:48] pedronis: see his pastebin [09:48] don't think so [09:49] pedronis: if it's snapd getting slow after a while because state in-memory means walking whatever bit of state takes 14+ seconds, then yes it's us [09:49] hahaha, its gone slow again already :D [09:49] popey: e-e-excellent. gib logs. [09:49] which logs, how? [09:49] popey: journalctl -u snapd [09:50] popey: from the latest 'started snapd' line [09:50] http://paste.ubuntu.com/p/syNYjKjB64/ [09:50] oh, too late, that's all of it [09:50] popey: that's not debug logs [09:50] :( [09:50] I typed what you asked [09:50] maybe snapd didnt restart [09:51] * popey tries again [09:52] my bad, line 4709 onwards http://paste.ubuntu.com/p/xKMMN3fSdg/ [09:52] popey: if « journalctl -u snapd | grep DEBUG: » is empty, it didn't work [09:52] ah [09:52] now yes :-D [09:52] haha, is that 200 stabs of the store api ? :) [09:52] oh [09:53] for the asserts refresh [09:53] assertions, it's 1 or 2 api calls per snap [09:53] we have plans to improve [09:53] that but not immediate plans [09:54] the snaps refresh api call itself is super fast: 10:51:29.410762 to 10:51:30.377175 [09:54] gold star for you :) [09:54] bah, 'super fast', less than a second [09:54] :) [09:54] dirty brown star for the store :) [09:54] it's python, less than a second is it doing its best [09:54] as a quick fix, could we do the assertion update only for non-interactive refreshes or something [09:55] * Chipaca suddenly finds himself to be evil [09:55] no [09:55] we can't [09:55] because validation [09:55] indeed [09:55] popey: could you file a bug, on snapd *and* snapstore, that having 200 snaps means refreshes take way too long [09:55] sure thing! [09:55] on it [09:55] popey: include these logs (from the last bit on), and the 'time' log [09:55] Thanks for spending time to help me debug [09:55] will do [09:55] taw [09:56] pedronis: dunno if you saw that we add prereqs twice and that breaks things [09:56] Chipaca: no [09:56] Chipaca: that's mvo actually :) [09:56] pedronis: true, that [09:56] mvo: it's all your fault! [09:56] Chipaca: meh [09:56] mvo: *all* of it [09:57] maybe [09:57] mvo: even brexit [09:57] * mvo crumbles in depair [09:57] i'm telling the bbc [09:57] it depends exactly how it fails [09:57] Chipaca: do you have a reproducer [09:57] mvo: it's racy, but yes: snap install gnome-system-monitor gnome-calculator [09:57] mvo: if it fails to repro, clean up with: snap remove gnome-system-monitor gnome-calculator gnome-3-26-1604 gtk-common-themes [09:58] theoretically there's only one prereq running at a time [09:58] basically a snap can have another snap more than once in its prereqs [09:58] and we don't check for that [09:58] Chipaca: a snap, or a pair of snaps? [09:59] ah, a snap [09:59] pedronis: just one should be enough [09:59] in fact it shold repro with just one [09:59] Chipaca: ok, once I finished wrestling with the snap-run-hook failure on core18 I'm on this one [09:59] mvo: already fixing it here [09:59] Chipaca: mvo: it's because they added themes to the gnome lib snap itself [09:59] Chipaca: \o/ even better [09:59] so now some snaps get to it through the snap and through gnome [10:00] so we have a snap that is a default provider both for a snap and a default provider of that snap [10:00] fun [10:00] should also be easy to solve [10:00] https://bugs.launchpad.net/snapstore/+bug/1782112 tada [10:00] Bug #1782112: snap refreh is slow with lots of snaps installed [10:01] pedronis: wondering whether to solve it at install time, or at work-out-prereqs time [10:01] pedronis: the latter seems saner, the former more foolproof [10:02] Chipaca: mmh [10:02] Chipaca: in theory it should work already, but the theory is fragile [10:02] apparently [10:02] I'm surprised we don't send a blob of data to the store with the list of everything we want rather than 200 separate requests. [10:02] seems terribly inefficient [10:03] popey: that is the question [10:03] Ah okay. Ignore me :) [10:03] popey: everybody has a different list so caching would be defeated [10:03] it needs bit more cleverness, some compromise [10:06] pedronis: if you could have a look at 5491 again today that would be greatly appreciated [10:06] Chipaca: I mean in theory when we install gnome itself we should detect that the themes are already being installed, that doesn't seem to work though [10:06] I appreciate I'm a bit of an outlier with 200 snaps installed :) [10:06] blazing a trail [10:06] pedronis: because we create the task list and only add it to state if the whole thing worked [10:06] * mvo hugs pop'trailblazer'ey [10:06] Chipaca: yes, but we atomically do that for each snap [10:06] or at least that's the intention [10:07] needs bit of digging [10:07] Chipaca: anyway in one of the plans we would do all of this upfront [10:08] pedronis: I can show you where we're not checking for dupes, if you want [10:08] maybe I should check for dupes at both places, for extra extraness [10:08] Chipaca: but the dupes are indirect [10:08] in this case no? [10:08] so I'm not sure how there can be a place [10:09] (afair current code) [10:09] pedronis: snapstate's defaultContentPlugProviders already gets the duplicated names [10:09] ? [10:09] from which snap [10:09] so it's silly bug? not a deep bug? [10:09] I forget but give me a moment to undo my fix and di can tell you [10:10] pedronis: it seems to be, yes [10:10] ok [10:10] I see we were assuming that a snap will not repeat the default provider [10:10] but they do [10:10] *cough* thats a silly bug indeed. thanks pedronis and Chipaca [10:12] pedronis: for gnome-system-monitor, defaultContentPlugProviders returns: gtk-common-themes, gtk-common-themes, gtk-common-themes, gnome-3-26-1604 [10:12] wonderful [10:12] seems 2.34 material even [10:12] pedronis: for gnome-calculator, gtk-common-themes, gtk-common-themes, gnome-3-26-1604, gtk-common-themes [10:12] if that is still possible [10:12] it's surprising that it works some of the time :-) [10:13] I'll fix defaultContentPlugProviders to not return dupes, and leave the slightly hairy code in handlers alone [10:13] pedronis: yes, looks like we need a .1 anyway [10:13] it'll fix this issue and not make things harder for us [10:14] Chipaca: thx [10:14] * Chipaca looks for the unit tests of defautContentPlugProviders, and fails [10:20] jdstrand, i'd appreciate a manual apoproval at https://dashboard.snapcraft.io/snaps/xdmcp-client/revisions/1/ (uses the new mir-kiosk snap with a loopback x11 plug for Xwayland, source is at https://github.com/ogra1/xdmcp-client/ ) [10:20] oooh [10:21] i plan to extend that with a few other clients and turn it into a thin-client appliance ;) (vnc, rdp ... ) [10:21] great idea. [10:22] I had no success getting an app working with mir-kiosk, so will look at yours for inspiration [10:23] it sadly needs layouts to make Xephyr happy (hardcodes paths to binaries in the code) ... for normal apps it shoudl be a lot easier though [10:26] i keep getting cannot allocate new Google server for ubuntu-18.04-64: quota 'CPUS' exceeded. Limit: 600.0 in region us-east1 from travis [10:28] pstolowski: I saw this as well and asked zyga to ask Gustavo about it. Hopefully he or cachio can help [10:28] mvo: ack, thanks [10:30] ogra_: i'm debugging the snapcraft.yaml build issue for xenial and bionic, feel free to drop it if you have anything else [10:31] ppisati, https://forum.snapcraft.io/t/linux-4-15-0-snaps/6401/5 [10:32] ppisati, and there seems to be an issue with "no such target "firmware_install" at the end of the build [10:32] ogra_: yep, i have patch similar to that [10:33] right, then only the missing firmware_install target is left .... [10:33] ogra_: 3794440b47ceada080b365e281b76313c7a2e255 [10:33] grepping the whole tree the only mention of firmware_instakk is in debian/scripts which i thought wouldnt be run ... [10:33] *firmware_install [10:43] Hello, I added some values to the environment of my machine ( proxy ) and need the snapd service to reload those, is this possible without a full restart of the service? [10:43] And second if I wish to pin a package version, is that doable? [10:43] IE disable auto refresh [10:46] boxrick: there are several knobs you can use to change when a refresh happens, but you can't disable it from the device, no [10:46] mvo, pedronis, #5522 is the fix [10:46] PR #5522: overlord/snapstate: dedupe default content providers [10:46] feel free to tag it and target it appropriately [10:47] boxrick: what're you wanting to stop refreshing, and why? [10:49] Since if and when it breaks, I want it to be in a controlled fashion. Or perhaps a checked upgrade in my dev environment before I roll it out in the real world. [10:50] Similar reasons I pin to software versions in config management [10:50] If I have a working environment, I need that to remain consistent. [10:52] boxrick: as far as I know you get that level of control when you use a branded store [10:53] boxrick: if this is for a device, you can also do revision pinning (but i'm not sure that isn't also tied to a branded store) [10:53] i think it is [10:54] (you actually pin it in the store ... ) [10:54] ogra_: k [10:54] (AFAIK at least ... :) ) [10:55] boxrick: depending on the snap, in some cases it'll be appropriate to let production track stable and have a canary tracking candidate [10:55] Any of example of how I can pin a version? [10:56] or I could just be talking to a wall [10:56] * Chipaca wanders off [10:58] Perhaps I have missed your point I apologise. [10:59] Well kind of unrelated, reloading enviroment. Any tips without a full snapdrestart? [11:02] Or some way of printing out the current environment that it has loaded up. [11:05] boxrick, i'd just try "systemctl reload snapd" ... technically that should reload the units config, parctically no idea if snapd picks it up though [11:05] *practically [11:11] Sadly not, seems I will have to poke the systemd service to find the PID then check the environment of that PID [11:12] boxrick: how did you set the environment? [11:12] Just setting /etc/environment [11:13] ( Proxy settings in this case ) [11:13] Which I then need snapd to read [11:13] And just restarting it every time is super ugly. [11:13] boxrick: 'every time'? [11:14] boxrick: (not sure why it's super ugly, but you can't change the environment of another process; not easily at least) [11:14] Well for example, I am deploying this with config management [11:15] So as part of a task it needs to make sure the environment is installed and snapd is running with it for example [11:15] well, systemctl reload should send a kill -HUP to the daemon ... and i would expect it to also process the .service file ... if snapd respects the -HUP it should then pick up the new env [11:16] Reload just bombs out with unsupported which is ashame [11:16] Not to worry anyway, I have a workaround. Its just a little long winded [11:16] ogra_: systemctl reload does not work like that [11:16] oh ? [11:17] Heya [11:17] mvo: Any good hints about why the systems were hanging? [11:17] ogra_: systemctl reload needs the service to have an ExecReload line [11:17] I'm gcing just now [11:17] ah [11:17] so it isnt sysv-init ... who'd have thought that ! :) [11:17] ogra_: and, on the other hand, sending a HUP does not cause an environment re-read :-) [11:18] depends on the daemon ... i have seen some do it [11:27] Hello, after I restart my computer I regularly get the following error when running some snap applications: 'cannot change profile for the next exec call: No such file or directory \n snap-update-ns failed with code 1: No such file or directory'. I don't know whether it's a distribution specific issue or not and I'm not able to fix it myself. Does anyone have a clue what might be the problem here? Thank you in advance. [11:28] r3r57: what's the output of "snap version" please? [11:28] popey, 'snap 2.32.9 snapd 2.32.9 series 16 solus 3.9999 kernel 4.17.3-81.current' [11:29] ahh, i think zyga mentioned this problem recently. I believe there is a workaround. [11:29] niemeyer, zyga, mvo: woohoo! https://pagure.io/flock/issue/87 [11:29] note the tag on the right ;) [11:29] yay [11:30] Hey r3r57 [11:31] There is a workaround but ikey needs to fix solid [11:31] Solus [11:31] Hey Son_Goku [11:31] zyga, I just sent the confirmation email to bexelbie for our talk [11:32] That looks very good [11:33] zyga, okay thank you, did you mind tell me the workaround or should I ask in #solus? [11:33] this is good news to wake up to after having to stay up late last night for an OpenStack server migration last night :) [11:35] r3r57: no, Just harder to type from my phone [11:35] Do this please [11:36] sudo apparmor_parser -r /var/lib/snapd/apparmor/profiles/* [11:37] magic [11:38] zyga, popey, thanks for your help, everything is working again. [11:38] and good luck with your talk ;) [11:38] Excellent, thanks zyga [11:38] Chipaca: I updated the wiki https://github.com/snapcore/snapd/wiki/REST-API/_compare/2b0fc8772a1ac9cb57585f32e24ebd9a169fe940...acf58ecbbb071c4be741431ac648f9c5f0b63202 if you could double check [11:47] mvo: hey, a question to https://github.com/snapcore/snapd/pull/5456 [11:47] PR #5456: snapstate: refuse to remove bases or core if snaps need them === pstolowski is now known as pstolowski|lunch [11:57] mvo: there? [11:57] mvo: please fetch my remote [11:57] mvo: and see if 47665f978a1d94ed44b3943a1d33f0e39be96459 makes any difference [12:22] can anyone help me understand why /proc/mounts is significantly different from inside a snap that from the host? [12:22] /dev/loop30 /snap/spotify/13 squashfs ro,nodev,relatime 0 0 [12:22] on the host [12:22] and [12:22] kenvandine: because snaps have their own private mount namespace [12:22] /dev/loop30 /var/lib/snapd/hostfs/snap/spotify/13 squashfs ro,nodev,relatime 0 0 [12:22]