| mup | PR snapd#9745 closed: [RFC] seed: enable uc20 devmode snaps in dangerous models <Bug> <UC20> <Created by anonymouse64> <Closed by anonymouse64> <https://github.com/snapcore/snapd/pull/9745> | 00:11 |
|---|---|---|
| mborzecki | morning | 06:45 |
| mup | PR snapd#9762 closed: gadget: prepare gadget kernel refs (0/N) <Skip spread> <UC20> <Created by mvo5> <Merged by bboozzoo> <https://github.com/snapcore/snapd/pull/9762> | 06:47 |
| pstolowski | morning | 08:02 |
| mborzecki | pstolowski: mvo: morning guys | 08:09 |
| mvo | good morning mborzecki and pstolowski | 08:11 |
| mborzecki | awfully quiet | 09:54 |
| * ogra rattles the chains | 10:00 | |
| pstolowski | yes i just restarted my irc client, thought it was misbehaving ;) | 10:07 |
| mborzecki | haha | 10:08 |
| mborzecki | holiday season is clearly upon us | 10:08 |
| mvo | haha | 10:10 |
| mvo | yeah | 10:10 |
| ogra | on that note ... did anyone see my question about https://forum.snapcraft.io/t/scummvm-snap-failing-to-install-on-rpi-4/21394 yesterday ? | 10:18 |
| ogra | (install hook failing because snapctl is not allowed due to "install in progress") | 10:18 |
| pstolowski | ogra: is this with stable snapd, or edge? | 10:23 |
| ogra | thats with stable ... | 10:23 |
| ogra | armhf, debian buster based OS | 10:23 |
| pstolowski | ogra: ok. i'll check this thread later (it's pretty long!), and see if i can reproduce | 10:23 |
| ogra | you need PiOS and an rpi for it though | 10:24 |
| pstolowski | ogra: ah, it's Pi specific? doh | 10:25 |
| pstolowski | ogra: anyway, i'll see if i can deduce anything from the forum posts then | 10:26 |
| ogra | yeah and sadly scummvm is one of the apps heavily promoted by the pi foundation so it could be a very typical target for a "first snap" people install on their new pi400 they got for christmas | 10:26 |
| ogra | i got HW and an install here, the error is pretty clearly some ordering problem (but obviously only happening on that HW/OS) | 10:27 |
| ogra | Dez 07 11:54:01 raspberrypi scummvm.daemon[2935]: error: error running snapctl: snap "scummvm" has "install-snap" change in progress | 10:27 |
| ogra | thats the message i get | 10:28 |
| pstolowski | ogra: i can reproduce it also with x86 vm (on focal, 2.48.1) | 10:42 |
| pedronis | pstolowski: hi, is #9429 now ready for re-review? | 10:42 |
| mup | PR #9429: o/daemon: validation sets api and basic spread test <Needs Samuele review> <validation-sets :white_check_mark:> <Created by stolowski> <https://github.com/snapcore/snapd/pull/9429> | 10:42 |
| ogra | oh, wow ... | 10:42 |
| pstolowski | so that's "good", i can play around with it | 10:42 |
| ogra | i cant reproduce it on the same system using 20.10 desktop | 10:42 |
| pstolowski | interesting | 10:43 |
| ogra | (and others in the thread see the same) | 10:43 |
| ogra | perhaps we're just lucky there though | 10:44 |
| ogra | (race wise) | 10:44 |
| alan_g | It is always fun to prove races are really fixed | 10:48 |
| mborzecki | pstolowski: interesting, can you post snap change? | 10:55 |
| pstolowski | mborzecki: https://pastebin.ubuntu.com/p/ktRYVjdndZ/ | 10:56 |
| pstolowski | snap.scummvm.daemon.service: Scheduled restart job, restart counter is at 5. | 10:57 |
| pstolowski | what is that? | 10:57 |
| mborzecki | pstolowski: hmm weird, what is the hook trying to run? | 10:59 |
| ogra | mborzecki, https://github.com/snapcrafters/scummvm/blob/master/snap/hooks/install | 11:05 |
| ogra | just "snapctl set" | 11:05 |
| ogra | (it shoudl admitedly perhaps just call "snapctl stop --disable snap.scummvm.daemon") | 11:06 |
| ogra | (and start it from the configure hook) | 11:06 |
| pstolowski | pedronis: hi, sorry, missed your question, yes | 11:14 |
| mup | PR snapd#9771 opened: boot: boot config update & reseal <UC20> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9771> | 11:23 |
| pstolowski | need 2nd review for https://github.com/snapcore/snapd/pull/9732 | 11:27 |
| mup | PR #9732: asserts: snapasserts method to validate installed snaps against validation sets <validation-sets :white_check_mark:> <Created by stolowski> <https://github.com/snapcore/snapd/pull/9732> | 11:27 |
| mborzecki | pstolowski: ogra: taht would be a configure hook calling snapctl restart? | 11:31 |
| pstolowski | i'm unclear what this snap is doing / should do, i will take a closer look later today | 11:43 |
| pstolowski | but it clearly shouldn't fail like this | 11:44 |
| pstolowski | will also check if edge fixes it | 11:46 |
| alan_g | pstolowski, I wrote the hook scripts. It just checks for and sets a default configuration option that is later read by a launch script. | 11:57 |
| pstolowski | alan_g: hi, yes, sorry, i understand that; what i mean is i don't have the big picture wrt services of this snap, i need to take a closer look | 11:58 |
| pedronis | pstolowski: reviewed | 12:00 |
| alan_g | Oh, the launch script just stops the service depending on the configuration option. | 12:00 |
| pstolowski | pedronis: ty | 12:05 |
| mup | PR snapd#9772 opened: desktop/notification: test against a real session bus and notification server implementation <Created by jhenstridge> <https://github.com/snapcore/snapd/pull/9772> | 12:09 |
| mup | PR snapd#9162 closed: gadget: change mountedfilesystemwriter to use resolvedSource (3/N) <Squash-merge> <UC20> <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/9162> | 12:14 |
| ogra | mborzecki, (sorry, was in a meeting) currently it is the install hook calling "snapctl set" to set a parameter that tells the daemon wrapper to start or not start the service | 12:28 |
| ogra | while i'm sure just moving the hook to be configure instead of install would help with the race, using snapctl from an install hook should indeed still work | 12:29 |
| mup | PR snapd#9773 opened: interfaces/apparmor: do not fail durin initialization when there is no AppArmor profile for snap-confine <Needs security review> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9773> | 13:04 |
| mborzecki | package dmitri.shuralyov.com/go/generated: unrecognized import path "dmitri.shuralyov.com/go/generated": https fetch: Get "https://dmitri.shuralyov.com/go/generated?go-get=1": dial tcp 172.93.50.41:443: connect: connection refused | 13:19 |
| mborzecki | wth? | 13:19 |
| mborzecki | why is this package even being pulled? | 13:21 |
| pedronis | I don't see anything that refers to it | 13:27 |
| jamesh | mborzecki, pedronis: it is a dependency of https://github.com/gordonklaus/ineffassign, which is run by the static checks. It looks like the domain in question has expired | 13:33 |
| mborzecki | jamesh: hm whois says it expires next yearhttps://paste.ubuntu.com/p/wpvGK4xSQr/ | 13:34 |
| pedronis | mmh | 13:35 |
| mborzecki | https://paste.ubuntu.com/p/wpvGK4xSQr/ | 13:35 |
| jamesh | mborzecki: you're right. I put in the wrong query | 13:35 |
| mborzecki | anyways, whether shady or not, it is kind of a bummer it's not on github or somehing | 13:36 |
| jamesh | and maybe it is back now? https://dmitri.shuralyov.com/go/generated | 13:40 |
| mborzecki | heh, urls in import paths | 13:48 |
| jamesh | If we switched to modules, I guess we'd avoid this by only depending on the module proxy being up | 13:56 |
| ijohnson | we would avoid many things if we could switch to modules :-) | 13:57 |
| pstolowski | ah i think i understand the issue with scummvm snap | 13:58 |
| pstolowski | it is the daemon script calling snapctl, which conflicts when the install change that is still running. and i think it might be racy and may succeed | 14:01 |
| pstolowski | ijohnson: hey, did you see my question yesterday about lxd install hook / namespace slowness? i quit irc shortly after so if you answered this, then i missed that | 14:03 |
| mborzecki | i this case it's not even us, but rather the ineffassign tool | 14:10 |
| alan_g | Starting a daemon before install completes sounds pretty racy to me! | 14:11 |
| alan_g | Is there a way the daemon script could detect this? | 14:12 |
| ogra | alan_g, why do you use a script at all ? just use the hooks directly ... make the install hook always stop the daemon, put the logic about starting it based on the setting into the configure hook | 14:25 |
| alan_g | It's the first way I found that worked. Stopping in the install hook isn't enough to deal with reboots and restarts. | 14:27 |
| ogra | alan_g, https://github.com/ogra1/pi-fancontrol-snap/blob/master/snap/hooks/install#L8 similar to this | 14:28 |
| alan_g | And IIRC snapctl doesn't do disable | 14:28 |
| ogra | alan_g, and this in the configure hook https://github.com/ogra1/pi-fancontrol-snap/blob/master/snap/hooks/configure#L16 | 14:28 |
| ogra | we use a similar setup in a lot of customer snaps and that works reliabely | 14:29 |
| ogra | *reliable | 14:29 |
| ogra | just add some extra logic to check for the setting and drop the script alrogether | 14:30 |
| alan_g | Ack. I've not seen problems until now. And didn't see your approach when I first came up with this. | 14:30 |
| alan_g | But how does your approach avoid the service starting after a reboot? | 14:31 |
| ogra | it is dsabled | 14:32 |
| ogra | the install hook calls: "snapctl stop --disable ${SNAP_NAME}.${SNAP_NAME} 2>&1 || true" | 14:32 |
| alan_g | Oh! When did that become possible? | 14:33 |
| ogra | the configure hook checks if it is inactive (and can check additionally for the setting) and then calls "snapctl start --enable ${SNAP_NAME}.${SNAP_NAME} 2>&1 || true" | 14:33 |
| alan_g | Or did I just not find the right docs? | 14:33 |
| ogra | i think that was always there | 14:33 |
| ogra | its after all just a frontend to systemd features | 14:34 |
| pstolowski | yes it has always been there ;) | 14:34 |
| alan_g | Its a long time ago, but I *wanted* to disable and never figured out how. /o\ | 14:35 |
| pstolowski | alan_g, ogra i'm in the standup, give me a moment, i've suggestion for this snap | 14:35 |
| ogra | no hurry | 14:35 |
| ogra | (before christmas is fine i think 😄 ) | 14:35 |
| alan_g | Not my snap anyway | 14:35 |
| alan_g | But I have the same logic in several of mine | 14:36 |
| pstolowski | alan_g: so, 1) yes, snapctl stop --disable will be the cleanest | 14:43 |
| pstolowski | alan_g: 2) not sure why install hook has logic around snapctl get, install hook is only run once for the first intallation of the given snap where by definition there is no configuration... Such logic should live in configure hook. | 14:45 |
| alan_g | pstolowski, it's just to make it the same as post-refresh. I didn't realise that configure would be run in both cases. | 14:47 |
| pstolowski | 3) the error we're seeing here is caused by daemon.sh calling snapctl when (re)starting during install. we currently detect such situation as a conflict so it fails. The solution to this is to use snapctl get.. to get all the configuration from configure hook and generate a config file from that (in snap data dir), and the daemon just reads the config file on start. | 14:48 |
| pstolowski | anyway, i suppose you won't need a config file after using --disable, but mentioning in case you need something more sophisticated elsewhere | 14:49 |
| alan_g | Thanks. I'll try updating one of my snaps and report back on the forum | 14:53 |
| mborzecki | alan_g: that hook will need an update to work on core20 reliably, there's no snap_core, so the check should be modified to `grep -e snap_core= -e snapd_recovery_mode=` | 14:55 |
| ogra | it would really be nice if we could have a "snapctl is-core" or something in a future release to not having to grep /proc/cmdline from packages | 14:57 |
| alan_g | @mborzecki, thanks, but that's already in my snaps. But I can mention it to the author | 14:57 |
| ijohnson | alan_g: I think the other way you could get around "snapctl get daemon" from the daemon script is to just run it in a loop until it works, that way it will start working when the install-snap change is finished | 14:58 |
| ogra | well, the daemon should really only start on core ... the snap is for both, core and desktop ... so you dont want something looping constantly in the background | 14:59 |
| alan_g | I wondered about that. But now I know how to disable from the install hook I don't think it is needed at all. | 14:59 |
| ijohnson | oh I see | 14:59 |
| ogra | --disable from install, --enable from configure and dropping the wrapper is really the cleanest solution | 15:00 |
| alan_g | I think (conditional) --disable from install and leave the user to enable if they want to covers it. | 15:01 |
| ogra | not sure what you want to make conditional in install here | 15:02 |
| ogra | i'd just install with stopped by default and do the conditional stuff from configure | 15:02 |
| ogra | (there are no conditions to check on install since you cant "snap set <snapname>" before the snap is installed) | 15:03 |
| alan_g | The condition is "if grep -q -e snap_core= -e snapd_recovery_mode= /proc/cmdline" | 15:04 |
| ogra | oh, that ... | 15:04 |
| ogra | i'D still do that from configure .. but yeah indeed | 15:04 |
| ogra | ... missed that | 15:04 |
| alan_g | Well, configure might run if the user configures something | 15:05 |
| alan_g | I just want it on install | 15:05 |
| ogra | installation calls configure once | 15:06 |
| ogra | so you dont need to duplicate code | 15:06 |
| alan_g | What duplicate code? | 15:07 |
| ogra | you'D still check "snapctl get daemon", no ? | 15:08 |
| alan_g | Why? I'd do away with the configuration option and let the user enable the daemon | 15:09 |
| ogra | so make install just disable it by default and have configure check for both conditions (on core or daemon=true) and enable it if required | 15:09 |
| pstolowski | i've summarized what i wrote above in the forum | 15:09 |
| ogra | well, i thought you want it to start on core in any case ... but also allow the user to start it as daemon on desktop optionaly | 15:10 |
| ogra | so you write a single conditional in configure and have it always come up disabled in install | 15:11 |
| alan_g | if /* not on core */; then snapctl stop --disable $SNAP_NAME.daemon; fi | 15:12 |
| alan_g | in install. Nothing in post-refresh, nothing in configure | 15:12 |
| ogra | sure and an additional chck for daemon= in configure ... | 15:13 |
| alan_g | Why? | 15:13 |
| ogra | i'm just proposing to have both conditionals in configure to have a central place | 15:13 |
| ogra | so you never need to bother about install anymore | 15:13 |
| ogra | even if conditions change | 15:13 |
| ogra | but up to you really ... i just find it a lot more elegant .. but thats personal taste 🙂 | 15:14 |
| * cachio lunch | 15:14 | |
| cachio | mvo, this is failing in debian https://paste.ubuntu.com/p/y4kCcgyrqT/ | 15:15 |
| cachio | mvo, any idea bout how to fix it | 15:16 |
| cachio | ? | 15:16 |
| mvo | cachio: oh, fun - looks like the archive is inconsistent. could you try a "apt full-upgrade -y" before the "apt build-dep" ? | 15:16 |
| alan_g | I think I'm missing your point. What condition might change? | 15:16 |
| cachio | mvo, sure, thanks | 15:16 |
| ogra | well, you just had one that changed 😉 from snap_core to snapd_recovery_mode ... | 15:17 |
| ogra | but really, do as you like ... lets not discuss style as log as we get a fix out 🙂 | 15:18 |
| alan_g | AFAICS its harder in configure as we only want to disable during an initial install | 15:20 |
| alan_g | Not on any random change | 15:20 |
| ogra | yes, thats why i'D unconditionally always disable it in install | 15:20 |
| ogra | and have all the enablement logic in configure | 15:20 |
| alan_g | The logic is "if (first time && On desktop) then disable" | 15:22 |
| ogra | just do as you like, really | 15:22 |
| ogra | both hoos run in succession anyway | 15:22 |
| ogra | *hooks | 15:23 |
| alan_g | I still feel that logic is simpler in install as you know it is first time | 15:23 |
| ogra | well, you still need the daemon= logic in configure in any case | 15:23 |
| alan_g | Why? | 15:24 |
| ogra | because your user might want to run a kiosk on classic ? | 15:24 |
| ogra | i thought thats the purpose of having daemon= | 15:24 |
| ogra | so you give additional control | 15:24 |
| alan_g | So the user enables $SNAP_NAME.daemo | 15:24 |
| mborzecki | mvo: pedronis: i've updaed #9629 to the latest version of license data | 15:25 |
| mup | PR #9629: spdx: update to SPDX license list version: 3.10 2020-08-03 <Needs Samuele review> <Simple 😃> <⛔ Blocked> <Created by bboozzoo> <https://github.com/snapcore/snapd/pull/9629> | 15:25 |
| ogra | ah, so you would ask the user to snap start --enable scummvm.daemon ... instead of snap set ... sure ... that works but wouldnt be usable from a gadget on classic | 15:25 |
| mborzecki | i suppose i can drop the blocked label now too | 15:26 |
| alan_g | "wouldnt be usable from a gadget on classic" is the point I was missing. Thanks! | 15:26 |
| ogra | not a super common case ... but possible | 15:26 |
| ogra | (up to now we talked all diigtal signage users into using core anyway 🙂 ) | 15:27 |
| mborzecki | ijohnson: a quick observation, i was able to reproduce the rsa veirification error quite reliably every couple of runs when i was building a kernel with yocto in the background | 15:37 |
| mvo | mborzecki: thank you! | 15:39 |
| ijohnson | mborzecki: interesting | 15:47 |
| ijohnson | perhaps it is so difficult to reproduce for me because I have so many cores that are not busy :-p | 15:47 |
| mborzecki | 2020-12-09T14:35:03.9833068Z Dec 09 14:32:59 ubuntu snapd[1702]: 2020/12/09 14:32:59.119476 stateengine.go:150: state ensure error: devicemgr: cannot mark boot successful: cannot check for fde-setup hook in reseal: cannot get kernel info: no state entry for key | 16:09 |
| mborzecki | weird | 16:09 |
| mborzecki | mvo: any clues what that might be about? ^^ | 16:09 |
| mborzecki | hm found more weird logs: | 16:10 |
| mborzecki | 2020-12-09T14:35:03.9588849Z [ 55.176506] snapd[1702]: 2020/12/09 14:33:01.515032 stateengine.go:150: state ensure error: devicemgr: cannot mark boot successful: cannot identify kernel snap with bootloader grub: cannot read dangling symlink kernel.efi | 16:10 |
| mborzecki | looks like this happens right after install too, this appears when booting into run mode for the first time: | 16:13 |
| mborzecki | 2020-12-09T14:35:03.8893265Z [ 32.599045] snapd[885]: stateengine.go:150: state ensure error: devicemgr: cannot mark boot successful: cannot check for fde-setup hook in reseal: cannot get kernel info: no state entry for key | 16:13 |
| N3bulaK | Hey guys | 16:18 |
| N3bulaK | I have been referred to ask a question here | 16:18 |
| N3bulaK | I am trying to copy a snap to another machine with a different username | 16:18 |
| N3bulaK | if I just copy the folder over, that doesn't work | 16:19 |
| ogra | technically you should install the snap newly, take a snapshot on the old host and restore it on the new host ... but that will likely not handle changed user name or changed UID for user data | 16:20 |
| ogra | perhaps someone with more insight into snapshots can give a hint if it is possibel to restore snapshots to a new user | 16:22 |
| N3bulaK | BTW, snap is question is bluemail | 16:23 |
| N3bulaK | ogra: tried that but that doesn't work due to username | 16:23 |
| N3bulaK | :( | 16:23 |
| ogra | you will definitey need to install the snap anew ... you can surely also restore the system bits from a snapshot ... perhaps then copying the ~/snap/bluemail/current/* content is enough | 16:24 |
| mvo | pedronis: I have this feeling that 9149 has too much in it, it's a bit messy, should I split it into one PR that does the "$kernel:ref" validation, one PR that implements gadget.ResolveContentPaths() and one that uses ResolveContentPaths() ? wdyt? | 16:25 |
| pedronis | mvo: that's fine with me, it's not very large, but that sequence seems easier to review | 16:26 |
| mup | PR snapd#9149 closed: gadget: provide new gadget.ResolveContentPaths() (2/N) <Needs Samuele review> <Squash-merge> <UC20> <Created by mvo5> <Closed by mvo5> <https://github.com/snapcore/snapd/pull/9149> | 16:30 |
| mup | PR snapd#9774 opened: o/snapshotstate: don't set auto flag in the snapshot file <Needs Samuele review> <Created by stolowski> <https://github.com/snapcore/snapd/pull/9774> | 16:35 |
| pstolowski | pedronis: ^ | 16:39 |
| pedronis | pstolowski: thx | 16:41 |
| * ijohnson short break | 16:56 | |
| alan_g | pstolowski, would there be the same problem with using `snapctl is-connected` in a launch script? | 17:00 |
| ogra | why would you do that from a launch script instead of a hook ? | 17:07 |
| ogra | alan_g, https://github.com/ogra1/pi-fancontrol-snap/tree/master/snap/hooks ... se the connect hooks | 17:09 |
| ogra | (and how the configure hook uses is-connected alongside) | 17:09 |
| alan_g | I've existing scripts that check for the wayland and x11 interfaces to figure out how to launch | 17:10 |
| ogra | i doubt it makes any difference what comes after snapctl ... the call itself is the issue | 17:10 |
| alan_g | I suspect as much too. But hoped... | 17:11 |
| alan_g | So configure runs on connect/disconnect? | 17:11 |
| ogra | no, the connect hooks do | 17:12 |
| ogra | configure just uses is-connected and exits zero if a connection is missing | 17:12 |
| ogra | before it starts (or restarts) the service .... | 17:12 |
| * alan_g is tempted to keep calling snapctl until it works | 17:13 | |
| ogra | in a crazy loop 🙂 | 17:15 |
| pstolowski | alan_g: no, that should be fine | 17:17 |
| pstolowski | alan_g: but also i was slightly wrong about the source of the conflict, it's actually 'snapctl stop ..' in the daemon-start.sh (not snapctl get) triggering this (it's fine to do this from hooks, but in daemon it conflicts with install as explained earlier) | 17:19 |
| pedronis | it seems we have tests that generate real notifications? | 17:20 |
| alan_g | pstolowski, that seems less awkward. But that means a daemon can't stop itself in the case of persistent problems? | 17:22 |
| pedronis | it can but needs to deal with conflict errors | 17:23 |
| pedronis | maybe we need those to be more detectable | 17:23 |
| alan_g | until snapctl stop --disable $SNAP_NAME.daemon; do sleep 1; done # Ugh! | 17:28 |
| pstolowski | hmm | 17:31 |
| pstolowski | why the loop? | 17:31 |
| alan_g | to deal with conflict errors | 17:32 |
| alan_g | Or have I misunderstood the failure mode? | 17:32 |
| pstolowski | alan_g: if you do this from hook then it should just work | 17:33 |
| pstolowski | i.e. won't conflict | 17:33 |
| alan_g | But the hook doesn't know that the daemon has encountered a persistent error | 17:33 |
| mup | PR snapd#9775 opened: gadget,o/devicestate,tests: drop EffectiveFilesystemLabel and instead set the implicit labels when loading the yaml <Cleanup :broom:> <Run nested> <UC20> <Created by pedronis> <https://github.com/snapcore/snapd/pull/9775> | 17:35 |
| alan_g | But it could `snapctl set killme=true` and the configure hook would process that? | 17:36 |
| pstolowski | sorry, i need to run, need to taxi my daughter, let's talk tomorrow (oe maybe ijohnson can help) | 17:39 |
| pstolowski | o/ | 17:40 |
| * ijohnson is back | 17:46 | |
| ijohnson | alan_g: I'm a bit confused where you're at right now | 17:47 |
| alan_g | ijohnson, I understand the immediate problem and solution. But I'm just imagining the hypothetical circumstance of a daemon that hits a persistent problem at runtime and elects to stop itself. In that case it is necessary to "deal with conflict error". I see two ways to do that: | 17:50 |
| alan_g | 1. until snapctl stop --disable $SNAP_NAME.daemon; do sleep 1; done | 17:50 |
| alan_g | 2. `snapctl set killme=true` and the configure hook would process that? | 17:51 |
| ijohnson | so just to make sure we are on the same page, `snapctl stop --disable ...` needs to be run in a loop because if the daemon runs very fast, snapctl may fail due to an conflict in progress ? | 17:52 |
| ijohnson | i.e. install-snap in progress or some such error message | 17:52 |
| alan_g | Yes | 17:53 |
| alan_g | It's not blocking anything right now. Just want to confirm my understanding. | 17:54 |
| ijohnson | ok, then imho having `snapctl stop --disable` run in a loop until it works is the cleaner solution | 17:56 |
| ijohnson | I think there is maybe things we can do in snapd to make `snapctl stop --disable` work when there is a conflict like this, but it's unclear how exactly that would be implemented | 17:56 |
| ijohnson | I guess just as a user seeing `snap get <your-snap>` and seeing `killme: true` would be a bit unexpected and confusing | 17:57 |
| ijohnson | oh wait actually you can't do that | 17:57 |
| ijohnson | because when you run `snapctl set` that does _not_ trigger the configure hook to run | 17:58 |
| ijohnson | so `snapctl set killme=true` would be racing with the configure hook itself and would fail anyways | 17:58 |
| ijohnson | *could | 17:58 |
| alan_g | The problem with the loop is that it isn't obvious it is needed and "just works" without most of the time. (Until a user hits a weird problem on some new device.) | 18:01 |
| ogra | but why do you need it at all ? | 18:02 |
| ogra | the hooks offer everything ou need | 18:02 |
| ogra | and they save you from having to use a wrapper at all usually | 18:02 |
| alan_g | ogra, I understand the immediate problem and solution. But I'm just imagining the hypothetical circumstance of a daemon that hits a persistent problem at runtime and elects to stop itself. The hooks are not running. | 18:03 |
| ijohnson | well sometimes things are not obvious, that's what code comments are for :-P | 18:04 |
| === ijohnson is now known as ijohnson|lunch | ||
| ogra | well, thats something you'd probably manage via an additioanl watcher service then | 18:04 |
| * alan_g hits EOD | 18:06 | |
| mup | PR snapd#9776 opened: gadget: add validation for "$kernel:ref" style content <UC20> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9776> | 18:15 |
| === ijohnson|lunch is now known as ijohnson | ||
| mup | PR snapd#9777 opened: gadget: add gadget.ResolveContentPaths() <UC20> <Created by mvo5> <https://github.com/snapcore/snapd/pull/9777> | 18:50 |
| * cachio afk | 19:29 | |
| ijohnson | pedronis: do we have any current assertions or assertion examples where a list is empty? it seems to me that we have no such example and I can't seem to convince the assertion decoding function to understand what an empty list is, which leads me to believe that the assertion format doesn't support empty lists and only allows fields to be omitted if they are empty | 19:49 |
| ijohnson | indeed, if I try signing a system user assertion json with an empty string for serials, serials is just omitted from the produced assertion | 19:49 |
| N3bulaK | @ogra tried copying the /current/* but still nothing :( | 20:18 |
| pedronis | ijohnson: yes, empty and omitted are equivalent | 20:18 |
| ijohnson | ok | 20:19 |
| ijohnson | thanks for clarifying | 20:19 |
| N3bulaK | is there a way I can backup snaps to be deployed on another machine with a different username | 20:19 |
| ogra | N3bulaK, so yu installed a fres snap from the store, took a snapshot of the system config of the old one and copied the content of current/* (making sure all "dot dirs are included) ? | 20:20 |
| ogra | *fresh | 20:20 |
| ogra | N3bulaK, also whats the actual issue you see with that ? is just data missing, does the app not start etc etc | 20:20 |
| pinusc | Hello! I'm having some trouble with my snap install of LXD | 20:28 |
| pinusc | I think the problem is that I configured LXD to set storage pools in /data/lxd (note: /data is a btrfs drive), and snap is not mounting them in /var/lib/snapd/hostfs/data/lxd, which apparently is where lxd expects them | 20:29 |
| pinusc | The strange thing is, this setup worked flawlessly two weeks ago! Then I shut off the server for a while, powered it on today, and lxd won't even start anymore | 20:30 |
| pinusc | Here's the output of `sudo lxd --debug --group lxd`: https://l.termbin.com/rlaiu | 20:30 |
| mup | PR snapd#9479 closed: tests: replace pkgdb.sh (library) with tests.pkgs (program) <Created by zyga> <Merged by sergiocazzolato> <https://github.com/snapcore/snapd/pull/9479> | 20:36 |
| ogra | stgraber, ^^^ | 20:38 |
| stgraber | pinusc: your error is about: EROR[12-09|20:11:51] Failed to start the daemon: Failed initializing storage pool "default": Failed to mount '/var/lib/snapd/hostfs/data/lxd/common/lxd/disks/default.img' on '/var/snap/lxd/common/lxd/storage-pools/default': not a directory | 20:58 |
| stgraber | pinusc: oh, I see | 20:58 |
| pinusc | Yes | 20:59 |
| stgraber | pinusc: what the tell is this mess, we don't support disks/ being anywhere other than /var/snap/lxd/common/lxd/disks/ | 20:59 |
| pinusc | I honestly have no idea | 20:59 |
| pinusc | This was my first time installing lxd and I'm very confused about configuring storage | 21:00 |
| stgraber | pinusc: Does /data/lxd/common/lxd/disks/default.img exist on your system? | 21:00 |
| pinusc | Yes | 21:01 |
| pinusc | I'm not sure I exactly understand how snap works, but is /var/lib/snapd/hostfs supposed to contain some sort of bind mount of the root fs? Because right now it's completely empty, which I think is the problem | 21:02 |
| stgraber | what does `sudo nsenter --mount=/run/snapd/ns/lxd.mnt ls -lh /var/lib/snapd/hostfs/data` show you? | 21:02 |
| stgraber | you can't see the content of /var/lib/snapd/hostfs from outside the snap, that's normal | 21:02 |
| pinusc | It shows me the contents of /data | 21:04 |
| stgraber | what does `sudo nsenter --mount=/run/snapd/ns/lxd.mnt readlink -f /var/lib/snapd/hostfs/data/lxd/common/lxd/disks/default.img` show you? | 21:05 |
| stgraber | and `readlink -f /data/lxd/common/lxd/disks/default.img` without the nsenter stuff for good measure | 21:06 |
| pinusc | the nsenter one prints /var/lib/snapd/hostfs/data/lxd/common/lxd/disks/default.img | 21:06 |
| pinusc | Without nsenter it prints nothing and exits with 1 | 21:07 |
| stgraber | what does `sudo nsenter --mount=/run/snapd/ns/lxd.mnt ls -lh /var/lib/snapd/hostfs/data/lxd/common/lxd/disks/default.img` show you? | 21:08 |
| pinusc | -rw------- 1 root root 11G Nov 18 21:19 /var/lib/snapd/hostfs/data/lxd/common/lxd/disks/default.img | 21:09 |
| stgraber | ok, so that environment looks happy enough now, what does `lxc info` show you? | 21:12 |
| pinusc | Error: Get "http://unix.socket/1.0": dial unix /var/snap/lxd/common/lxd/unix.socket: connect: permission denied | 21:13 |
| pinusc | Which is expected because lxd won't start at all | 21:13 |
| mup | PR snapd#9778 opened: asserts/repair.go: add "bases" and "modes" support to the repair assertion <UC20> <Created by anonymouse64> <https://github.com/snapcore/snapd/pull/9778> | 21:16 |
| N3bulaK | ogra: App doesn't start, I have tried copying the folders etc but to no avail | 22:33 |
| N3bulaK | but If I remove the copied folders then it works fine without any previous data | 22:33 |
| pinusc | stgraber: any more ideas on what I could try? Sorry for the insistence, I'm completely lost | 23:06 |
| stgraber | pinusc: tried `sudo lxc info`? | 23:07 |
| pinusc | Yup, same Error: Get "http://unix.socket/1.0": dial unix /var/snap/lxd/common/lxd/unix.socket: connect: connection refused | 23:07 |
| stgraber | pinusc: ah, that's better than the permission denied | 23:07 |
| stgraber | pinusc: try `systemctl restart snap.lxd.daemon` | 23:07 |
| stgraber | well, `sudo systemctl restart snap.lxd.daemon` | 23:08 |
| pinusc | The command exits cleanly, but the service fails soon | 23:09 |
| pinusc | Oh, there's a (new?) error | 23:10 |
| pinusc | Failed initializing storage pool \"default\": Source path '/var/lib/snapd/hostfs/data/lxd/common/lxd/disks/default.img' isn't btrfs" | 23:10 |
| stgraber | I really wonder how you managed to get yourself into such a broken situation in the first place, LXD shouldn't have ever let you put default.img anywhere other than /var/snap/lxd/common/lxd/disks/ | 23:11 |
| pinusc | That's a very good question | 23:12 |
| pinusc | The weird thing is, it used to work | 23:12 |
| stgraber | the first failure you got was because LXD started before your /data mount was mounted, now you're hitting an error because LXD assumes that any source path outside of /var/snap/lxd/common/lxd/disks refers to a block device or a path, which your setup definitely doesn't match | 23:12 |
| pinusc | I *might* have created the default.img and then moved it somewhere else and then changed the config to reflect that | 23:12 |
| stgraber | hmm, yeah but LXD wouldn't have let you change the source property, it's read-only. The only way you could make such a mess short of us having a bug that let you do it another way is through `lxd sql` by directly updating the DB | 23:13 |
| pinusc | That I am sure I did not do | 23:13 |
| stgraber | anyway, do you have enough space on /var/snap/lxd/common/lxd/disks to store that default.img file? | 23:14 |
| N3bulaK | I will ask the question again as there are more people active at this very moment :D | 23:15 |
| N3bulaK | I am trying to copy snap data to another machine with a different username but can't make it work | 23:15 |
| pinusc | stgraber: yes, I do | 23:15 |
| N3bulaK | snap is question is bluemail | 23:16 |
| stgraber | pinusc: ok, then move it where it should be at /var/snap/lxd/common/lxd/disks/default.img | 23:16 |
| N3bulaK | in* | 23:16 |
| stgraber | pinusc: do you have more than one storage pool configured? | 23:16 |
| pinusc | stgraber: Also, I went through my .bash_history and I did indeed mv the directory under /data/lxd, and then the next relevant command is `sudo lxc storage edit default` | 23:16 |
| pinusc | stgraber: nope | 23:17 |
| pinusc | No idea what I did in storage edit, but I guess just changed the path? | 23:17 |
| stgraber | pinusc: yeah, apparently there's a bug that lets you change it... I'll have someone sort that out tomorrow | 23:18 |
| pinusc | Oh, that would be good... I just assumed that this setting was fina | 23:18 |
| pinusc | Are you a lxd maintainer? | 23:18 |
| stgraber | pinusc: onece you have default.img moved back where it belongs, you can create a file at "/var/snap/lxd/common/lxd/database/patch.global.sql" containing "UPDATE storage_pools_config SET value='/var/snap/lxd/common/lxd/disks/default.img' WHERE key='source';". Then restart LXD. The database should get updated with the correct path and hopefully things will start back up. | 23:20 |
| stgraber | pinusc: I'm the LXD project leader. | 23:20 |
| pinusc | Oh wow, thank you for helping | 23:22 |
| pinusc | The lxd daemon now starts up fine! | 23:22 |
| pinusc | Though containers fail to start for some reason... | 23:22 |
| pinusc | I'll see if I can debug that | 23:22 |
| pinusc | stgraber: I'm getting Failed to mount rootfs "/var/snap/lxd/common/lxd/containers/synapse/rootfs" onto "/var/snap/lxd/common/lxc/" with options "(null)" | 23:36 |
| pinusc | When I try to launch a (existing) container | 23:37 |
| pinusc | New containers, however, run fine | 23:37 |
| stgraber | pinusc: what's `ls -lh /var/snap/lxd/common/lxd/containers/` showing you? | 23:37 |
| pinusc | Links to /var/snap/lxd/common/lxd/storage-pools/default/containers/CONTAINERNAME | 23:38 |
| stgraber | ok, that part is good then | 23:38 |
| stgraber | ls -lh /var/snap/lxd/common/mntns/var/snap/lxd/common/lxd/storage-pools/default/ | 23:38 |
| pinusc | Some dirs, including containers/ | 23:39 |
| pinusc | Oooh, inside containers/ is one dir per container, but the owner might be wrong. I have root:root for everything, except the one I just created (which is the only one which works), which has 1000000:root as permission | 23:41 |
| pinusc | Also, the old ones are empty, except for a backup.yaml, whereas the new one has other stuff---including rootfs | 23:44 |
| stgraber | Can you check if you see anything at `/var/snap/lxd/common/lxd/storage-pools/default`? you shouldn't but given the current mess, it's not impossible that some of the data ended up there somehow? | 23:46 |
| pinusc | Nope, empty | 23:47 |
| stgraber | pinusc: oh, I think I may know what happened but you're not going to like it | 23:52 |
| stgraber | pinusc: were those containers created after default.img got moved but prior to the next system reboot? | 23:53 |
| pinusc | Very likely | 23:53 |
| stgraber | pinusc: and are /data and /var/snap/lxd on different partitions? | 23:53 |
| pinusc | Yes | 23:53 |
| stgraber | right, then I'm afraid that you're screwed. You see there is no such thing as moving a file between two mounts, when you `mv` between two mounts, the source is copied and then deleted. In your case, the source was still mounted and actively being used. When that happens Linux succeeds in deleting the file but actually keeps it active on disk until such time as the last thing that has it open closes it. | 23:55 |
| stgraber | pinusc: so when you moved default.img, LXD never actually used the moved path in /data, instead it just kept using the now delete file under /var/snap/lxd | 23:55 |
| pinusc | Ooh, I see | 23:55 |
| stgraber | pinusc: after a reboot, the data in /var/snap/lxd is gone forever and your data in /data is effectively a copy of a very old state | 23:56 |
| pinusc | So that also explains why it was working before. On a reboot, it tried to actually access /data for the first time, and failed | 23:56 |
| pinusc | I have to say, it sucks that I lost all that was in the containers... but this is a satisfying answer | 23:57 |
| pinusc | I was dumb and I got bitten | 23:57 |
| pinusc | Before I proceed, I'll make sure to actually read the documentation and properly set up a storage pool on an external media | 23:58 |
| pinusc | Meanwhile, i guess I'll just have to delete my containers and start from scratch... | 23:58 |
| stgraber | the best is to have a dedicated disk or partition for LXD | 23:59 |
| stgraber | then during LXD init you will be prompted for whether you have one of those for your storage pool | 23:59 |
| stgraber | LXD will then automatically mount it for you on startup all inside its mount namespace | 23:59 |
| pinusc | Yeah, I guess I'll make a btrfs subvolume | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!