=== utlemming_away is now known as utlemming === chihchun_afk is now known as chihchun [07:00] good morning [07:05] good morning [07:20] Morning === zyga_ is now known as zyga [09:06] ogra_: Hi, I looked into not using the py3 from the snappy-core image this morning by providing something better in snapcraft. but it seems like this is not trivial as there is no offical binary build for python . what did you do for py-snapper here? [09:07] mvo, nothinng, since snapcraft is now supposed to grow that we decided to drop py-snapper [09:07] ogra_: ok - I was wondering if you had investigated if there are semi-official py3 builds we could use [09:07] mvo, but else i would just have re-used the nodejs install bits from node-snapper (without the npm parts) [09:13] Good morning all; happy Monday, and happy Embrace Your Geekness Day! 😃 === chihchun is now known as chihchun_afk [10:45] good morning [10:46] mvo: the lack of a python binary is complicated, can't we debuild a custom one and use that with snapcraft? [10:55] why cant you use the one from the archive ? [11:00] ogra_: it was hard to relocate without a lot of work from what I saw [11:01] weird [11:06] sergiusens: yeah, its tricky, it seems building a custom one is indeed the best option, its actually not that terrible, took ~2min only on my box (which is relatively fast but not super fast) [11:07] mvo, and now build it for armhf :P [11:07] ogra_: *cough* [11:08] also, isnt there a way to do a static build of the interpreter ? [11:09] we should perhaps just have a python3-static package in the archive that we can consume [11:12] so, is it possible to build aargh64 images yet? [11:13] * tbr started playing with his dragonboard [11:13] there are still packages missing i thinnk [11:13] the linaro ubuntu desktop image is a bit 'special needs' [11:15] sergiusens: ogra_: static micropython ftw [11:15] +1 [11:16] sergiusens: ogra_: https://micropython.org/ [11:16] or, well, https://github.com/micropython/micropython [11:16] is it fully py3 compatible ? [11:16] no [11:17] sad [11:17] mostly around things that don't make sense on a PIC [11:17] but the unix port is zippy :) [11:17] also i somewhat trust contributor #5 [11:18] anyway, that was in jest [11:18] i need help with mir :( [11:18] you trust him ? really ? [11:18] :) [11:19] well, somewhat, as i say [11:20] Chipaca: he looks a lot like you! Might have stolen your identity ;) [11:21] sergiusens: probably not me; i'm a lot greener and paler right now [11:22] is that a mir sideefect ? [11:27] ogra_: yes [11:27] ogra_: hazardous work environment [11:27] ogra_: i should sue [11:27] heh [11:27] ;-p [11:27] did kgunn have that working already ? [11:31] ogra_: allegedly yes [11:33] but only the deb2snap way [11:33] that is, the failing is mine [11:34] tbr, btw: [11:34] The following packages have unmet dependencies: [11:34] ubuntu-snappy : Depends: ubuntu-core-security-utils but it is not installable [11:34] E: Unable to correct problems, you have held broken packages. [11:34] (arm64) [11:34] ogra_: ok, so I won't bother for now :) [11:37] ogra_: this http://paste.ubuntu.com/11872034/ is mostly working for cross-build, but make install fails with some obscure ensurepip error, I'm not sure its worth debugging, its seems like a less than ideal path forward [11:39] mvo, well, why not have a statically built version in the archive so you could just consume the binary from there [11:40] * ogra_ isnt a fan of building on-demand if we can have archive binaries [11:48] ogra_: indeed, the archive is probably the best option [12:20] kgunn: hiya [12:20] kgunn: does mir always load xkb? [12:23] Chipaca: for that, might wanna ping anpok or alf in #ubuntu-mir [12:24] * Chipaca goes with alf, because anpok isn't there afaict [12:48] good morning. [13:41] ogra_: oh, re ubuntu-core-security-utils, I just need to update it-- seccomp in wily now supports arm64 [13:41] * jdstrand does quick upload [13:41] whee ! [13:41] not getting the image build failure mails every day would be so awesome :) [13:41] jdstrand: \o/ [13:44] \o/ on no failure emails === chihchun_afk is now known as chihchun [13:48] well, they kind of were an reliable notifier that the build has run :) [13:56] fgimenez_: after doing ifup, I'm able to ssh into rolling edge. But how are you running the tests? [13:59] elopio, whats the content of /etc/network/interfaces.d ? is there more than one file ? [14:00] ogra_: only one, ens3. [14:00] ok [14:00] ogra_: I think we established that that file needs to be created way earlier in the boot process [14:01] udevadm trigger --sysname-match=ens3 ... [14:01] http://paste.ubuntu.com/11872571/ [14:01] sergiusens, thats what pitti said on friday ... we could add it somewhere in the boot process [14:02] sergiusens, i fear we wont catch the actual uevent since that will likely happen inside the initrd where we cant yet bring up the interface [14:03] so i guess having the udev trigger added to some systemd unit is best here [14:03] https://bugs.launchpad.net/snappy/+bug/1473838 [14:03] oh ! [14:04] shouldnt that have an "auto ens3" line ? [14:04] elopio, could you add such a line and check if it comes up after reboot ? [14:05] I can. Lets see... [14:05] elopio, once the instance is reachable i execute them with the -ip flag, even for localhost [14:07] yes, don't try to create config files earlier than the kernel detects devices, that's doomed to fail [14:07] just re-trigger them [14:07] oh, missing auto... [14:08] pitti: that file is created after reading /proc/net/dev so I guess the kernel has seen it [14:08] from reading* [14:11] ogra: added the auto line, rebooted and it's up. [14:11] ha ! [14:12] sergiusens, where do we add it ? [14:12] is that ubuntu-core-config ? [14:14] ogra_: firstboot [14:14] ogra_: we still have a potential race, or do we not anymore? [14:14] and where is the code for that ? [14:15] not sure we have a race .... we should try with auto added in advance [14:15] ogra_: http://bazaar.launchpad.net/~snappy-dev/snappy/snappy/view/head:/snappy/firstboot.go#L134 [14:15] ogra_: well that is surely a bug we need to fix :-) [14:15] i'm not sure there is a race [14:15] ogra_: just edit the "data := ..." line [14:18] mterry: why are you using camel case in snapcraft ? [14:20] elopio, is that considered bad? (I thought both it and underscores were valid python strategies) [14:20] sergiusens, https://code.launchpad.net/~ogra/snappy/fix-interfaces-file/+merge/264578 [14:21] mterry: well, bad is relative. But it doesn't follow pep8, so yes, it's bad :) [14:21] https://www.python.org/dev/peps/pep-0008/#function-names [14:23] elopio, I see that link, but the pep8 command doesn't warn about it... huh (we run that with our tests) [14:23] elopio, also I have this giant escape clause: "mixedCase is allowed only in contexts where that's already the prevailing style (e.g. threading.py), to retain backwards compatibility." [14:23] elopio, it's already the prevailing style in snapcraft! ;) [14:23] elopio, well I can switch if we want to [14:25] ogra_: you should probably remove that changelog line, we are building them on release [14:26] mterry: yeah, it's all relative to which snapcraft you are talking about... :) But if the idea was to use python because more developers will find it familiar, those developers will find the use of camelCase unfamiliar. [14:27] sergiusens, "On subsequent uploads I think it only reads 'name' and 'version'." -- are you saying that if I have a snap at version A that depends on mir and ships a foo binary, when I update it to version B, which no longer depends on mir and ships a bar binary instead, those notices aren't picked up by the store? That can't be right. [14:28] sergiusens, bah ... ok [14:28] elopio, noted, I can put it on my TODO. My poor pinky finger... [14:29] sergiusens, done [14:29] writing pretty python is painful, I know :) [14:29] s/pretty// [14:29] :P [14:30] I can help with this. I'm also not convinced about allowing the long lines either :p [14:30] mterry: when I run the tests I get: http://paste.ubuntu.com/11872692/ [14:30] am I missing something? [14:31] elopio, run on wily or add ppa:hardware-certification/public to your system [14:31] ack. [14:31] elopio, long lines I am more comfortable defending [14:32] 80 chars was already limiting a decade ago [14:32] ogra_: the goal is to have something like git-dch auto-build our changelogs for us :) [14:32] mterry: yes, I'm not going to discuss about that. It's ok, I guess... I'll just have to control the freak-pep8-follower in me. [14:32] whatever that is :P [14:32] mterry: and last pita comment, there's no README. [14:33] ogra_: its pretty nice, it takes the git history and builds a debian/changelog file from it, once you get used to it, you never want to go back :) [14:33] * ogra_ doesnt mind to use debian/changelog :) ... similar to how we use it in livecd-rootfs (thats actually my favorite way of managing a package) [14:33] elopio, a good place to add a comment about the ppa for sure :) [14:34] mvo, well, it involves git ... i still cant imagine that i "never want to go back" from git any time :) [14:35] mvo: I love the git changelog magic, do things once and automate from there [14:35] sergiusens: yeah, exactly [14:36] ogra_: its not tied to git as such, a bzr-dch would be equally possible, just noone bothered writing one [14:36] heh [14:36] yeah, nobody bothers to write anything for bzr anymore :) [15:01] ogra_: you need to fix the tests as well ;-) [15:02] pffft ... tests ... who does them anyway :P [15:15] did we resolve the /boot enospace issue? [15:15] well, sergiusens prepared an MP that would ive /boot 512M [15:15] *give [15:15] which is a bit much on a 4G image i suppose :) [15:22] right [15:22] also that doesn't resolve the existing systems upgrade right? [15:22] what do we do for those? === ubott2 is now known as ubottu [15:28] seb128: hey, is this the right place to ask about ubuntu-personal? are there docs for generating ubuntu-personal images? [15:30] jdstrand: ubuntu-device-flash personal rolling --channel edge --output personal.img [15:30] jdstrand: if not on wily you need the ppa:snappy-dev/tools-proposed [15:31] sergiusens: yeah, I tried that and got: [15:31] failed to find user uid/gid [15:31] generic-amd64 failed to install: unpack /tmp/generic-amd64709922737 to /tmp/diskimage222895386/system/oem/generic-amd64/1.4 failed with exit status 1 [15:31] * jdstrand -> steps into a meeting [15:31] jdstrand, you need u-d-f 0.26 [15:32] $ apt-cache policy ubuntu-device-flash [15:32] 0.27-0ubuntu1 [15:32] oh [15:32] hmm, works here [15:32] * jdstrand really goes to meeting [15:32] (well, used to on friday ) [15:33] jdstrand, what sergiusens said [15:33] I didn't try udf today, let me do that [15:37] jdstrand, what ubuntu-snappy version do you have? [15:56] is there some documentation on how to make a compiled snap available for both x86 and arm? [15:57] stgraber, there are some example packages ... [15:57] iirc something like the xkcd webserver [15:57] it boils down to "have a wrapper script that calls uname to set your vars to the right arch dirs) [15:59] ok [16:05] ogra_: let us know when you're able to promote the image to alpha [16:05] rsalveti, after landing meeting (30min) [16:12] ogra_: thanks! [16:14] stgraber: are you planning to add lxc templates for snappy? [16:17] sergiusens, do i need to do anything to trigger a new tarmac run on my MP ? [16:17] ogra_: reapproval [16:17] ah [16:18] @reviewlist [16:18] https://code.launchpad.net/~ogra/snappy/fix-interfaces-file/+merge/264578 | Approve: 1 (less than a day old) [16:18] https://code.launchpad.net/~fgimenez/snappy/enable-all-for-remote-testbeds/+merge/264523 | No reviews (less than a day old) [16:18] https://code.launchpad.net/~sergiusens/snappy/rollYourOwnCorePersonal/+merge/264405 | Needs Information: 1 (2 days old) [16:18] https://code.launchpad.net/~mvo/snappy/snappy-review-tools-reenable/+merge/264301 | No reviews (3 days old) [16:18] https://code.launchpad.net/~fgimenez/snappy/failover-with-fake-update/+merge/264251 | No reviews (3 days old) [16:18] do we usually do that ourselves if there was an initial approval already ? === chihchun is now known as chihchun_afk [16:18] ogra_: depends on the change, I did it just now fwiw [16:19] thx [16:26] elopio: not until apparmor supports profile stacking [16:26] elopio: for now what I'm working on is getting LXD into the snappy app store [16:31] stgraber: I saw that, it's great. [16:31] rsalveti, elopio, have fun with 15.04/alpha image 4 for armhf and amd64 :) [16:32] ogra_: thanks! [17:23] so, if I want to install rolling on my bbb, is it 15.04 edge? [17:39] beuno: it's rolling edge. [17:40] sudo ubuntu-device-flash core rolling --channel edge --oem beagleblack --install=webdm --enable-ssh --output ubuntu-rolling-snappy-armhf-bbb.img === dpm is now known as dpm-afk [17:43] elopio, thanks [17:43] I looked everywhere online and in cdimage [17:44] and couldn't find it [18:02] beuno: https://developer.ubuntu.com/en/snappy/guides/channels/ [18:02] if that is not clear, please file a bug. [18:03] elopio, right. I guess my confusion is that I was looking for an image to download [18:04] instead of running a command [18:04] beuno: yes, I think that's bad. Sometimes we use wget and sometimes ubuntu-device-flash. [18:04] beuno: we don't do releases for rolling [18:05] sergiusens, I understand releases. Why no images? [18:05] beuno: we have images for 15.04 [18:05] sergiusens, sorry, it seems you're answering a different question? :) [18:06] beuno: 'Why no images?' needs some extra words for a more specific answer [18:06] :-) [18:07] sergiusens, why are we not creating images for rolling on a certain cadence? [18:07] beuno: to avoid support questions for something that can break easily [18:08] beuno: and the fact that we have no where to do that automatically ;-) [18:08] IMO, the docs should only document u-d-f. Not cdimage.ubuntu.com. [18:09] sergiusens, it's documented already, as elopio pointed out [18:09] the latter I can understand [18:09] I'm hoping to provide a service for that once all the parts are snaps [18:35] hrm, I can't rollback on bbb from alpha 4 to alpha 3. [18:37] elopio: hm, what is the issue? [18:37] did it work with kvm? [18:37] rsalveti: it worked with kvm. I'm collecting the serial log to see what it says. [18:37] alright [18:42] http://paste.ubuntu.com/11873742/ [18:43] I flashed #3, upgraded to #4, rebooted. Then rollback, and reboot. [18:43] that paste is the serial console after I rebooted. [18:43] rsalveti: ^ [18:43] spl_load_image_fat_os: error reading image args, err - -1 [18:49] elopio: it seems that might be a red herring [18:49] elopio: and after the last reboot, which image are you running as host? [18:50] it's system-b [18:50] so would imagine image 4 [18:50] Starting kernel ... [18:50] U-Boot SPL 2014.04-00015-gb4422bd (Apr 22 2014 - 13:24:29) [18:50] rsalveti: yup. http://paste.ubuntu.com/11873784/ [18:50] so it tried booting the kernel, but it seems it just rebooted [18:50] no kernel message at all [18:51] elopio: what happens if you try to rollback again? [18:51] let me get by bbb [18:51] maybe sergiusens can give us a hand as well ^ [18:52] rsalveti: same thing. I tried three or four times. I end up in #4 every time. [18:52] hm, that sucks [18:56] it's weird that it shows no message saying why it didn't like kernel #3. [18:58] yeah, it's like it rebooted while booting the kernel [18:58] here's my test log, http://paste.ubuntu.com/11873826/ [18:58] adding earlyprintk to the kernel cmdline arguments might help [18:58] yeah, setting up my beagle now to see [19:06] rsalveti: elopio k [19:07] rsalveti: alpha, right? [19:08] sergiusens: right. flash alpha -1, update and rollback. [19:25] updating still [19:45] sergiusens: elopio: worked fine here [19:45] * sergiusens is still dd'ing [19:45] http://paste.ubuntu.com/11874036/ [19:45] elopio: either your kernel is corrupted or something similar [19:46] I'll retry. [19:46] try changing /boot/uboot/snappy-system.txt, append 'earlyprintk' after fixrtc [19:48] elopio: check the free disk space at your boot partition [19:48] but it worked fine here [19:49] I did get one FSCK0000.REC at my vfat partition [19:52] Syncing boot files is taking ages =\ [19:52] 53% used in /boot [19:53] that's fine [19:53] using 61% here [19:53] rsalveti: rollback should not take ages though [19:53] wonder why it's not the same [19:53] sergiusens: no, that was an update for an image that was already there [19:53] was trying to get back to image 4 with snappy update [19:54] hm, one problem [19:55] did an update to 4 after finishing the rollback (and booting into it), and now the kernel used at image 4 is the same from image 3 [19:55] Welcome to emergency mode! After logging in, type "journalctl -xb" to view [19:55] system logs, "systemctl reboot" to reboot, "systroot@localhost:~# [ 28.475311] libphy: PHY 4a101000.mdio:01 not found [19:55] yay :-) [19:55] [ 19.017145] FAT-fs (mmcblk0p1): IO charset iso8859-1 not found [19:55] [FAILED] Failed to mount /boot/uboot. [19:56] since the kernel was the wrong one, it couldn't find the modules [19:56] and exploded [19:57] omg, cloud-init is so slow [20:01] and now it's trying to automatically reboot again :-) [20:01] snappy autopilot triggered a reboot to boot into an up to date system-- temprorarily disable the reboot by running 'sudo shutdown -c' [20:01] The system is going down for reboot at Mon 2015-07-13 20:10:48 UTC! [20:01] earlyprintk doesn't add anything to the logs. [20:02] and yeah, kernel a and b are the same here [20:02] let me see if I can reproduce this issue again [20:02] I'm going to reflash #3. [20:02] elopio: so still unable to boot? [20:02] rsalveti, why the automatic reboot? I didn't realize snappy would do that. Is there something to switch that off? [20:02] rsalveti: we usually see the iso8859-1 issue when we have filesystem corruption [20:03] sergiusens: that is because it couldn't find a matching module [20:03] because the kernel is wrong [20:03] it booted image 4 with the kernel from image 3 [20:03] balloons: I didn't know we had an automatic reboot either :-) [20:03] rsalveti: hmmm, did we backport anything wrt that? [20:04] rsalveti, LOL.. I found that during the open house and was surprised to see it [20:04] sergiusens: backport what exactly, the feature? [20:04] or possible bug fixes around it [20:05] * rsalveti reflashing image 3 again [20:05] rsalveti: anything related to lp:snappy//partitions [20:05] * sergiusens uses double / to avoid confusion with the series notation [20:06] rsalveti: I bet this is just an untested path [20:06] sergiusens: not sure [20:06] sergiusens: right :-) [20:06] it should be easy to reproduce, trying again [20:06] flash 3 -> update -> rollback -> update -> boom [20:07] sergiusens: how it finds out the kernel that needs to be copied over when doing the 'Syncing boot files'? [20:07] that is the process that ended up copying the wrong kernel [20:07] rsalveti: I bet that is the issue [20:07] rsalveti: does that run on rollback? [20:07] rsalveti: if so, that is wrong [20:07] sergiusens: no, that was after I did the rollback, booted and called update [20:08] so, updating from 3->4 again [20:08] rsalveti: oh, bummer [20:09] rsalveti, the kernel messages are quiet because they go to the wrong console ... [20:09] they are not always quiet [20:09] (we set two console= options pointing to different console devices) [20:09] [ 0.000000] Kernel command line: console=ttyO0,115200n8 root=/dev/disk/by-label/system-a init=/lib/systemd/systemd ro panic=-1 fixrtc rootfstype=ext4 rootwait [20:09] not on bbb [20:09] oh [20:15] rsalveti: I see the bug [20:15] in the code that is [20:15] * ogra_ votes for including vfat byy default in the core kernel [20:15] sergiusens: what is the issue? [20:15] rsalveti: elopio https://plus.google.com/hangouts/_/canonical.com/bummer?authuser=1 [20:16] (regardless of the issue i mean) [20:16] elopio: if you can join ^ [20:17] trying to join. [20:25] balloons: rsalveti http://bazaar.launchpad.net/~snappy-dev/snappy/snappy/view/head:/docs/autopilot.md [20:33] this "updating boot files" should have a progress thingy. [20:36] sergiusens: https://bugs.launchpad.net/snappy/+bug/1474125 [20:36] Launchpad bug 1474125 in Snappy 15.04 "Image got in a broken state after update -> rollback -> update (wrong kernel)" [Critical,In progress] [20:39] elopio: yeah, open a bug for it [20:39] I got the same error as the first time. [20:39] now why the hell it can't rollback [20:39] keeps trying to load the kernel and aborts eventually [20:40] rsalveti: I can fix that easily but not without breaking the channel switching hack for upgrade testing [20:40] it's like the kernel got corrupted or something, but nothing really changed [20:41] rsalveti: fix the bug we just talked about [20:41] right, maybe we can update the channel switch hack? [20:41] elopio: ^? [20:41] rsalveti: I need to think about all the test cases this breaks [20:43] https://bugs.launchpad.net/snappy/+bug/1474126 [20:43] Launchpad bug 1474126 in Snappy "failed to rollback from 15.04 alpha #4 to #3" [Undecided,New] [20:44] sergiusens, rsalveti : what is the channel switch hack? Not sure what you are talking about. [20:44] sergiusens: why are we mounting partition a (when booted with b) as writable? [20:44] /dev/mmcblk0p2 on /writable/cache/system type ext4 (ro,relatime,data=ordered) [20:45] shouldnt that be a disk/by-label/ path for the device ? [20:45] why is there a /dev/mmcblk* at all [20:46] might be mounted via label I guess? [20:46] well, it should also use the by-lable path in fstab [20:46] elopio: rsalveti the mount something mvo wanted to be able to run snappy list --updates as non root [20:46] actually it's not rw, just ro [20:47] got it [20:47] rsalveti: the 'as writable' you mean /writable/cache/system right? [20:47] sergiusens: yeah, but it's not writable, mounted as ro :-) [20:48] just the path that is confusing [20:48] rsalveti: no arguments there ;-) [20:50] I didn't know about snappy list --updates. [20:50] sounds handy. [20:50] for our current testing, it doesn't matter that sudo is required, as we can call sudo anytime. [20:51] I suppose mvo had a reason for wanting that without sudo. [20:51] rsalveti: elopio I have mapped a workable solution in my head that is minimally invasive (6 lines of code sans tests) which I will work on tonight [20:53] sergiusens: cool [20:53] elopio: yeah, something is quite broken with the rollback, failed again =\ [20:53] I'm starving, so I'll go and have lunch. [20:54] checking the md5 and such now to see what happened [20:54] ping me on telegram if you need me and I'll get back. [20:54] sure, enjoy [20:55] sounds like no release today either then ... [20:55] nops, need to get these bugs fixed first [20:56] yeah [21:00] $ sudo snappy update [21:00] another snappy is running, try again later [21:00] hahah, what a weird message [21:00] there is a snappy running around here [21:01] hide ! [21:15] sergiusens, ty for the doc.. However, if it's disabled by default then we have a bug in the image, as I certainly didn't enable it. So the docs or the implementation aren't quite right [21:46] elopio: sergiusens: updated bug https://bugs.launchpad.net/snappy/+bug/1474126 [21:46] Launchpad bug 1474126 in Snappy "failed to rollback from 15.04 alpha #4 to #3" [Critical,Confirmed] [21:46] but basically the kernel/initrd got corrupted during the update process [21:47] which explains why it works sometimes [21:47] bbl, dinner