[00:00] <ali1234> someone reported test results for it already :)
[00:08] <TJ-> The mailing list said Thursday
[00:09] <TJ-> haha, I thought I had a bright idea to stop the Xorg on :1 being spawned. "chmod -x /usr/bin/Xorg" -- that killed the Xorg on :0 !! I thought once a program was mapped and running that shouldn't affect it
[00:09] <TJ-> Interesing denial of service avenue :)
[00:21] <ali1234> hmm
[00:22] <ali1234> you might find this useful
[00:23] <ali1234> sometimes when i am debugging system components that you can't run directly, what i do is replace the binary with a shim that logs when it is started and the arguments and then runs the real binary
[00:23] <ali1234> this can help find race conditions if eg something is run twice accidentally
[00:23] <TJ-> yeah, I use the method sometimes
[00:24] <TJ-> I usually use a --bind mount over the original
[00:33] <ali1234> hmm i am confused
[00:33] <ali1234> alt-f1 on the login screen makes the font bigger?
[00:33] <ali1234> and alt-f2 switches to high contrast?
[00:34] <ali1234> alt-f3 = osk?
[00:34] <ali1234> but how do i get to a console?
[00:38] <TJ-> On an X server it's Ctrl+Alt+F1-Fx
[00:39] <TJ-> But when on a console VT it only needs Alt+Fx
[00:39] <ali1234> yeah, but it doesn't seem to work on the xubunut login screen
[00:39] <ali1234> it works after login
[00:39] <TJ-> works here on 3 laptops, just tested. You have to release the Alt key and re-press it though, you can't sit your finger on it
[00:40] <ali1234> not for me
[00:40] <TJ-> I've got 3 very different laotops here all are OK
[00:40] <ali1234> probably vbox being weird
[00:41] <TJ-> ahhh, yes
[00:43] <ali1234> i tried your dbus line, it just prints usage :(
[00:44] <ali1234> oh, there should be a space between / and org
[00:45] <TJ-> yeah, DBus is a pain with it's replication of naming in service, interface, and path
[00:46] <TJ-> If you ever want to understand it better, install and run the GUI d-feet which allows you to browse the system and session trees as well as execute methods with or without arguments. Great aid in figuring out how to use the dbus-send CLI
[00:47] <ali1234> so i noticed that during first login, xfsettingsd uses 100% CPU for several seconds
[00:47] <ali1234> that is something that does not happen on subsequent logins
[00:48] <ali1234> exit
[00:51] <ali1234> hmmmm
[00:51] <ali1234> got a thread to pull on...
[00:52] <ali1234> log out -> delete ~/.cache -> login - the delay comes back
[00:53] <TJ-> is it the thumbnailer ?
[00:53] <ali1234> that doesn't appear to generate anything in .cache
[00:53] <ali1234> or hasn't yet anyway
[00:53] <ali1234> oh, it could be
[00:54] <ali1234> nope
[00:54] <TJ-> does 'top' reveal what's running? or you could start "inotifywait -rm $HOME/.cache/" from a tty then login the GUI, see what files/paths are accessed/written
[00:55] <ali1234> it's the fontconfig subdirectory that triggers it
[00:55] <ali1234> or rather the absense of it
[00:56] <ali1234> i will keep pulling :)
[00:56] <ali1234> top doesn't reveal as much as i thought it might... seems like everything is running at once
[00:58] <TJ-> ahhh, yes, GUI font cache.. prerenders the glyphs first time
[00:58] <ali1234> but why so slow?
[00:59] <TJ-> takes a while if it generates bitmaps for all glyphs on all fonts and sizes configured by the theme, etc.
[00:59] <TJ-> I'm guessing but it makes sense
[00:59] <ali1234> so what you're saying is... it's ochosi's fault? ;)
[01:00] <TJ-> haha no... it could be something entirely different
[01:00] <knome> that's very likely though
[01:00] <knome> ;)
[01:01] <knome> anyway, nighty :)
[01:01] <ali1234> we had a missing icon cache before, maybe there is some font cache missing too
[01:03] <ali1234> all of the cache files in fontconfig are generated within 1 second of each other
[01:03] <ali1234> or... wait, that's minutes
[01:05] <ali1234> hmmmm
[01:08] <ali1234> remember these numbers: d0972 6333f
[01:08] <ali1234> TJ-: is there a way to make inotifywait print timestamps?
[01:10] <TJ-> yes, use the --format xxxxxxxx option (see "man inotifywait" )
[01:13] <ali1234> it seems to be tied to a specific cache file
[01:13] <ali1234> it better not be noto sans. i hate noto sans
[01:14] <TJ-> at least you've got a good lead now
[01:15] <ali1234> i'm deleting half the files and re-logging
[01:15] <ali1234> down to [89ab]* now
[01:20] <ali1234> down to 3 files now
[01:22] <ali1234> hmmmmmm... got you
[01:22] <ali1234> deleting just one font cache file delays login by 60 seconds... it's the one that starts 9b...
[01:23] <TJ->  9b89f8e3dae116d678bbf48e5f21f69b-le32d4.cache-7 ?
[01:23] <ali1234> yes
[01:23] <ali1234> NotoSerif
[01:23] <ali1234> argh
[01:23] <ali1234> and also NotoSans
[01:24] <ali1234> i knew it :(
[01:24] <ali1234> Noto is google's replacement for Droid font. it supports ALL OF UNICODE, so it is absolutely enourmous
[01:26] <TJ-> Yes
[01:26] <ali1234> 9b* is specifically the CJK variants
[01:26] <ali1234> even with how big it is, 1 minute still seems excessive
[01:26] <TJ-> how big is it though?
[01:27] <ali1234> 85MB for the stff referenced in 9b*
[01:27] <TJ-> I've got the same font selected and that file is only 44KB
[01:28] <ali1234> Noto is like 50 different font files though
[01:28] <ali1234> /usr/share/fonts/truetype/noto and /usr/share/fonts/opentype/noto
[01:29] <ali1234> hmm something is messed up here
[01:29] <ali1234> all the other fonts in opentype/ are otfs
[01:29] <ali1234> noto is .ttc files... what even are those?
[01:30] <ali1234> a "font collection" apparently
[01:30] <TJ-> treutype-collection /
[01:31] <ali1234> well, time to add my findings on the bug
[01:33] <TJ-> I wonder why it's doing that. I've just deleted the .cache/fontconfig/ directory and logged in again and it's not being recreated
[01:39] <TJ-> OK, I've found out how to run it: "fc-cache -f" ... and the cache is taken a long time to build
[01:39] <TJ-> only 1.1M but took about 30 seconds
[01:40] <TJ-> "fc-cache -vf" is more helpful
[01:40] <ali1234> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=845058
[01:40] <ubottu> Debian bug 845058 in fonts-noto-cjk "fonts-noto: Installing the package freezes all graphical applications." [Normal,Open]
[01:40] <TJ-> and it is sitting on /usr/share/fonts/opentype/noto
[01:41] <TJ-> says there's only 24 fonts in there
[01:42] <ali1234> yeah. the four collections have 8, 8, 4, and 4 fonts
[01:42] <ali1234> according to file
[01:42] <ali1234> but each one has all of the CJK unicode
[01:42] <TJ-> the source is 85MB
[01:43] <ali1234> the installed files are 85MB too
[01:45] <ali1234> the non-CJK part of Noto is at /usr/share/fonts/truetype/noto and is 26MB but it is 156 fonts
[01:45] <TJ-> fc-list -v | less  helps to see the extend of the Noto* faces
[01:46] <TJ-> What language is that system set to? Mine are set to en_GB-UTF-8 
[01:46] <ali1234> same
[01:46] <TJ-> I don't see those big files being generated
[01:46] <ali1234> what big files?
[01:47] <TJ-> oh, thought you said the cache files were large, but you were talking about the file referenced
[01:47] <TJ-> as in the /usr/share/opentype/oto
[01:47] <ali1234> right. the cache is like 50kb
[01:47] <ali1234> it just takes a really long time to generate it when you log in
[01:48] <TJ-> right, so the delay is worse on slower CPUs presumably, and/or slow I/O 
[01:48] <ali1234> fc-cache -vf takes 12 seconds
[01:48] <ali1234> most of which is spent on noto cjk
[01:48] <TJ-> is yours generating a system cache in /var/cache/fontconfig/ too?
[01:49] <ali1234> i didn't run it as root, so i guess not
[01:49] <TJ-> no, I'm wondering if any system job is also run first time, that adds to the delay
[01:50] <ali1234> deleting user's cache causes 60 seconds of delay
[01:51] <TJ-> aha, postinst
[01:51] <TJ-> /var/lib/dpkg/info/fontconfig.postinst:  fc-cache -s -f -v 1>/var/log/fontconfig.log 2>&1 || (printf "failed.\nSee /var/log/fontconfig.log for more information.\n"; exit 1)
[01:51] <TJ-> how about if you only remove the content of $HOME/.cache/fontconfig/ rather than the entire .cache directory?
[01:51] <TJ-> maybe it's several tasks doing a similar thing?
[01:51] <ali1234> that's what i did
[01:52] <TJ-> oh, thought you meant you were removing the entire .cache
[01:52] <ali1234> i only delete the file 9b89f8e3dae116d678bbf48e5f21f69b-le32d4.cache-7 - nothing else
[01:52] <ali1234> this delays login by 60 seconds
[01:52] <TJ-> so the slowness is accessing /usr/share/font/opentype/noto/*
[01:52] <ali1234> seems like it
[01:53] <TJ-> you could monitor that with inotifywait 
[01:56] <ali1234> hmmmmmmmmm
[01:57] <ali1234> i uninstalled fonts-noto-hinted (listed in the "differences" file) and the problem goes away
[01:57] <ali1234> not sure what that actually removed
[01:59] <ali1234> hmmmmmmmmmm... with that package removed... nothing is generated in ~/.cache/fontconfig
[01:59] <ali1234> it also removed xubuntu-default-settings
[02:00] <ali1234> hmmmmmmmm
[02:01] <ali1234> i reinstalled all the packages and still nothing gets generated in the cache on login
[02:08] <ali1234> time to reinstall i guess
[02:10] <Unit193> donofrio: software-properties (0.96.24.25) may help your issue.
[02:17] <ali1234> hmm
[02:19] <ali1234> there is some kind of problem with /var/cache/fontconfig
[02:28] <Woowoo678> How/where should I report a bug related to upgrading the distro from version 16.04?
[02:29] <ali1234> depends on the bug
[02:31] <Woowoo678> it looks like some residual upstart configurations break lightdm after the switch to systemd
[02:32] <Woowoo678> prevented me from logging into any sessions
[02:32] <ali1234> not sure about that
[03:05] <ali1234> oh no
[03:06] <ali1234> it isnt the mtime
[03:07] <ali1234> the cache files are corrupt
[04:48] -SwissBot:#xubuntu-devel- ::xubuntu-default-settings:: [trunk] r677 Launchpad automatic translations update. (by Launchpad Translations on behalf of xubuntu-dev)
[07:29] <knome> blog article prepared for announcing the results but don't go wild before we have a confirmation from all winners
[09:58]  * bluesabre tries being less wild
[10:01] <Unit193> https://launchpad.net/ubuntu/bionic/+queue?queue_state=1&queue_text=, bluesabre.
[10:02] <bluesabre> Unit193: woohoo numlockx fix
[10:02] <Unit193> Sort of.
[10:03] <bluesabre> ?
[10:03] <Unit193> Not accepted.
[14:08] <TJ-> My issue with a resume-time greeter vt8 Xorg constant restart - on upgrade to 18.04 nvidia-340 had replaced nouveau. Rid the system of nvidia and it works fine.
[16:17] <slickymasterWork> willem, Im almost sure you're aware of https://lists.ubuntu.com/archives/xubuntu-devel/2018-April/011634.html but in all cases here it is
[16:19] <willem> ochosi, yesterday you said "willem: thats not default adwaita, by default it has a bright panel" and asked "did you use something like gtk-theme-config?"
[16:21] <willem> I didn't. Not knowingly that is. Trying to run gtk-theme-config in terminal says "Command 'gtk-theme-config' not found, but can be installed with:"
[16:22] <willem> Still... you've got more important things to do. I thought this would be a simple fix...
[16:23] <willem> rebooting now to start testing the final beta...
[16:49] <willem> slickymasterWork, I had sort of gotten the message ;-) Testing live session now. Does not go to plan unfortunately... will report on the tracker
[16:58] <flocculant> ali1234: thanks for the font cache stuff - I did read most of it this morning - also saw you pinging in -desktop
[16:58] <flocculant> Unit193: does the numlock script definitely check for laptops with seperate kbd and turn it on in those cases? if it does could you comment on the bug for me please :
[16:59] <flocculant> I wondered why I couldn't tab to Unit in here 
[17:11] <willem> testing live session final beta, quick question: shouldn't the live session, after building up the desktop, when it finds wifi networks, pop up a message in the right top corner telling me about wifi networks being present?
[18:08] <willem> disregard my question... remembered I had another laptop I could use to test; so I found the answer myself
[20:09] <flocculant> bluesabre: I assume that we'll release this beta with the issues?
[20:10] <flocculant> don't particularly want to do that - but guessing I'll not have a choice - also don't mean that to sound like it does, but I'm too tired now to dig out the thesauraus :)
[20:11] <flocculant> read replies at stupid o'clock tomorrow
[20:43] <Unit193> flocculant: Hah, and yes it's certainly *supposed* to at least.
[20:57] <bluesabre> flocculant: I don't think we have a beta release blocker bug
[20:58] <bluesabre> Unless there's one I've missed. Seems like everything is hunky dory once you login once
[21:05] <ali1234> flocculant: will cooke mentioned wanting to get it fixed for the beta, but i think the problem is a little bit tricky due to it being an interaction with the livecd
[21:06] <ali1234> the good news is we know exactly the cause of the problem now. someone just needs to figure out the right way to fix it
[21:39] <bluesabre> ali1234: that's in relation to the bluetooth bug?
[21:40] <ali1234> yeah. bluetooth timeout is a red herring
[21:40] <ali1234> it was fontconfig all alon
[21:40] <bluesabre> wowza
[21:40] <TJ-> hehehe, ali1234 did some great debugging last night
[21:41] <bluesabre> And it's because we carry the monstrous noto font library?
[21:41] <ali1234> i just posted this complete description of the full problem: https://bugs.freedesktop.org/show_bug.cgi?id=103652#c6
[21:41] <ubottu> Freedesktop bug 103652 in fc-cache "fontconfig requires nano second precison for fontcache" [Normal,New]
[21:41] <ali1234> noto makes it slow but the rebuild should not run at all
[21:43] <ali1234> and it seems like fontconfig taking ~1 minute to rebuild the cache delays all other processes from starting, so they all time out too. bluez, and ubuntu has a similar problem with g-s-d
[21:45] <bluesabre> ali1234: excellent work figuring that all out
[21:46] <bluesabre> and TJ- too, I saw you also poking at it
[21:46] <ali1234> a lot of people have been poking this one because it's so widespread
[21:46] <ali1234> someone noticed it happening in debian in november :)
[21:47] <bluesabre> wowza
[21:48] <TJ-> could we use a sneaky fix? just 'touch ...' the files to stop fc rebuilding the cache?
[21:48] <ali1234> no, touch doesn't work because the timestamps are embedded in the cache files
[21:48] <ali1234> and also they have to exactly match, not just be newer
[21:48] <ali1234> we could do what ubuntu does and force a font cache rebuild inside the installed system during install
[21:49] <ali1234> then only the live desktop will be delayed
[21:49] <ali1234> first login on install will not
[21:50] <TJ-> read the file's internal timestamp and apply it ?
[21:50] <ali1234> teres a bunch of different ways to work around it. i'm sure laney will come up with something good :)
[21:54] <TJ-> the timestamp is at offset 0x30 in the cache file, it's a standard time_t
[21:55] <ali1234> yes
[21:56] <ali1234> but then there is a second timestamp at 0x38 which is nanosecond accurate. that is the problem
[21:56] <ali1234> squashfs doesn't support that, so that part of the timestamp gets zeroed
[21:56] <ali1234> (the second field is just the nanoseconds)
[21:57] <ali1234> squashfs wipes it from the actual files, but because it is embedded in the cache file, it survives there
[21:59] <TJ-> ali1234: right, I get that, but for the installed system a .postinst script could use touch to set the correct timestamp on each cache file by reading the internal timestamp
[21:59] <ali1234> could do, or it could just rerun fc-cache which is simpler :)
[22:00] <TJ-> Because the .postinst already has a stanza to trigger an fc-cache if needed
[22:00] <ali1234> that will happen anyway as soon as you update any font package
[22:00] <TJ-> but that is what we're trying to prevent isn't it? the fc-cache run
[22:00] <ali1234> well, not ubuntu
[22:01] <ali1234> they want to fix it for the live cd, because they already rebuild the cache during install, so they are not affected on first login
[22:01] <ali1234> but if it's fixed in the live cd, that will fix it for us as well
[22:01] <TJ-> the /postinst fc-cache run is ignored because the cache files are copied in but then when fontconfig is called on it sees wrong timestamps and calls fc-cache - do I have that correct?
[22:02] <ali1234> yes but you're missing one part - when fontconfig gets called it doesn't have root so it rebuilds it only for one user
[22:02] <TJ-> for squashs I wonder if an overlayfs would help ... not sure we can do metadata only though
[22:02] <ali1234> that's why it happens again on first login
[22:02] <ali1234> during the live desktop it rebuilds it for the live user and that's wiped on reboot
[22:04] <TJ-> yeah, the patch is the way to go I think. I was just thinking for the Thursday deadline for the beta ISOs, depending on what time tomorrow. I read earlier the ISO gen started this morning
=== GridCube_ is now known as GridCube
[22:08] <ali1234> the ultimate fix would be to add nanos support to squashfs but of course that would be a huge change during beta
[22:09] <TJ-> and ultimately not very useful
[22:09] <TJ-> the assumption by fontconfig is the problem
[22:09] <ali1234> the assumption is only that files will never be moved between a filesystem that supports nanos and one that doesn't
[22:09] <ali1234> another possible fix would be to strip the nanos before building the iso cache
[22:10] <ali1234> they are going to disappear anyway, that would just mean doing it slightly earlier
[22:11] <TJ-> yeah, it's a weird one though, worrying about the nanoseconds for a font cache
[22:11] <ali1234> it looks like fontconfig can be built entirely without support for nanos so that would be another option
[22:11] <TJ-> I wish I could build a system entirely without support for bugs :D
[22:25] <TJ-> We could also build the ISO images on ext4 without the extra_isize feature to disable nanosecond  timestamps :)
[22:25] <ali1234> yes
[22:26] <ali1234> that would mean big changes to LP builders though