/srv/irclogs.ubuntu.com/2011/04/01/#ubuntu-kernel.txt

=== _LibertyZero is now known as LibertyZero
cdbs3.12-1ubuntu5 of module-init-tools has produced a regression as it passess 11n_disable=1 to iwl3945 which the iwl3945 driver isn't able to recognize, as it doesn't support it at all07:51
=== smb` is now known as smb
fairuzHi morning08:10
fairuz__raw_readl is used with pa or va?08:10
cdbsAnyone around who can sponsor my debdiff?08:20
smbcdbs, I don't think here. We rather work git oriented for the kernel. I'd probably try #ubuntu-devel08:23
cdbssmb: :o08:23
smbcdbs, Oh I see, module init tools08:25
cdbssmb: bug #74705908:26
smbcdbs, Sorry missed that in the scrollback08:26
cdbssmb: sorry its bug #74702508:26
ubot2Launchpad bug 747025 in module-init-tools "Modprobe passes 11n_disable=1 option to iwl3945 which doesn't support the option" [High,Triaged] https://launchpad.net/bugs/74702508:26
RAOFHm.  There appears to be an entirely blank Yes/No dialog box in the kernel's apport hook…08:27
* RAOF guesses ‘yes’08:27
smbRAOF, maybe08:30
RAOFEat hot aufs kernel warning!08:41
smbcdbs, The patch looks ok to me, but I do not seem to have the necessary rights to upload. But I will talk to the right person later today.08:49
qwebirc44173hello08:50
qwebirc44173need help to shutdown  the computer08:50
qwebirc44173cpu and chassis fans keep spinning08:51
qwebirc44173my motherboard is amd pc chips m810lmr08:51
qwebirc44173im uusing ubuntu server 32bit08:51
qwebirc44173i tried with pci=noacpi option un grub08:51
qwebirc44173tried to disable apm/acpi in bios08:51
qwebirc44173any tips?08:52
cdbssmb: okay, thanks a lot!08:59
qwebirc44173can anyone help me with my issue above09:03
smbqwebirc44173, I sort of feel inclined to say pull the power cable... There is a lot of 32bit ubuntu server versions around too (hardy, lucid, ...). But you may try to look at /sys/power/disk. It should be either set to platform or shutdown and you may try the other one09:03
qwebirc44173smb: happened the same xubuntu 10.0409:04
qwebirc44173i could not shutdown completly09:04
qwebirc44173smb: what do you mean with pull the power cable09:05
smbqwebirc44173, That will surely stop the fans. Sorry I should have used sarcasm tags.09:06
qwebirc44173:P09:06
qwebirc44173i though it was just an expression09:06
qwebirc44173thought09:07
sorenqwebirc44173: Like I already explained... Disabling ACPI is only going to make absolutely sure the machine won09:45
soren't shut down.09:45
qwebirc44173soren: ei09:56
qwebirc44173output of /sys/power/disk is test testproc [shutdown] reboot09:58
qwebirc44173Why is shutdown between []09:59
smbthat is active09:59
qwebirc44173hm?09:59
smbIs that still a boot with disabled acpi?10:00
qwebirc44173smb: what is active?10:00
qwebirc44173yes10:00
qwebirc44173i think10:00
qwebirc44173let me reboot10:00
qwebirc44173smb: it's using pci=noacpi10:00
smbThat would explain why there is now platform. I mean the option within [] is selected10:01
smbI mean there is no platform10:01
qwebirc44173smb: no platform to shutdown?10:13
smbqwebirc44173, Platform would be the default on most machines, but that uses acpi methods, so if acpi is disabled I won't expect platform. Well actually without acpi and probably no apm there is hardly any way to shutdown automatically as soren told you10:15
qwebirc44173smb: yes but i tried with those enabled10:16
qwebirc44173before10:16
qwebirc44173i dont know what else i can do10:16
qwebirc44173im gonna try another distro10:16
smbqwebirc44173, well I suspect rather another bios or a newer release/kernel... 10:22
sorenNo matter what you do, if you keep disabling acpi, there's no way it's going to shut down.10:24
sorenIf you can't turn on the lights in your hours, and you think it's due to the light switch, removing the light switch and shouting at the hole in the wall where it used to be isn't going to turn on your lights.10:26
sorens/hours/house/10:26
soren..and moving into a different house, and removing all the light switches there will yield the same results.10:27
smbsoren, Still when people want to move I won't stand in front of the truck. :)10:28
_rubenunless it uses apm (which is not tied into acpi, is it?)10:28
smb_ruben, right, that would be independent. But it could be disabled in the bios or not implemented10:28
soren_ruben: apm is a different system, yes. With its own set of problems. Like, say, not supporting SMP.10:29
sorenI seriously doubt is very well tested these days.10:29
qwebirc44173smb: now I tried with everything enabled10:35
qwebirc44173it still shows [shutdown]10:36
qwebirc44173and wont turn off fans10:36
smbIT sounds a bit like something is wrong with the acpi bios interface, but that is not something that can easily be solved on irc. You should file a bug report about it, and possibly see whether you can try newer kernels from the mainline builds.10:40
qwebirc44173im gonna try acpi=force10:40
qwebirc44173the bios supports APM and ACPI10:40
smbyou would see what is used in dmesg10:41
qwebirc44173k10:42
qwebirc44173smb: it's using IRQ10:47
smb??? most computers do... Without context this makes not much sense. 10:51
qwebirc44173Allocate IRQ for PCI VGA yes10:51
smbReally, it would be better to open a bug report with gathered info. Depending on release that is done with "ubuntu-bug -p linux" or without the need for -p.10:53
qwebirc44173This is the exact problem that I have http://ubuntuforums.org/showthread.php?s=c9437dac0fddfb56461dc26eadd4fb11&t=169287711:03
qwebirc44173smb11:05
=== yofel_ is now known as yofel
qwebirc44173apm -s No APM support in kernel11:50
qwebirc44173:O11:50
=== doko_ is now known as doko
qwebirc44173bah12:36
qwebirc44173How do I enable APM support?12:44
qwebirc44173need to know12:44
fairuzqwebirc44173: you have CONFIG_APM=y in your kernel config?12:49
fairuzor M12:49
qwebirc44173it's M12:49
qwebirc44173:\12:50
fairuzah it's a module12:50
qwebirc44173how to change to y12:50
fairuzqwebirc44173: just load it then? using modprobe?12:50
qwebirc44173oh12:50
qwebirc44173:)12:50
qwebirc44173like sudo modprobe apm?12:50
qwebirc44173fairuz: im not linux geek :)12:50
qwebirc44173ill try12:51
fairuzqwebirc44173: i'm not too. just a user12:51
qwebirc44173fairuz: "Yes the kernel is not processing the APM command (advanced power management). Try this command as the root user: /sbin/modprobe apm."12:51
qwebirc44173brb12:51
qwebirc44173error12:53
fairuzi don't know either, just giving ideas :D12:54
fairuztake a look at this, maybe can give you some ideas12:54
fairuzhttp://www.linuxquestions.org/questions/debian-26/no-apm-support-in-kernel-266977/12:54
qwebirc44173FATAL: Module amp not found.12:54
qwebirc44173through ssh12:54
qwebirc44173from local machine is different but gives error12:54
fairuzbut do you know where they store this module?12:55
qwebirc44173no idea12:55
fairuz:D afaik, you have to know the path to use modprobe12:55
qwebirc44173lolz12:56
fairuzthe path of the module of course12:56
qwebirc44173Sometimes a Linux kernel is not configured properly for shutting down a computer properly. You can recompile the kernel to fix this problem.12:56
qwebirc44173fail12:56
fairuzsearch the path for the apm module12:57
fairuzand load it from there12:57
qwebirc44173modprobe apm FATAL: Error inserting apm (/lib/modules/2.6.35-28-generic-pae/kernel/arch/x86/kernel/apm.ko): No such device12:59
qwebirc44173fail12:59
qwebirc44173yea ikonia :) 12:59
qwebirc44173fairuz need to know path :x12:59
smbmodprobe does not need a path, insmod would. But the reason for the module failing to load is only visible in dmesg. And even when loaded, at least that is how it used to be, apm will only be used when acpi is not active at all. And again using apm will take away other features only acpi has.13:13
qwebirc44173smb: i dont care about acpi13:14
qwebirc44173:p13:14
qwebirc44173smb: ERROR: Module acpi does not exist in /proc/modules13:15
qwebirc44173it's disabled right13:15
smbNo its build-in13:16
qwebirc44173in motherboard?13:17
qwebirc44173hmhmh13:17
qwebirc44173like ac97 codec and stuff?13:18
smbNo into the kernel. But sort of into you motherboard too because its part of the bios13:18
fairuzqwebirc44173: tried to modify grub and enable apm + disable apci from there?13:18
qwebirc44173yes13:18
qwebirc44173fairuz: acpi=off apm=on apm=power_off"13:18
qwebirc44173and /etc/modules13:18
qwebirc44173apm power_off=113:19
fairuzdamn i forgot, what is the function to convert hexa in string (e.g. "0x68") to int? :D13:28
_ruben"brain"13:29
smbecho "ibase=16; 68"|bc ? :)13:29
fairuz:D13:30
qwebirc44173smb fairuz apm: BIOS not found.13:42
qwebirc44173fail13:42
smbtgardner, Did you do uploads to module-init-tools in the past? At least the changelog seems to say so...13:46
tgardnersmb, yes13:46
smbThere was a proposal fix here https://bugs.launchpad.net/ubuntu/+source/module-init-tools/+bug/74702513:47
ubot2Launchpad bug 747025 in module-init-tools "Modprobe passes 11n_disable=1 option to iwl3945 which doesn't support the option" [High,Triaged]13:47
=== diwic is now known as diwic_afk
smbIt looked ok to me, but I believe I got no rights for that package13:47
qwebirc44173smb: so linux cant work with my bios?13:47
tgardnersmb, for Natty? I just uploaded something a couple of days ago13:47
smbtgardner, For natty, yes. It seemed newer that current bzr and apt-get source at least13:48
smbqwebirc44173, It sounds at least like your bios does not provide the interfaces to enable apm at least.13:48
qwebirc44173smb: there is an option to enbale13:49
qwebirc44173enable13:49
qwebirc44173:x13:49
qwebirc44173it shows APM and ACPI13:49
smbqwebirc44173, Just to make sure, have you checked with the mainboard vendor whether ther is any newer bios?13:49
qwebirc44173smb: there is but im afraid to flash it13:50
qwebirc44173:(13:50
* smb bites into his desk...13:50
qwebirc44173smb: http://www.pcchipsusa.com/PCCWebSite/Downloads/ProductsDetail_Download.aspx?detailid=175&DetailName=New&DetailDesc=M810  (V7.1)&CategoryID=1&MenuID=8213:50
qwebirc44173broken link13:50
qwebirc44173not13:51
qwebirc44173check this link pls http://www.pcchipsusa.com/PCCWebSite/Downloads/ProductsDetail_Download.aspx?detailid=175&DetailName=New&DetailDesc=M810  (V7.1)&CategoryID=1&MenuID=8213:51
smbSo the "newer" bios still is 2003... Quite old... Maybe you want to check out the firmware test suite to see how badly your current state is or is not...13:54
smbhttps://launchpad.net/~firmware-testing-team13:54
smbtgardner, So I would have a source deb, though I got no doubt that you probably have that done already by now13:56
tgardnersmb, pretty close to done.13:57
smbtgardner, Yeah too slow is me13:57
Shred00how come linux-source-2.6.32_2.6.32-02063212_all.deb has nothing in it except an empty ./usr/src/linux-source-2.6.32/ and a few ./usr/share/doc/ files?14:08
qwebirc44173smb: sorry but how can I use firmware test14:10
smbqwebirc44173, If you follow that link, there are PPAs for various releases. If you add the one for yours then you can apt-get install fwts and use the command of the same name14:11
qwebirc44173smb: im going to use Firmware Test Suite (Maverick Stable)14:13
Shred00where can i get the patches that were applied to the mainline 2.6.32.12 kernel?14:17
smbShred00, Hm, the separate patches started only later and I am pretty sure the kernel-source package was broken often. Maybe it works taking a 2.6.32.12 kernel and try to get the patches from http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.32.20-lucid/ applied...14:22
Shred00wouldn't it be closer to use 2.6.32.11-lucid's patches?14:23
Shred00it's a real shame that i can't reproduce the mainline build exactly since it's that build that i need to start bisecting patches out of to find the breakage14:23
smbYeah. Funny why those are there14:23
qwebirc44173smb: just sudo poweroff in archlinux14:37
qwebirc44173it worked14:37
smbqwebirc44173, Ok, so its something in the kernel between whatever version archlinux uses and Maverick. And maybe it is fixed in the kernel Natty uses.14:40
qwebirc44173I hope so :)14:40
qwebirc44173smb: maybe it's fixed in Lucid?14:40
qwebirc44173smb: can I boot up using only cli in desktop livecd?14:41
smbqwebirc44173, Not sure it works on the livecd but you may try to add "text" to the boot commandline14:42
qwebirc44173will try14:42
smbcd14:43
=== dannf_ is now known as dannf
tgardnerogasawara, have you tried natty this morning? my hp mini restarts gdm every time I touch the mouse.15:02
ogasawaratgardner: I haven't, /me tries15:02
tgardnercnd, you around? I see an xserver-xorg-input-synaptics upload this AM. I cannot correlate my GDM mouse crash, but it does look coincidental.15:09
cndtgardner, yes15:09
cndthere was an upload last night15:09
cndit has a bug15:09
cnda new upload has been made15:09
cndit just needs to be built in the archive and published15:09
cndyou can downgrade to a previous version for now15:10
tgardnercnd, ah, known issue. good.15:10
cndtgardner, this is a temporary fix too: http://people.canonical.com/~cndougla/utouch/xserver-xorg-input-synaptics_1.3.99+git20110116.0e27ce3a-0ubuntu10~test1_i386.deb15:10
cndif you're running i38615:10
tgardnerogasawara, did you catch that ? ^^15:10
ogasawaratgardner: yep15:10
tgardnercnd, amd6415:10
cndtgardner, ogasawara: https://launchpad.net/ubuntu/+source/xserver-xorg-input-synaptics/1.3.99+git20110116.0e27ce3a-0ubuntu815:11
cndthat's the previous version that works fine15:11
tgardnercnd, ok, that fixes it.15:22
cndgood15:23
tgardnerogasawara, are there any show stoppers on skaet's kernel bug list?15:45
ogasawaratgardner: not that I see15:45
skaettgardner, ogasawara - bug list not updated yet with input from yesterday.15:46
ogasawaraah, so there may be some surprises15:46
skaetwill let you know as I do the processing if anything kernel specific makes the list that wasn't already there.15:46
tgardnerskaet, I'm going on last weeks search. there are still some linux bugs in there.15:46
skaetthere's a draft with today's date there15:47
skaetjust didn't link it to agenda15:47
skaethowever,  not all bugs are scrubbed, and pulled in from yesterday's triaging.15:47
skaethas bugs from wednesday at bottom if you want to scan.15:48
tgardnerskaet, s'alright. there are plenty on the list I have that are worthy of cleanup.15:48
skaet:)15:48
skaetyeah, no shortage of things to work on, I'm sure.15:49
skaetlol15:49
JFonever is :)15:51
ogasawaratgardner: I think the powerpc bug that you already applied the patches for was the only one that really caught my eye15:51
tgardnerogasawara, yeah, that one was a real problem for older ppc platforms.15:52
ogasawaratgardner: is apw back on mon or wed?16:00
tgardnerogasawara, not sure. perhaps pgraner knows.16:02
=== cmagina-afk is now known as cmagina-office
pmatuliscan one do a net install with an LTS backports kernel?16:27
tgardnerpmatulis, only from the DVD bits16:31
pmatulistgardner: the DVD bits?16:31
pmatulistgardner: so no net install?16:32
tgardnerjjohansen, I just assigned you to bug #746751. can you see if you can diagnose it?16:32
ubot2Launchpad bug 746751 in linux "kernel: [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is 30)" [Critical,In progress] https://launchpad.net/bugs/74675116:32
tgardnerpmatulis, frankly, I'm not sure. Can you develop a net install from the DVD ?16:33
tgardnerpmatulis, the CD had no space.16:33
pmatulistgardner: ok.  btw, the DVD does not work in this regard for a server install16:33
tgardnerpmatulis, "does not work" ? what do you mean?16:34
pmatulistgardner: bug #73069016:34
ubot2Launchpad bug 730690 in ubuntu-cdimage "10.04.2 DVD not made to allow server install with 2.6.35 kernel" [High,Fix released] https://launchpad.net/bugs/73069016:34
pmatulisbug #73076416:35
ubot2Launchpad bug 730764 in ubuntu-cdimage "10.04.2 DVD does not install server with 2.6.35 kernel when told to" [High,Triaged] https://launchpad.net/bugs/73076416:35
tgardnerpmatulis, well, shit. that was the whole point of the backport.16:35
pmatulis:(16:35
tgardnerpmatulis, ok, they are on my todo list, though its likely an installer issue.16:37
pmatulistgardner: yeah, colin is looking at that16:38
tgardnerpmatulis, I wonder if we can release a 10.04.2.1 DVD ?16:39
pmatulistgardner: i asked colin the same.  no response yet16:40
tgardnerwell, he _is_ kinda busy with the natty release.16:40
pmatulisyes, absolutely16:40
JFotgardner, did you see that bug I showed jjohansen last night?16:41
* JFo digs for the #16:41
JFobug 74675116:41
ubot2Launchpad bug 746751 in linux "kernel: [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR 38d is 30)" [Critical,In progress] https://launchpad.net/bugs/74675116:41
tgardnerJFo, I just assigned it to him 15 minutes or so ago.16:42
JFoah :)16:42
tgardnerit came up in the release meeting16:42
JFoI see that now :-D16:42
JFook cool16:42
JFoI thought it might16:42
tgardneryeah, it seems like kind of a pisser16:42
JFoyeah16:43
JFoI just let it slip my mind earlier or I'd have asked right off16:43
tgardnerJFo, I keep a dynamic todo list. that way I only have to remember _1_ thing, look at my todo list.16:45
JFoheh16:45
JFoI do too, sadly it was toward the bottom16:45
JFonot of importance, of course, just where I put it16:46
jjohansenJFo, tgardner: ack.  I started poking but didn't get to far yesterday16:55
tgardnerjjohansen, thanks. this likely has overall priority until we make some progress on it. I think the server team is pretty well stuck.16:57
jjohansentgardner: axk16:57
jjohansenack even :)16:57
JFogonna grab some lunch, bbiab17:02
=== herton is now known as herton_lunch
=== herton_lunch is now known as herton
* tgardner is finding 2.6.23.12 quite tedious to apply. so damn many patches.18:21
* tgardner --> lunch18:46
=== skaet is now known as skaet_afk
tgardnerbjf, sconklin: pushed 2.6.35.12 stable update19:33
bjftgardner, whee!19:34
sconklin\o/19:34
tgardnerbjf, checkout the patch list: http://bugs.launchpad.net/bugs/74752019:34
ubot2Launchpad bug 747520 in linux "Maverick update to 2.6.35.12 stable release" [Undecided,Fix committed]19:34
sconklintgardner: jebus!19:35
bjftgardner, the good news is most of those already have some testing in other series19:35
tgardnerbjf, you mean the patches we already had? there were 1219:36
bjftgardner, no, many of those also hit Lucid already and have been tested there19:37
tgardnerbjf, ah, right19:37
bjf.me -> food20:02
* bjf -> food20:02
=== bjf is now known as bjf[afk]
ailoI'm trying to debug the kernel. Never done this before. 20:14
ailoThese are the commands I've been instructed to use:20:15
ailosudo -s20:15
ailocd /sys/kernel/debug/tracing20:15
ailoecho 0 > tracing_enabled20:15
ailoecho wakeup_rt > current_tracer20:15
ailoecho 1 > tracing_enabled 20:15
ailoI'm wondering, how can I catch the output?20:15
ailoIs it logged somewhere?20:15
manjoailo, are you using ftrace ? 20:17
ailomanjo, What's that? I really don't much about this tbh.20:18
manjoailo, I guess you could start here http://lwn.net/Articles/289558/20:19
manjoailo, or Documentation/ftrace.txt in the kernel src 20:19
manjoailo, are you debugging a kernel oops ? 20:20
manjoailo, there is some debugging tricks here: https://wiki.ubuntu.com/Kernel/Debugging20:21
ailomanjo, Thanks. I'm doing realtime testing for audio. Using jackd. I'm supposed to see what causes xruns.20:21
manjoailo, ftrace might be useful20:22
manjoailo, you might want to set CONFIG_DYNAMIC_FTRACE in your kernel 20:24
manjoconfig20:24
ailomanjo, It's set to =y. I'm on 2.6.38-7. Haven'20:28
manjoailo, you could search for Steven Rostedt's presentation on google he migh have a quick how to some place 20:29
manjoailo, he presented it at various conf and also demoed it .. so a video probably exists as well 20:29
vanhoofwasnt there a tpm related s3/s4 issue that was fixed in the last couple of SRU updates?20:30
vanhoofsound familiar to anyone20:30
tgardnervanhoof, maverick?20:31
ailomanjo, Great info. Thanks. I was only supposed to do this one thing right now, but I'll want to learn more anyway, so :P20:33
vanhooftgardner: yeah20:33
tgardnervanhoof: glp v2.6.35..HEAD -- drivers/char/tpm8436f415bc7343a99b284661762456e697dae489 tpm_tis: Use timeouts returned from TPM20:34
tgardner9555b769cd6d973d6e84c805371eaa5d5986a25c TPM: Long default timeout fix20:34
tgardner123c6a0ede60b268266c3cce0c341f6427bc7044 tpm: fix panic caused by "tpm: Autodetect itpm devices"20:34
tgardner7d0809ee4b9eaaf7436e23c992aaebfc2408d782 tpm: Autodetect itpm devices20:34
tgardnervanhoof, those are the stable updates.20:34
=== bjf[afk] is now known as bjf
vanhooftgardner: cool let me check those out20:34
vanhoofthanks tgardner 20:34
tgardnernp20:35
manjotgardner, looks like the patch that was SRUed and applied got dropped ? PM / Hibernate: Make default image size depend on total RAM size20:45
manjotgardner, or is it some other tree for maverick ?20:46
manjotgardner, it was applied around 03/1820:46
manjo03/21 rathr 20:46
tgardnermanjo, checking20:46
tgardnermanjo, 0a2fa938da1cc726477a3a94239132fe0973ae49 in master-next20:47
robbiewtgardner: hey...have you all announced the chosen kernel for 11.10 yet...or is that something normally done at UDS20:47
manjotgardner, ah see it!20:47
tgardnerrobbiew, oh, we won't know for sure by then. it'll be an educated guess at best.20:48
manjotgardner, any idea when it might get to proposed ?20:48
vanhoofmanjo: believe its available in pre-proposed right now20:48
robbiewtgardner: ok, figured it was something like that20:48
manjovanhoof, was looking for a wag on the timeframe it might make proposed 20:49
tgardnerrobbiew, likely 2.6.41, maybe .4220:49
robbiewtgardner: ok...works for me20:49
robbiewthnx20:49
vanhoofmanjo: once this current one gets out of the door, and post natty20:49
vanhoofmanjo: there are some delays to expect for the next sru cycle for maverick20:50
tgardnermanjo, inquire of sconklin or bjf20:50
vanhoofmanjo: not kernel team related20:50
bjfmanjo, we are more or less following: https://wiki.ubuntu.com/NattyReleaseInterlock20:50
manjobjf, so possibly May time frame then 20:51
manjofor it to get to updates 20:52
vanhoofmanjo: so once -28.50 (which is in proposed) ships, it gives a bit more flexibility on the hwe team to look at20:52
bjfmanjo, yes20:53
* tgardner is off to trash a server by installing ESXi21:01
=== tgardner is now known as tgardner-afk
=== skaet_afk is now known as skaet
=== sconklin is now known as sconklin-afk
savvas0hello! I was asked to test kernel mainline builds for Bug 721493 -- is it ok if I download the ubuntu 11.04 beta-1 live cd and test it through the live environment?21:54
ubot2Launchpad bug 721493 in linux "dmidecode shows wrong memory information" [Undecided,Incomplete] https://launchpad.net/bugs/72149321:54
hertonsavvas0: it isn't the same thing, because the live cd doesn't come with mainline kernel, but it helps anyway if you get latest natty livecd as it's a 2.6.35 vs 2.6.38 kernel22:03
hertons/mainline kernel/mainline build/22:04
savvas0thank you :)22:05
savvas0I suppose I could install these 3 little packages22:05
jbarnesyay /sbin/installkernel is fixed!22:52
jbarnesthanks to whomever finally fixed it22:52
=== emma is now known as em

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!