RoyK | patdk-lap: I know, and it's not easily configurable, and it's not globally configurable | 00:07 |
---|---|---|
RoyK | seems to me either lucid has some stone age version of pastebinit, or it should be fixed up a bit | 00:08 |
patdk-lap | hmm, lucid normally does | 00:09 |
patdk-lap | if work lets up some, I need to submit a crapload of bug reports | 00:09 |
patdk-lap | for things like that | 00:10 |
patdk-lap | mytop needs a small patch to make it mysql 5.0+ compatable | 00:10 |
patdk-lap | and other things like that | 00:10 |
patdk-lap | have them all pushed out to my ppa | 00:10 |
patdk-lap | so all patches are already in place, just need to track it all down and submit them | 00:10 |
RoyK | seems a lot has happened between pastebinit 1.1 and 1.2, and that 1.0 perhaps should have been named 0.1 :P | 00:18 |
=== jtgh is now known as jh8i | ||
=== jason is now known as Guest62563 | ||
arooni-mobile | once i set up a /etc/init.d script; how do i give it "life" i.e.. make it so it gets run on every startUP? | 03:18 |
=== jrwr is now known as jrwr|offline | ||
smw | arooni-mobile, I like to use sys-rc-conf | 03:32 |
=== jason is now known as Guest38588 | ||
=== Guest38588 is now known as jasef | ||
philipballew_ | How hard would it be to set up a vpn server through ssh remotely? | 06:23 |
=== jason is now known as Guest19072 | ||
=== lenios__ is now known as lenios_ | ||
args[0] | I just installed darkstat on my ubuntu VPS, this is my first time using it.. read on some forums that this daemon saves charts and data on web pages and can be accessed by localhost:666 , how can i access it if im not on the system and using it as VPS? myIP:666 doesn't work. thanks | 07:28 |
greppy | args[0]: you could setup an ssh port forward to see it. | 07:43 |
=== jason is now known as Guest74183 | ||
=== smb` is now known as smb | ||
kklimonda | philipballew: hmm, it should be doable but rather risky | 08:18 |
=== mendel__ is now known as mendel_ | ||
sattu94 | Hi, I am trying to install Ubuntu 10.04 LTS Server to on a virtual RAID1 drive created using two 1TB HDDs. during the Installation when it tries to detect disks, it says that it has found some RAID devices, and asks if I want to activate them. After selecting 'Yes'. It Scans all the devices but only shows the 'sdc', which is the pendrive through which the installation is taking place. I dont see an option to partition the virtual RAID1 drive. | 09:05 |
sattu94 | It is an Intel Rack Server, SR26UR | 09:06 |
sattu94 | It is an Intel Rack Server, SR2600UR | 09:06 |
_ruben | what is virtual raid1 .. you mean fakeraid or so? .. if so, i suggest disabling the raid features in the bios and use plain software raid | 09:19 |
sattu94 | _ruben: yea, i guess i'll do that. | 09:22 |
_ruben | sattu94: in most cases software raid will perform way better than fakeraid, and is better to manage as well | 09:23 |
sattu94 | _ruben: Thank You. | 09:25 |
alex88 | hi guys | 10:48 |
alex88 | i've a nfs where are stored some images to be processed, i want to have multiple machine that process these images, which can be scaled to increase the processing power, how should i implement the queue? i was thinking to use a mysql table with polling and use locks to prevent multiple istances work on same image | 10:48 |
Jeeves_ | Is it just me, or does upstart render /etc/init.d/networking useless | 12:02 |
Jeeves_ | A restart of the service does not deconfigure any interfaces | 12:02 |
Jeeves_ | It just runs ifup -a | 12:02 |
Jeeves_ | But no ifdown anywhere | 12:02 |
Jeeves_ | bug #890189 | 12:10 |
uvirtbot | Launchpad bug 890189 in upstart "`restart networking` does not deconfigure an interface" [Undecided,New] https://launchpad.net/bugs/890189 | 12:10 |
=== Guest74183 is now known as jasef | ||
eagles0513875_ | hey guys i am having some issues trying to compile some source code for libreoffice | 12:46 |
eagles0513875_ | i get an error saying a cups dev file isn't installed when it is actually installed. | 12:46 |
eagles0513875_ | does anyone have any ideas what the issues might be | 12:47 |
nuscly | eagles0513875_: a version mismatch between cups | 12:51 |
eagles0513875_ | ok | 12:53 |
eagles0513875_ | nuscly: i don't think so as it seems others i have asked don't seem to have this issue | 12:54 |
nuscly | eagles0513875_: perhaps it's a dependancies of cups | 12:55 |
=== smb` is now known as smb | ||
eagles0513875_ | nuscly: the missing dev file that the source code is telling me I'm missing is installed | 12:56 |
nuscly | eagles0513875_: find the code that detect the version to understand the problem | 12:58 |
eagles0513875_ | there isn't much code in this bash script :( to tell me what the issue is | 13:01 |
hallyn | SpamapS, what is the deal with this annoying /lib/init/failsafe.conf? | 13:10 |
Daviey | morning hallyn o/ | 13:10 |
hallyn | Daviey, hey | 13:10 |
hallyn | oh, i see what's going on. i have a dummy br0 being defined, and failsafe.conf gets uppity about it not being on the net | 13:12 |
TREllis | Daviey: o/ | 13:27 |
Daviey | hey TREllis | 13:27 |
Daviey | TREllis: silly question, but is ifupdown-extra required? | 13:30 |
TREllis | Daviey: er, required? don't think so. if so, epic fail as I don't have that installed | 13:30 |
TREllis | ifenslave was the only thing I grabbed | 13:31 |
Daviey | TREllis: I have NFI, sorry. | 13:31 |
Daviey | TREllis: The system boots, but you have no networking right? | 13:31 |
Daviey | TREllis: if so, can you restart networking using serial console, and see if it works correctly? | 13:31 |
Daviey | (and provide dmesg output?) | 13:32 |
TREllis | Daviey: https://help.ubuntu.com/community/UbuntuBonding has no mention of ifupdown-extras, so don't think it's required | 13:32 |
TREllis | Daviey: restarting has no affect, but starting the bonding manually works | 13:32 |
TREllis | Daviey: I have hundreds of these " bonding: bond0: Warning: Found an uninitialized port" as mentioned in public bug lp#889423 | 13:33 |
TREllis | Daviey: but let me grab a proper dmesg | 13:33 |
Daviey | TREllis: seen, bug 482419 ? I wonder if that regression has bene re-introduced | 13:35 |
uvirtbot | Launchpad bug 482419 in ifenslave-2.6 "802.3ad interface bonding fails if started too early" [Medium,Fix released] https://launchpad.net/bugs/482419 | 13:35 |
Daviey | TREllis: *certain* that restarting networking doesn't fix it? | 13:35 |
TREllis | Daviey: yeah saw that, fix released tho? | 13:36 |
Daviey | TREllis: yeah, wondered iof it had been re-introduced | 13:37 |
TREllis | Daviey: ah ok | 13:37 |
TREllis | Daviey: let me try some tests | 13:37 |
Daviey | TREllis: did you see comment #69 btw? | 13:39 |
Daviey | TREllis: yes, so the patch from that Fix Released has been dropped | 13:41 |
Daviey | However, it does have a "early_setup_master" addition, which i guess should have mitigated it. | 13:42 |
Daviey | TREllis: It would be interesting to see if moving setup_master, directly under add_master solves the issue. | 13:43 |
TREllis | Daviey: confused now | 13:46 |
TREllis | Daviey: early_setup_master is a interfaces file option? | 13:46 |
Daviey | TREllis: no, sorry.. it's in a pre-up script. | 13:47 |
Daviey | TREllis: hold fire, let me create a patch | 13:47 |
TREllis | k | 13:47 |
Daviey | TREllis: $ wget http://pb.daviey.com/tt12/ -O - | sudo patch /etc/network/if-pre-up.d/ifenslave | 13:52 |
_ruben | i ran into that very bug last week .. the only "reliable" workaround was doing it all "by hand", as in: up modprobe bonding... up ip link.. up ifenslave.. etc | 13:54 |
Daviey | _ruben: on oneiric? | 13:54 |
_ruben | yes | 13:54 |
zul | good morning | 13:56 |
_ruben | Daviey: the patch you mentioned worked properly like once out of 10 reboots | 13:56 |
_ruben | Daviey: what could've been playing a part in it was the fact that the 3 interfaces participating in the bond didn't have any link (staging setup) | 13:57 |
Daviey | Well it seems to be known that it might not work properly with upstart. :/ | 14:00 |
Daviey | _ruben: TREllis is currently trying the Debian experimental current version on Oneiric | 14:00 |
Daviey | (and i just prepaired that patch against Oneiric archive, doh.. hope it still applied.) | 14:01 |
uvirtbot | New bug: #890219 in apache2 (main) "suexec-custom is not working correctly: only reading the www-data user config file." [Undecided,New] https://launchpad.net/bugs/890219 | 14:01 |
uvirtbot | New bug: #890230 in samba (main) "package winbind (not installed) failed to install/upgrade: il sottoprocesso vecchio script di pre-removal ha restituito lo stato di errore 128" [Undecided,New] https://launchpad.net/bugs/890230 | 14:01 |
TREllis | Daviey: I've got side tracked a bit, should be able to test it a little later :) | 14:03 |
Daviey | TREllis: slacker. :) | 14:03 |
TREllis | Daviey: lol | 14:03 |
Daviey | TREllis: I bet you are busy watching The Return of Frank James. | 14:04 |
zul | oh crap....i have a dead fish in the office | 14:05 |
jasef | Omnomnom | 14:06 |
jasef | Wait... is it a pet fish? | 14:06 |
jasef | Cause if so, not omnomnom | 14:06 |
* koolhead17 is awake finally | 14:13 | |
jamespage | morning all | 14:18 |
irvie | have to move my backup server to another room so i want to do any updates at this time so i only have to reboot it once | 14:20 |
irvie | already did an apt-get update and upgrade | 14:20 |
irvie | anything else i should check? i believe it's 10.04 server | 14:20 |
koolhead17 | irvie, sounds good. cat /etc/lsb-release will tell you about ubuntu server version you are uing :) | 14:30 |
irvie | yep, 10.04.3 LTS | 14:36 |
irvie | so now i can just shutdown and power it back upa nd everything shoudl be good | 14:36 |
irvie | :D | 14:36 |
irvie | hopefully :p | 14:37 |
irvie | 10:04:09 up 245 days, 1:29, 2 users, load average: 0.00, 0.00, 0.00 | 14:37 |
irvie | :( | 14:37 |
irvie | how can i see the link speed on my NIC? | 14:38 |
irvie | lolz RX bytes:9414931995665 (9.4 TB) TX bytes:220667948142 (220.6 GB) | 14:38 |
_ruben | irvie: depending on the driver, there might be some hints in the output of dmesg, ethtool should be able to tell it as well | 14:46 |
patdk-wk | irvie, use ethtool or mii-diag | 14:49 |
patdk-wk | hmm, mii-diag is reporting wrong results for me, but ethtool is correct | 14:51 |
uvirtbot | New bug: #453747 in samba "Wrong folder size on folder in smb shares" [Low,Triaged] https://launchpad.net/bugs/453747 | 15:05 |
SpamapS | hallyn: /lib/init/failsafe.conf? Is that new in precise? | 15:32 |
SpamapS | hallyn: or you mean /etc/init/failsafe.conf ? | 15:33 |
hallyn | SpamapS: it's on oneiric | 15:33 |
hallyn | it was making my netbook wait 2 minutes at boot, just bc i had 'auto br0' in /etc/network/interfaces but no connection at boot | 15:33 |
hallyn | well, it's not that simple, of course. it's just the failsafe, so somethign else is waiting. runlevel 2 is waitin gon a net connection? | 15:34 |
hallyn | SpamapS: you wrote it, at any rate :) | 15:34 |
SpamapS | hallyn: yes | 15:36 |
SpamapS | /etc/init/failsafe.conf then, yes I wrote it and smoser and I put together the changes to /etc/network/if-up.d/upstart to make static-network-up work, which is what is waited on in rc-sysinit now | 15:37 |
SpamapS | hallyn: if you had read the release notes, you would have known that all 'auto' interfaces will be waited on :) | 15:37 |
irvie | koolhead17, migration successful :] | 15:37 |
hallyn | SpamapS: that's assenine | 15:37 |
hallyn | SpamapS: there are auto devices that are not meant to be 'up' | 15:38 |
hallyn | or put another way, | 15:38 |
SpamapS | thats what network manager is for | 15:38 |
hallyn | you say 'up', but br0 was up - but plymouth couldn't ping | 15:38 |
hallyn | network manager is not usable if you dont' use gnome | 15:38 |
hallyn | and some peopel don't | 15:38 |
SpamapS | To quote the occupy folks.. you are, the 1% ;) | 15:38 |
hallyn | SpamapS: i know it's solving a problem, and don't have a better solution, we don't need to discuss it right now :) | 15:39 |
SpamapS | I have a solution for you 1%'ers for 12.04 | 15:39 |
hallyn | SpamapS: but i'd like to talk about it sometime | 15:39 |
hallyn | how will it solve it? | 15:39 |
SpamapS | We will create another group, auto-nowait | 15:39 |
hallyn | i'm happy with that solution | 15:39 |
hallyn | ok. i'll probably reinstall the netbookt from scratch (half lucid, half 12.04) so that's fine with me | 15:40 |
kirkland | morning folks | 15:40 |
hallyn | morning kirkland | 15:40 |
SpamapS | You can work around it right now by just changing /etc/rc-sysinit to not wait for static-network-up | 15:40 |
hallyn | nice cloudy morning for ya? | 15:40 |
SpamapS | err | 15:41 |
SpamapS | /etc/init/rc-sysinit.conf | 15:41 |
koolhead17 | irvie, :D | 15:41 |
koolhead17 | lynxman, hellos | 15:41 |
hallyn | SpamapS: for now i worked around it by getting rid of the auto br0 :) that was itself just a test af ew months ago anyway | 15:41 |
kirkland | <Kiall> 18:00:27> ubuntu seems to be pushing byobu over tmux .. <----- byobu is sort of a compliment, or enhancement layer on either screen or tmux | 15:41 |
=== matrix30001 is now known as Matrix | ||
=== Matrix is now known as Matrix3000 | ||
kirkland | Kiall> 18:00:58> Kinda getting used to byobu .. Its enabled by default on all the EC2/UEC/Cloud images now... | 15:41 |
kirkland | Kiall: unfortunately, that's being removed in an SRU very soon | 15:42 |
SpamapS | hallyn: AHA! | 15:42 |
hallyn | kirkland: paul t. is giving me hope of an improved dvtm so i don't need byobu-tmux for the inside-a-screen splitting :) | 15:42 |
hallyn | (see planet.u.c. from friday i think) | 15:43 |
SpamapS | oi.. my sup index just went over 100,000 messages | 15:43 |
* SpamapS should delete more | 15:43 | |
kirkland | hallyn: heh, byobu/tmux couldn't hold your attention, huh? | 15:43 |
hallyn | SpamapS: 0 inbox :) | 15:43 |
SpamapS | I have 0 inbox all the time! | 15:44 |
SpamapS | since in sup, you just "archive" what you don't want to deal with now. :) | 15:44 |
hallyn | kirkland: it's great for somet hings, but i prefer dvtm's tiling behavior | 15:44 |
kirkland | hallyn: cool, i'm installing dvtm now | 15:44 |
SpamapS | but right now... having shunned email for 3.5 days.. I'm looking at 2000 inbox | 15:44 |
hallyn | but it doesn't do backscrolling so i dont' use it all that much | 15:44 |
kirkland | hallyn: does dvtm replace gnome-terminal, or byobu/tmux? | 15:45 |
hallyn | kirkland: uh what? it doesn't replace anything, it enhacnes :) | 15:45 |
hallyn | kirkland: i use dvtm inside screen inside gnone-terminal or xterm or st | 15:45 |
hallyn | (except i don't) | 15:46 |
kirkland | hallyn: neat, i'll have to play with it | 15:46 |
hallyn | kirkland: you may hate it. it depends on what you're used to i think. But it's ideal behavior for me. | 15:46 |
zul | Daviey: ping | 15:47 |
hallyn | kirkland: hae you ever used dwm? | 15:47 |
kirkland | hallyn: nope | 15:47 |
hallyn | kirkland: cool then i'll be especially interested in what yout hink of dvtm :) | 15:48 |
Daviey | zul: | 16:04 |
zul | Daviey: so python-passlib made it into the archive i think we should revist doing an SRU for keystone with the port change | 16:04 |
Daviey | zul: \o/ | 16:05 |
zul | Daviey: so i take it you agree? :) | 16:06 |
Daviey | zul: well if keystone doesn't currently work, changing the default port will not break systems | 16:06 |
zul | Daviey: yeah that will make it easier | 16:06 |
=== mendel__ is now known as mendel_ | ||
zapotah | a good network monitoring tool that shows realtime bandwidth stats per application | 16:29 |
zapotah | does the server distro ship with one or which one do you recommend | 16:30 |
RoyK | zapotah: dunno about any tool for that, but I guess you could configure munin to do it with some iptables tweaks | 16:34 |
RoyK | zapotah: that'll be for incoming connections, though, dunno for outgoing, but I guess it should be possible | 16:35 |
zapotah | hmm | 16:41 |
zapotah | hoping for an easy solution | 16:41 |
zapotah | like top for cpu and mem usage | 16:41 |
zapotah | etc | 16:41 |
RoyK | zapotah: network is a bit more tricky | 16:41 |
RoyK | zapotah: try asking on #munin - someone has probably done that already | 16:44 |
just-a-visitor | zapotah: There is iftop, but it is per-connection not per process. | 16:45 |
RoyK | or iptraf | 16:46 |
just-a-visitor | Yes, that is what I was looking for... but still they are based on packets. | 16:47 |
RoyK | just-a-visitor: you can do stuff like iptables [...] --uid-owner to add counters per uid or gid, but I don't think there's an easy way to monitor network usage per process | 16:50 |
RoyK | zapotah: nethogs, perhaps | 16:52 |
just-a-visitor | RoyK: Cool, I did not know about it. Thanks! | 16:59 |
zapotah | nethogs shows tcp only apparently | 17:02 |
zapotah | id need to monitor mainly udp traffic | 17:02 |
RoyK | just-a-visitor: you might want to try to build 0.8.0 from source - http://sourceforge.net/projects/nethogs/files/ | 17:08 |
RoyK | zapotah: sorry, that was for you ^^ | 17:08 |
just-a-visitor | RoyK: Btw, I am also looking at it right now. :) | 17:09 |
hallyn | stgraber: gah, something went wonky with the last lxc push, bc my branch stacked on the precise udd branch now won't fetch | 17:14 |
zapotah | ill have to look at this later... so tired after 48hrs and no sleep x.x | 17:25 |
zapotah | not making any progress | 17:25 |
Daviey | SpamapS: Your cobbler branch, are you uploading that to precise soonly? | 17:33 |
Daviey | rbasak has based a branch on yours, that would be good to co-upload. | 17:33 |
SpamapS | my branch? | 17:49 |
SpamapS | Daviey: refresh my memory, its been eons :p | 17:50 |
Daviey | https://code.launchpad.net/~clint-fewbar/ubuntu/oneiric/cobbler/misc-fixes/+merge/77771 | 17:51 |
Daviey | SpamapS: ^^ | 17:51 |
Daviey | SpamapS: Do you want to rebase your branch to current precise, and review + sponsor rbasak's changes at the same time? | 18:03 |
Daviey | it seems to be an entire security upload, so i guess we should see about prepairing a -security upload aswell. | 18:04 |
Daviey | (for oneiric) | 18:05 |
Daviey | zul: Are you planning a cobbler new upstream version upload soonly? | 18:08 |
zul | Daviey: yeah | 18:09 |
zul | 2.2.2 is suppose to be out soon | 18:09 |
Daviey | zul: When are you planning a snapshot? | 18:09 |
zul | my arm can be twisted for tomorrow | 18:09 |
SpamapS | Daviey: indeed.. ugh.. ok, just now finishing with the monday morning flood of email.. will look at this next. | 18:09 |
SpamapS | rbasak: where are your proposed cobbler changes? | 18:09 |
Daviey | zul: well can you talk to SpamapS and rbasak, about if they should base their branch on current or tommorrow's | 18:10 |
rbasak | SpamapS: https://code.launchpad.net/~racb/ubuntu/oneiric/cobbler/858878_858883/+merge/81996 | 18:10 |
SpamapS | Awesome sauce would be if zul just pulled in my changes and rbasak's changes ;) | 18:10 |
zul | yes that would be awesome :) | 18:11 |
zul | SpamapS: where are your changes | 18:11 |
Daviey | zul: see rbasak's branch, it includes SpamapS.. (but needs rebasing.) | 18:11 |
zul | SpamapS: nm found it | 18:11 |
zul | SpamapS: we should probably do an SRU as well | 18:12 |
Daviey | zul: rbasak & SpamapS's changes are all -security. | 18:12 |
SpamapS | argh.. 2 hours of inbox clearing has produced 96 more threads to deal with | 18:12 |
zul | Daviey: ah good...the csrf stuff should already made it in | 18:12 |
Daviey | SpamapS: You need something like, http://pastebin.com/ALiL1ksn on your Maildir. | 18:13 |
Daviey | I find it makes my mail processing much faster | 18:13 |
patdk-wk | people still use maildir? | 18:13 |
Daviey | patdk-wk: what do you use? | 18:14 |
patdk-wk | mdbox | 18:14 |
zul | Daviey: i have a secretary for processing email ;) | 18:14 |
lynxman | zul: what's her name? | 18:14 |
patdk-wk | it's just hell to backup all the inodes maildir uses :( | 18:14 |
adam_g | Daviey: lol | 18:14 |
zul | lynxman: big bertha | 18:15 |
Daviey | patdk-wk: wait, you really think mdbox is more mainstream than *either* maildir or mbox? | 18:15 |
patdk-wk | mainstream? no :) | 18:15 |
patdk-wk | but aren't you cutting edge? :) | 18:16 |
Daviey | patdk-wk: no. :) | 18:16 |
rbasak | "Note that with dbox the Index files actually contain significant data which is held nowhere else." | 18:17 |
Daviey | patdk-wk: well i have 15857 inodes left.. so that'll last until the end of the week. | 18:17 |
rbasak | So why are they called Index files then? | 18:17 |
patdk-wk | rbasak, cause they are kept in the path the index files where kept in before? | 18:18 |
Daviey | mail is one of the last tennents on one of my hardy xen servers that i really CBA to touch. | 18:19 |
patdk-wk | I redo mine often, the last hardy xen guest I have to move though is mail also, exchange 2007 :( | 18:20 |
Daviey | hah | 18:20 |
Daviey | <-- courier | 18:21 |
patdk-wk | I oviously use dovecot for most things :) | 18:21 |
patdk-wk | hmm, 400megs of indexs, and 3gigs of email (compressed) | 18:22 |
patdk-wk | the indexs would only be about 100megs if it wasn't for the search databases | 18:22 |
Daviey | patdk-wk: What do you use for searching mail? | 18:22 |
patdk-wk | imap | 18:22 |
patdk-wk | or you could use doveadm | 18:22 |
Igoru | i'm trying to compile a PHP extension, but it suddenly dies when the compiling process gets to line "config.status: executing libtool commands". any idea about how to track this problem? :( | 18:34 |
* mdeslaur is looking for someone to steal his puppet merge | 18:36 | |
mdeslaur | oh, wait, it's a sync...forget it | 18:37 |
potatoe | Is there a way to flag certain processes to be higher priority or lower priority when the system is swapping ? ( ie, process mysqld should never be swapped, process joe-bin should be swapped first when there is not enough memory ) | 18:44 |
kyconquers | I'm trying to stress test a few different configurations of mail servers, does anyone have any recommendations for good applications or libraries to use? | 18:46 |
Igoru | i'm trying to compile a PHP extension, but it suddenly dies when the compiling process gets to line "config.status: executing libtool commands". any idea about how to track this problem? :( | 18:59 |
Daviey | adam_g: Do you want to discuss cobbler-enroll? | 19:30 |
* zul perks his ears up | 19:31 | |
adam_g | Daviey: sure | 19:33 |
Daviey | smoser: here? | 19:33 |
smoser | here | 19:34 |
Daviey | adam_g, smoser, zul, roaksoax: Right.. | 19:35 |
Daviey | Fat image vs (ab)using d-i | 19:35 |
Daviey | lets get this cracked out. | 19:35 |
zul | im all ear | 19:35 |
zul | ears even | 19:35 |
kyconquers | Can anyone recommend a library or application for stress testing an email server? | 19:36 |
Daviey | adam_g, smoser, zul, roaksoax http://pad.ubuntu.com/OrchestraDiscoveryBloatedVsDI | 19:36 |
Daviey | Just so the topic is clear, this will mean moving away from the DI cobbler-enlist we currently have, to something probably python based in a fat image | 19:37 |
Daviey | who is mynameisjonas? | 19:38 |
Daviey | Using my own etherpad server allows me to ban people who annoy me :) | 19:40 |
hallyn | elitist | 19:42 |
smoser | why would this thing be python? | 19:43 |
adam_g | if all we're talking about is system hardware discovery and reporting (to cobbler?), we could accomplish that by 1, extending cobbler to store that data. 2, writing some shell to gather the info. 3, extending cobbler-enlist to post it | 19:45 |
adam_g | honestly, the facts that ship with facter related to this don't do anything that couldn't be rewritten in shell | 19:45 |
Daviey | zul: ? | 19:47 |
zul | doesnt matter to me really python or something else | 19:48 |
Daviey | zul: can you comment on, " - (We will need to do that anyway) " | 19:48 |
zul | Daviey: ah i mean we are going to have to add the security bits anyways | 19:48 |
Daviey | zul: on the pad please :) | 19:50 |
Resistance | Daviey: i'm curious why you posted the etherpad link if you want only specific people to read it :P | 19:51 |
Resistance | ;P | 19:53 |
smoser | DATA LOSS! | 19:54 |
Daviey | Resistance: that is not the case, but i want those that are inputting data to identify themselves. | 19:54 |
smoser | awesome! | 19:54 |
Resistance | i see. | 19:54 |
Daviey | smoser: blame jamespage | 19:54 |
zul | but jamespage is awesome | 19:55 |
smoser | good thing htat iddn't happen at UDS. | 19:55 |
jamespage | DATA LOSS == 'User Error' | 19:55 |
smoser | if this was written in go, it would have rocked. | 19:55 |
Daviey | it's a feature, called garbage collection, right jamespage ? | 19:55 |
zul | smoser: hehe | 19:55 |
jamespage | 'User Error' == 'Smoser Error' | 19:55 |
Resistance | Daviey: so i suppose that random users who want to lurk the data are kicked? ;P | 19:56 |
jamespage | Daviey: well I guess most things smoser writes are garbage so you might be right :-) | 19:56 |
smoser | this is all quite true | 19:56 |
Resistance | lool | 19:56 |
Daviey | Resistance: no.. not at all | 19:56 |
Resistance | Daviey: so if I were interested in lurking the data, i wouldnt be kicked when i attempted to lurk? ;P | 19:57 |
smoser | jamespage, enable chat on etherhpad on ubuntu.com | 19:57 |
jamespage | lol | 19:57 |
adam_g | is the plan to extend cobbler to store the hw data we gather at first boot? | 19:57 |
zul | adam_g: yes | 19:57 |
adam_g | can we add the pad what we plan on gathering and storing? | 19:58 |
Daviey | Resistance: no. | 19:58 |
adam_g | Oops! A server error occured. It's been logged. | 19:59 |
smoser | ok | 19:59 |
adam_g | jamespage: ^ ?! | 19:59 |
zul | adam_g: i dont see why not | 19:59 |
smoser | right | 19:59 |
smoser | at least the error has been loged | 19:59 |
smoser | logged | 19:59 |
smoser | we may have lost all your data | 19:59 |
smoser | but we logged an error | 19:59 |
adam_g | smoser: you want to email support@etherpad.com or shall i ? | 19:59 |
Daviey | Resistance: although, i'm always apprehensive of those that hide their id. | 20:00 |
smoser | do you think they can restore the data ? | 20:00 |
jamespage | I think that would be your best course of action | 20:00 |
Resistance | Daviey: true. Granted, my etherpad ID is still in there from UDS... shows up as Resistance (irc) or EvilResistance (irc) | 20:00 |
zul | adam_g: macaddr, cpu, cpu_core, arch, nics, mem, etc, etc | 20:00 |
Resistance | because i remoted in for UDS :P | 20:00 |
Daviey | Resistance: were you at UDS? | 20:01 |
Daviey | ahh | 20:01 |
Resistance | Daviey: no i remoted in | 20:01 |
adam_g | zul: etc etc is what im interested in getting down. :) to see if theres antyhing we can't get from /proc and /sys | 20:01 |
Daviey | Resistance: is this something which interests you? | 20:01 |
* Resistance wishes he was at UDS though | 20:01 | |
Resistance | Daviey: no, but i was just curious what you'd do ;P | 20:01 |
Resistance | and i agree with you, people hiding their IDs are evil | 20:02 |
Resistance | :Lp | 20:02 |
smoser | wwDWd | 20:02 |
Daviey | adam_g: I think everything can be grokked from /proc, /sys or parsing dmesg | 20:02 |
zul | adam_g: right....problem is we have to to take arm into account as well | 20:02 |
Daviey | zul: arm exposes all 3 of those data entry points :) | 20:02 |
smoser | Daviey, of course, its all easy. | 20:02 |
zul | Daviey: not if we want to use dmi info ;) | 20:03 |
smoser | i was actually just thinking that using /bin/sh seems like overkill to me for reading through /proc /sys and such. | 20:03 |
adam_g | Daviey: zul if thats what we need to do, we can depend on some shell scripts to aggregate all of that information somewhere, and cobbler-enlist to post it back to orchestra | 20:03 |
smoser | i was thinking i'd re-write a library of C functions like strlen and strdup and such | 20:03 |
smoser | and then use that | 20:03 |
zul | adam_g: yeah that sounds simple enought | 20:03 |
zul | smoser: bleah | 20:03 |
smoser | it can't be that hard to re-invent things, can it? | 20:04 |
smoser | </sarcasm> | 20:04 |
Daviey | smoser: we don't have that many options, really. | 20:06 |
smoser | so, reading data about the system from /proc /sys, "should" be easy | 20:07 |
adam_g | how about: shell to gather system information, dump it in a file or directory, preseeded cobbler-enlist runs with an option to read its arguments from that file, posts back the information thats there | 20:07 |
smoser | the fact is that things arent ever easy | 20:07 |
Daviey | smoser: I'm not saying easy, but for the limited data we require.. not overly hard to do in shell. | 20:08 |
smoser | you will find that what you get is not consistent or complete across systems by different manufacturors | 20:08 |
smoser | and then you'll start to account for those things | 20:08 |
smoser | and then you'll realize that facter (or other tools) already did those things | 20:08 |
smoser | and thats why they exist | 20:08 |
Daviey | smoser: okay, i think we need to review the "other tools" | 20:09 |
smoser | and then you'll decide that it wa | 20:09 |
Daviey | facter really can't be an option, due to being ruby - which we have in neither d-i env or too much to put in a fat image | 20:09 |
smoser | i'm largely just guessing. it may be that the kernel magically makes everything easy | 20:10 |
smoser | but i really doubt it. | 20:10 |
Daviey | smoser: well parsing text files is almost as bad as screenscraping, i see that | 20:10 |
adam_g | quickly peaking at facter.. its designed to be portable among different OS's. but for its linux purposes, its just parsing the standard places (/proc/cpuinfo, /proc/meminfo, etc) | 20:10 |
Daviey | it's not like i can say cpu = kernel.give-me-metric("cpu"), and get reliable output. | 20:11 |
smoser | well, you can parse all that garbage on the server side if you want | 20:11 |
smoser | which makes it easier | 20:11 |
Daviey | There is a python fork of facter, but that was less than clean | 20:11 |
Daviey | (we don't have python in d-i env either) | 20:11 |
zul | hdt-project.org but uses dmi info | 20:11 |
Daviey | smoser: post a blob back, and parse it in python via cobbler? | 20:12 |
Daviey | zul: we ruled out hdt for being the worst of both worlds, no? | 20:12 |
zul | Daviey: yeah but do we really need dmi info | 20:12 |
Daviey | zul: don't think so | 20:13 |
smoser | that is what i was suggesting, yes. | 20:13 |
zul | then i think it should be back on the table | 20:13 |
smoser | essentially: tar -C /sys cvzf - . | post-to-cobbler | 20:13 |
Daviey | smoser: well, we are modifying the api regardless.. so either way works for me | 20:13 |
Daviey | it does seem somehow cleaner to post params, rather than a blob.. but shrug | 20:14 |
Daviey | zul: but what advantage? | 20:14 |
zul | Daviey: that it already parses the information already? | 20:14 |
Daviey | zul: it gives us the worst of both worlds. | 20:15 |
Daviey | It's C, so speed of development is slower than Python. | 20:15 |
Daviey | It requires writes for tftp, so insecure. | 20:15 |
Daviey | we could extend the fake cobbler tftpd service for this. | 20:15 |
Daviey | but it seems we have neither the free enviroment of d-i, or the fat image benefits. | 20:16 |
adam_g | looking at facter some more, theres no reason why its linux-specific (or ubuntu-specific) functionality couldnt be easily reproduced easily in shell. im interested in lookin at other solutions as well, but i suspect its all the same | 20:16 |
zul | Daviey: gotcha | 20:17 |
Daviey | adam_g: the python fork just parsed those files, looked hacky | 20:17 |
Daviey | smoser: why do you feel posting back a blob is better than individual calls? | 20:18 |
Daviey | and parsing in the client? | 20:18 |
adam_g | Daviey: im not sure what other / better ways there are | 20:18 |
smoser | well if the client is d-i, then the parsing that stuff is just going to be more painful than it would be on the other side. | 20:20 |
smoser | if collecting the data is simply just grabbing some directories, then just grab those, and parse on the server where you have some sane programming language. | 20:20 |
smoser | you may even be able to trick facter into thinking that it is looking at that data | 20:21 |
smoser | or somehow otherwise hijack stuff. | 20:21 |
Daviey | adam_g: I think it is the best way, but my memory of the python fork was done hacky. | 20:22 |
adam_g | http://paste.ubuntu.com/738553/ | 20:22 |
smoser | well, clearly, re-implementing it in sh is not going to be hacky! | 20:22 |
smoser | :) | 20:22 |
Daviey | smoser: so, parsing in python will be safer than parsing in busybox shell, is your thought? | 20:22 |
smoser | s/safer/easier/ | 20:23 |
smoser | faster | 20:23 |
adam_g | i agree with smoser that parsing macaddr's out of 'ip addr list' is easier in python (or other) | 20:23 |
smoser | i've parsed macaddrs out of ip addr before | 20:23 |
Daviey | adam_g: the reason i wrote the mac address stuff in C was to avoid doing it in sh. | 20:24 |
Daviey | That is done.. so not a concern.. the other parts have similar concerns? | 20:24 |
adam_g | Daviey: if we standardize on something (shell, python on the server-side, whatever) i'd like to use that for macaddrs (instead of ioctl) as well as everything | 20:25 |
Daviey | I agree with that. | 20:25 |
Daviey | I am leaning towards it being easier to grok this data in shell, than add a parser component to cobbler. | 20:26 |
zul | oh hell yes | 20:26 |
zul | im afraid of adding bloat to cobbler as well | 20:27 |
Daviey | smoser: are you still leaning towards fat image vs d-i ab(using)? | 20:28 |
smoser | how would it be easier to grock this data in shell than add a parser compenent to cobbler? | 20:29 |
smoser | you're posting this somewhere | 20:30 |
smoser | the thing that takes the post can store the whole blob and then parse it in python | 20:30 |
smoser | the d-i abusing is really going to basically depend upon everything you want being available via /proc or /sys. anything more complex than that is going to get difficult. | 20:31 |
smoser | ie, like getting dmi, or some other bits. | 20:31 |
adam_g | which is why we should create a definitive list of everything we need to gather, to see what is easily available and what is not | 20:33 |
smoser | +1 | 20:33 |
zul | +1 | 20:34 |
adam_g | zul: are you planning on upstreaming cobblers hw inventory stuff? | 20:35 |
Daviey | smoser: grocking in shell is a standalone script, integrating this in cobbler will require more thought to make sure the workflow is followed | 20:35 |
Daviey | zul has shown that adding simple single fields is pretty straightforward. | 20:35 |
smoser | if you can add a simple single field | 20:36 |
smoser | then you add one that is "sysinfo-dump" | 20:37 |
smoser | you store in that a hex dump of what you got | 20:37 |
smoser | then you additionally populate whatever other fields you were going to add anyway | 20:37 |
zul | its easy for a single field probably a bit more work to store the sysinfo-dump | 20:37 |
smoser | why? | 20:37 |
smoser | its 1 field | 20:37 |
smoser | is there something particularly difficult about the string s-y-s-i-n-f-o ? | 20:37 |
zul | parsing the info and storing them on how cobbler stores them | 20:38 |
smoser | cobbler has some entry point to which "field" is posted. you just handle that field (sysinfo-dump) by populating it and others. | 20:38 |
smoser | but maybe i'm missing something. | 20:38 |
zul | smoser: cobbler only has the ability to store a single field afaik | 20:39 |
smoser | what? | 20:39 |
smoser | <Daviey> zul has shown that adding simple single fields is pretty straightforward | 20:39 |
zul | smoser: i dont think it has the ability to store large chunks of info | 20:39 |
smoser | i'm only proposing adding a single field. | 20:40 |
smoser | i dont really care. | 20:40 |
zul | anyways | 20:40 |
* adam_g lunch | 20:43 | |
Daviey | hmm | 20:45 |
Daviey | i walk away for 2 mins, and it falls apart :) | 20:45 |
Daviey | smoser: are you suggesting that the sysinfo is stored as a blob long term, and cobbler internals parse it on demand.. or parse it when first posted back? | 20:46 |
smoser | i was suggesting you store it long term just because there is no reason to throw it away | 20:46 |
smoser | but it would clearly not make sense to parse it on demand if it doesnt change | 20:46 |
Daviey | smoser: so when object foo is called, if it = None, it parses it and inserts it? | 20:48 |
Daviey | smoser: i missed your reply stating if you were still in the fat image, or d-i abusing camp.. did you comment? | 20:48 |
smoser | i was just suggesting the time when it is posted back, you go thorugh and update all the dpenedent fields and store it. | 20:48 |
smoser | i think that you're going to end up re-examining "fat image" either sooner or later. | 20:49 |
smoser | but i dont know which. | 20:49 |
smoser | i think the first thin gyou should do is decide what you wan tto collect, as adam_g suggested. | 20:49 |
smoser | if you can get all that from /proc and /sys then it makes sense for the moment to go with that. | 20:50 |
Daviey | So... Disk quantity and size, number of cpu cores, memory, arch .. i'm not sure there is anything else we /need/ is there? | 20:55 |
smoser | i dont knwo. people have mentioned dmi info. | 20:56 |
smoser | what is the stated goal of this exercise ? | 20:56 |
smoser | to be able to categorize a machine into some bucket similar to 'm1.small' 'm1.large' and the like ? | 20:57 |
Daviey | smoser: So the data can be manipulated to make decisions how how to install the boxes. | 20:57 |
Daviey | smoser: yeah, basically. | 20:57 |
Daviey | but i don't think it can be abstracted so closely to ec2 style strings. | 20:57 |
smoser | well, fwiw, amount of disk is almost certainly insufficient for actually classifying stuff. | 20:57 |
smoser | i'd suspect that you care more about the speed or reliability of disk than the size | 20:58 |
smoser | or at least sometimes you do | 20:58 |
Daviey | smoser: it's really for determining if the machine has lots of storage or not. | 20:58 |
Daviey | 10G vs 10TB. :) | 20:58 |
smoser | but thats almost certainly not enough informatoin | 20:58 |
smoser | isn't it? | 20:58 |
Daviey | smoser: I think it's /enough/ for 12.04.. agree? | 20:58 |
smoser | well, it depends. | 20:59 |
smoser | maybe it is. | 20:59 |
smoser | but if my goal is to let juju take control and dynamically decide whihc is a node and which is a api server or volume server, it sprobably not enough info | 20:59 |
smoser | right? | 20:59 |
smoser | wouldn't you need to know much more about what its connections are? | 21:00 |
smoser | i guess that'd be hard to get anyway | 21:00 |
Daviey | smoser: I think that is a >12.04 target really. | 21:01 |
zul | it all depends on what info you want and how you store it | 21:01 |
Daviey | well yes. | 21:01 |
Daviey | roaksoax: Currently we have an admin user where we give out the creds freely. What do you think about adding a user flag, which makes it so the user can only add/edit the same mac machine? | 21:05 |
Daviey | perhaps the password for the user would be the mac address? | 21:05 |
roaksoax | Daviey: From my point of view cobbler's user password should be set on install | 21:34 |
roaksoax | Daviey: maybe orchestra can then handle the creation of another user | 21:34 |
roaksoax | Daviey: cause, the password for cobbler user is encrypted so it is not publicly available to anyone | 21:34 |
roaksoax | is it? | 21:34 |
Daviey | roaksoax: yes, but for a remote enlistment service we need to give it away like free beer. | 21:44 |
Daviey | which, as you can understand, is less than cool | 21:44 |
roaksoax | Daviey: right, well cobbler has a feature on which you can add owners to certain stuff, i.e. add an owner for a system | 21:46 |
roaksoax | but that system I believe has to be added first | 21:46 |
roaksoax | Daviey: now, when registering remotely, we need to provide admin/user password in order to, obviously, add a new system | 21:47 |
Daviey | roaksoax: yes, but i was wondering about extending to have a user setting that only allows it to add/edit it's own mac addresses | 21:48 |
Daviey | not entirely secure, but /better/ than what we have atm | 21:48 |
Daviey | Unless you have a better plan? | 21:48 |
roaksoax | Daviey: afaik, you need the administrator user/pass (cobbler) to add systems, but you can have owners of the system that can only edit values within systems for example | 21:48 |
roaksoax | Daviey: from cobbler wiki: "If you want to control which users/groups can create objects, this will probably require modifying the python authz_ownership implementation slightly -- see the "Customization" section for more details. I am open to proposals on what this may require, though in general, it's important to remember the purpose of the ownership module is to help the users perform the tasks they need to do -- if they are being annoying an | 21:49 |
roaksoax | Daviey: https://fedorahosted.org/cobbler/wiki/AuthorizationWithOwnership | 21:49 |
roaksoax | Daviey: so what you are looking for is just an authentication module that allows adding systems only? | 21:51 |
Daviey | roaksoax: are you following what i am saying? | 21:52 |
Daviey | really yes, a module that only allows adding/removing of their own system | 21:53 |
Daviey | Having 100000's of users isn't a good idea | 21:53 |
smoser | Daviey, so above, i think that we should go forward with d-i scraping of /proc and /sys | 21:54 |
smoser | * get a list of all the data we want to have | 21:54 |
roaksoax | Daviey: right, but if you add a user that can add/remove their own sytem, is having 1 user per syustem, which isn't good idea as you say | 21:54 |
smoser | * start some little script to collect it | 21:54 |
smoser | my leaning towards collecting a ton of info and saving it off to the server was because we're almost certain to throw away useful information when we grab it in /proc or /sys in shell | 21:55 |
smoser | but if we throw it all to the server it is at least there later for subsequent re-examination and improvement. | 21:55 |
Daviey | smoser: it's probably harder for us to SRU the client component. | 21:56 |
=== skrewler_ is now known as skrewler | ||
Daviey | [Dmeaning overposting is /better/ | 21:56 |
roaksoax | Daviey: i personally don't see the point of adding a user that can add/remove its own system cause that would mean 1 user per system | 21:57 |
Daviey | roaksoax: I was wondering about a virtual user, where the password is the mac address or something? | 22:04 |
Daviey | thoughts? | 22:04 |
roaksoax | Daviey: but then it is a passwordless user then... cause... if it is gonna use its mac address (which cobbler does not know about), then it is the same as not authenticating at all | 22:11 |
lynxman | Daviey: roaksoax: smoser: hey guys, question for you, I have a broadcom USB 2.0 controller on a server and Oneiric doesn't detect any disks I connect to it (used to work in CentOS 5), doing lsusb shows the controller root hub being present, thoughts? | 22:17 |
roaksoax | lynxman: maybe there's no drivers for it | 22:18 |
roaksoax | broadcoms were always a PITA | 22:18 |
lynxman | roaksoax: yeah it's confusing because it shows up both in lspci and lsusb | 22:19 |
roaksoax | lynxman: yeah that's it most likely... i have always find problems with broadcoms | 22:19 |
lynxman | roaksoax: darn :/ | 22:20 |
roaksoax | Daviey: what probably makes more sense is to have a user that can only add/remove system but can't access anything else | 22:22 |
roaksoax | Daviey: i.e. orchestra user | 22:22 |
adam_g | roaksoax: currently, cobbler-enlist makes creates a new system and then modifies it (to set its mac, name, profile, etc) not sure what that means for access control | 22:22 |
roaksoax | adam_g: yeah basically cobbler has 2 modules, authentication, and authorization | 22:24 |
roaksoax | adam_g: by default we use authentication is based on users on a config file | 22:26 |
roaksoax | and authorization is all users have access to everything | 22:26 |
Daviey | where did lynxman go?! | 22:27 |
roaksoax | adam_g: there's another module on which allows the definition of owners/groups bu that's only to edit things | 22:28 |
roaksoax | such as systems | 22:28 |
lynxman | Daviey: here o/ | 22:28 |
roaksoax | adam_g: now, if we wanted to create a user that can *only* add systems, then, we would need to write a new authorization module | 22:28 |
lynxman | Daviey: still trying to solve this usb thing :) | 22:28 |
Daviey | lynxman: dmesg | pastebinit , pls :) | 22:28 |
lynxman | Daviey: yessir | 22:28 |
lynxman | Daviey: http://paste.ubuntu.com/738683/ | 22:30 |
lynxman | Daviey: http://paste.ubuntu.com/738684/ | 22:30 |
Daviey | lynxman: hmm | 22:33 |
Daviey | nothing interesting | 22:33 |
lynxman | Daviey: yeah, no clues at all except the disks not being detected | 22:34 |
lynxman | Daviey: which is annoying, used to work in centos5, and those are my backups | 22:34 |
Daviey | :( | 22:34 |
lynxman | indeed | 22:35 |
Daviey | lynxman: odd that ssh and apport exit non zero | 22:36 |
lynxman | Daviey: it was first boot, didn't reboot the machine again since its syncing glusterfs | 22:37 |
Daviey | ahh | 22:37 |
lynxman | Daviey: I should before going prod tomorrow... | 22:38 |
zul | Daviey: nak on the macaddr passwd | 23:05 |
uvirtbot | New bug: #890465 in puppet (main) "init script should cleanup environment" [Undecided,New] https://launchpad.net/bugs/890465 | 23:10 |
Daviey | zul: negativity isn't that helpful, counter a suggestion with a better one :) | 23:15 |
zul | Daviey: if i only had one :) | 23:15 |
roaksoax | Daviey: 1 user that can only add systems | 23:16 |
Daviey | roaksoax: just add, or edit aswell? | 23:17 |
roaksoax | Daviey: add/edit/remove | 23:17 |
Daviey | roaksoax: is that secure? | 23:17 |
roaksoax | Daviey: well we would have to write our own authorization module... | 23:18 |
roaksoax | Daviey: secure as in the user won't have access to anything else within cobbler but it is the same approach as using the cobbler user | 23:18 |
Daviey | roaksoax: I'd like that to be a plan B... If we can come up with a better (secure) solution, i'd be overjoyed. | 23:24 |
roaksoax | Daviey: ok | 23:27 |
Daviey | roaksoax: If we think what UEC did.. The central server published it's ssh public key (discovered via avahi)), which the node added as an authorized_key, allowing the server to ssh TO the client to $do-stuff. | 23:37 |
Daviey | This wasn't ideal, but more secure than what we currently have. | 23:37 |
kirkland | jcastro: ping | 23:41 |
kirkland | jcastro: i'm releasing a byobu with a feature you requested :-) I thought you might like to test it out | 23:42 |
roaksoax | Daviey: right, in this case we have a slightly different escenario as we are doing things over the API and doing it autmoatically, or manually, they both require user/password authentication, which is, in turn same level of security on both cases (auto registration/manual registration) | 23:50 |
Daviey | roaksoax: aye, but in the old scenario - a node couldn't fiddle with other nodes central registration. | 23:51 |
roaksoax | Daviey: right, but we can make this special user to only *add* systems and not allow it to edit/remove | 23:52 |
Daviey | roaksoax: but as adam_g said, we add a base profile, then edit it. | 23:53 |
roaksoax | Daviey: right, but isn't it better to add the profile with all the required information? | 23:54 |
roaksoax | Daviey: cause, how will this work. Are we having a bloated image PXE booted? | 23:54 |
roaksoax | Daviey: cuase, from my point of view, the "registration" process should already provide all the details we want to gather | 23:56 |
roaksoax | and should do in one step | 23:56 |
roaksoax | we can still have 1 user per system with password its mac address and as user its hostname | 23:57 |
roaksoax | but even so, the admin will hve to add the system first, and then assign the ownership | 23:57 |
Daviey | roaksoax: I might be mistaken, but i believe it has to be a multi-stage API process. | 23:58 |
roaksoax | Daviey: well I guess that will depend on how we are registering the system | 23:59 |
roaksoax | in the first place | 23:59 |
roaksoax | caus eif we use a bloated image then we can just acess the API once | 23:59 |
roaksoax | and that;s it | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!