[01:21] <twb> Where's UDS this year?  Some canonical dude just emailed me offering to pay for my travel and board.
[01:21] <twb> Oh, it says in the subject: "Orlando"
[01:21] <Pici> twb: florida again irc.
[01:21] <Pici> er, iirc.
[01:21] <Pici> Yes, there.
[01:21] <twb> No way I'm going near .us
[01:56] <IdleOne> twb: why not? free trip and all...
[01:59] <twb> Well, it boils down to 1) I don't like Ubuntu; 2) I *really* don't like USA; and 3) I don't deal well with meatspace.
[02:04] <jdstrand> twb: you're funny :P
[02:05] <twb> jdstrand: do I need to wave https://secure.wikimedia.org/wikipedia/en/wiki/Unix_hater%27s_handbook under your nose?
[02:06] <jdstrand> twb: apparently so. I've actually not read that
[02:06]  * jdstrand adds to his todo
[02:07] <jdstrand> I do recall coming across it in various places, but never actually read it
[02:07] <IdleOne> twb: honest answer. I can appreciate that.
[02:07] <twb> It says something like "we hate unix not out of ignorance, but out of experience.  Only people who deal with its idiocies every day can truly appreciate how horrible it is"
[02:07] <jdstrand> heheh
[02:08] <jdstrand> I definitely need to read it :)
[02:15] <twb> "see Peter Norvig’s detailed argument that Python is Scheme with funky syntax."
[02:15] <twb> No, esr, it isn't.  Case in point: call/cc and TCO.
[02:15] <twb> (http://esr.ibiblio.org/?p=538, linked from the above wiki page)
[05:16] <sticht> quit
[07:48] <jamespage> morning
[08:08] <kev009> any opinions on zenoss vs opsview for monitoring?
[08:15] <twb> collectd
[08:40] <Daviey> I like twb, he is suitably angry.
[08:40] <twb> Daviey: (on reflection, I suppose kees' security work is mostly good.)
[08:41] <koolhead11> hi all
[08:41] <twb> Most other stuff just annoys me though
[08:41] <twb> Early adoption of slapd cn=config databases, for example
[08:42] <twb> And because I only see LTS releases, usually by the time something affects me, that's already been the way Ubuntu has done it for two years
[08:42] <twb> Anyway, really going home now.
[08:42] <Daviey> twb: This UDS is planning for the next LTS. :)
[08:42] <twb> Daviey: yeah, but e.g. unity is already in
[08:42] <Daviey> twb: Yeah, we did gamble with slapd.. Some like it, others hate it.
[08:43] <Daviey> twb: Yeah, we are having unity on Server this time.
[08:43] <Daviey> it looks pretty good in ascii.
[08:43] <twb> I have end users
[08:43] <twb> They have netbooting diskless workstations.  They are prisoners.  In a prison.
[08:43] <twb> I'm not looking forward to retraining them to switch to unity.
[08:43] <Daviey> heh
[08:44] <Daviey> twb: I'm really not convinced it will be /that/ much pain.
[08:44] <twb> Yeah, that might be a bad example
[08:44] <twb> Or say having to make sure 3D is working because I can't just replace unity with metacity like I could with compiz
[08:44] <Daviey> evolution -> thunderbird probably more painful.
[08:44] <Daviey> But they don't /need/ to switch.
[08:45] <twb> Daviey: btdt
[08:45] <Daviey> twb: Ah, already use tbird?
[08:45] <twb> And we are running ephy because sqlite3 was having problems with either NFS or prisoners just pulling the plug out rather than doing a "clean" shutdown
[08:45] <twb> *sqlite3 under ff
[08:46] <Daviey> twb: Well do sleep on your invite.. you are most welcome.
[08:46] <twb> And then ephy/webkit has a problem with authenticated proxies, where it displays the proxy error and then the real page beneath it...
[08:46] <twb> Not that I blame ubuntu for that
[08:47] <rbasak> doesn't sqlite3 say in large letters not to use it with NFS
[08:47] <twb> rbasak: yep
[08:47]  * rbasak never liked NFS
[08:47] <twb> rbasak: and ff hard-depends on using sqlite3 out the wazoo
[08:47] <twb> the alternative was CIFS and I didn't want to deal with machine accounts
[08:47] <twb> I did try kerberized nfsv4 but I ran out of time and money before it actually worked
[08:48] <rbasak> I would either forward the screen or have a local disk
[08:48] <rbasak> s/disk/ramdisk/ if you like
[08:48] <Daviey> shame
[08:48] <twb> I'd quite like to get kerberized SMB2 running in a homogeneous debian/ubuntu environment, but samba4 isn't ready and it's all too hard
[08:48] <twb> rbasak: forwarding the screens isn't as scalable
[08:49] <Daviey> twb: If/when you do do that, i'd love to read a write up of it. :)
[08:49] <Daviey> twb: Have you tried LTSP?
[08:49] <rbasak> twb: but basic stuff will work without immense pain
[08:49] <twb> Unfortunately LTSP was still at 4.0 when I built the existing infrastructure, and we are still using it out of inertia
[08:50] <twb> I have rolled out a couple of small LTSP5 and debian-live sites
[08:50] <koolhead11> TeTeT: hello there
[08:50] <TeTeT> hi koolhead11
[08:50] <twb> rbasak: oh and I do run ff over the network here, because I won't allow anything that bloated on my own desktop, and that one line is struggle-central, because ff is so bad
[08:51] <rbasak> in general, doing anything unusual results in pain, so unless it's central to what I'm doing, I don't do it
[08:51] <Daviey> twb: You should do this, http://lwn.net/Articles/262570/
[08:51] <twb> I do it deliberately so I can file snarky bug reports
[08:52] <twb> Daviey: I know RMS' browsing habits
[08:52] <twb> My main browse is emacs, tho
[08:52] <Daviey> It is very efficient use of your time :)
[08:52] <rbasak> Daviey: speaking of pain, qemu and fifos!
[08:53] <rbasak> AFAICT, if there is a buffer it's not bigger than 8M
[08:53] <rbasak> And not listening on the fifo will cause qemu to throw stuff away
[08:53] <twb> Isn't buffering the kernel's problem?
[08:53] <twb> like ulimit
[08:53] <rbasak> (either that or it's hanging the system)
[08:54] <rbasak> I think it's the former though
[08:54] <twb> -q     The maximum number of bytes in POSIX message queues ?
[08:54] <twb> I guess that's IPC not fifo
[08:54] <twb> so ignore me
[08:54] <Daviey> twb: other ideas for bug 832507, welcome
[08:54] <Daviey> *sigh*
[08:54] <twb> haha
[08:54] <rbasak> Also it won't start the system unless there is a listener, but that's not too bad
[08:55] <rbasak> I've found the code, it just tries a write(2) and if that fails it throws it away AFAICT
[08:55] <twb> I get that too http://paste.debian.net/128809/
[08:55] <Daviey> bug 832507
[08:55] <twb> rbasak: can't you just use udp ports or something?
[08:56] <twb> Daviey: I call savelog in the init script's start job
[08:56] <twb> At least, in the related lxc case
[08:56] <twb> http://paste.debian.net/128810/
[08:56] <Daviey> twb: The problem is that a guest can fill up the hosts hard disk, with cat /dev/urandom > /dev/ttyS0 , from within the instance.
[08:56] <rbasak> twb: udp ports how?
[08:56] <twb> Daviey: ah, DOS nice
[08:56] <twb> rbasak: I dunno
[08:56] <soren> Sorry, what's the problem now?
[08:56] <soren> I thought we had a way forward?
[08:56] <Daviey> soren: seems not.
[08:57] <soren> How come?
[08:57] <rbasak> well hang on, I didn't say that :)
[08:57] <twb> Obviously the solution is to mount a tmpfs on /var/log/libvirt-bin :P
[08:57] <twb> -osize=8M
[08:57] <rbasak> I was just brushing up on what the actual behaviour wasw
[08:57] <rbasak> I'm not fully clear on soren's plan but now I might be capable of understanding it :)
[08:57] <Daviey> soren: Ah, might have jumped too fast.
[08:58] <twb> Oh, I see, it's the same problem
[08:58] <twb> rbasak: I thought you were using -net blah to set up some kind of handy-dandy fifo-backed thing
[08:58] <soren> I thought we would just tell kvm to output to a fifo and read from that. Leave it to nova to implement a ringbuffer-style behaviour.
[08:58] <rbasak> soren, where and how is the fifo going to be listened to? Only on a get_console_output call, or periodically too, and is the fifo going to get persistently held open or opened and closed?
[08:58] <twb> soren: pump it into logger(1)
[08:59] <soren> If Nova dies and the guest fills up the buffer and kvm can't deal with that in a nice way, screw it. They did this to themselves.
[08:59] <twb> Then it becomes rsyslog's problem
[08:59] <soren> rbasak: Hold it open, read from it periodically *and* on get_console_output.
[09:00] <soren> rbasak: That'll make sure we keep flushing it even if noone calls get_console_output, but someone calling get_console_output will also get completely up-to-date responses.
[09:00] <rbasak> OK and where do we write the data that we read, and how are we managing the space usage of that?
[09:00] <soren> Flush it out the filesystem.
[09:00] <soren> Imposing a limit on its size.
[09:00] <rbasak> to a console.log file that can grow indefinitely? :-P
[09:01] <rbasak> ah ok
[09:01] <rbasak> What do we do if it hits the limit?
[09:01] <twb> All logs can grow indefinitely
[09:01] <twb> That's what logrotate is for
[09:01] <soren> Pretend it's a ringbuffer.
[09:01] <twb> The issue just becomes can you -HUP the writer to make it acquire a new fd
[09:01] <rbasak> So just seek to the beginning again?
[09:01] <rbasak> and are you expecting get_console_output to grok the ringbuffer and start reading at the right point?
[09:02] <soren> I'm expecting everything to do The Right Thing[tm] :)
[09:02] <twb> It's not different from e.g. DOSsing the logfile by making sshd emit a "rejected connection for user root from host 1.2.3.4" messages
[09:03] <soren> twb: We don't want to let these logs grow indefinitely.. and logrotate doesn't do that either. By default, at least. It expires old logs and throws them away.
[09:03] <Daviey> well logrotate cannot act fast enough.
[09:04] <soren> rbasak: It's probably worth investigating how big of a message you can get through the message queue (which is how the console output will make its way from the compute node to the API node).
[09:04] <Daviey> rotatelog can, as it's piped through..
[09:04] <twb> Daviey: why?  How is it different from filling the logs via some other program?
[09:04] <Daviey> but i'd rather rip my head off than use rotatelogs
[09:04] <Daviey> twb: Isn't that the same reason people often put /var/ on a seperate partition?
[09:05] <twb> Indeed
[09:05] <Daviey> The difference here, the server admin currently has no way of controlling the speed of the log writes.
[09:05] <twb> He doesn't have that with e.g. sshd either
[09:05] <twb> Unless there's something in rsyslog I'm unaware of
[09:06] <twb> Hell, apache would probably be an even better exaample, because it logs user-supplied string by default
[09:06] <rbasak> OK so I need to patch nova to: 1) create a fifo and open it persistently before it calls libvirt to start the process (I assume right now it relies on qemu to create console.log; we'd be creating that as a fifo in advance). 2) Periodically suck data out of the fifo and give it to the handler. 3) Also do step 2 before processing get_console_output. 4) Write a handler that will write round robin to a real console.log, which will wrap if seek position
[09:06] <rbasak> goes above a limit, and keep track of where it is in memory (so a ringbuffer implementation managed in memory with a disk data store)
[09:06] <twb> So make requests as fast as you can with a 1024kB-long user-agent
[09:06] <rbasak> 5) get_console_output now needs to read through the ringbuffer implementation
[09:06] <twb> What I'm geetting at here is: you're not special, everybody has this problem
[09:07] <twb> Rolling an app-specific fix is silly
[09:07] <rbasak> twb: it'd break the API
[09:07] <rbasak> twb: get_console_log really needs to output the last x bytes written to the console
[09:07] <twb> rbasak: throw away the API then ;-)
[09:07] <rbasak> twb: what happens if the call comes just after the log was rotated?
[09:07] <soren> twb: We're perfectly happy to discard old data.
[09:07] <twb> Oh, hang on, I think I misunderstood
[09:08] <Daviey> We only /need/ 16K IMO.
[09:08] <twb> You're saying that ttyS0 in the guest is directly connected to a file on the server?
[09:08] <rbasak> twb: currently, yes
[09:08] <twb> And you can't connect it to a pty instead?
[09:08] <soren> Daviey: Whuh? That won't even fit all of the bootup-messages.
[09:08] <rbasak> twb: it can be done but painfully
[09:08] <soren> twb: How would that help?
[09:08] <twb> I guess becaus then you don't get any history until you connected
[09:09] <twb> s/until/before/
[09:09] <twb> THat's what I'm doing for my libvirt KVM  VMs, so I don't have to use VNC
[09:09] <soren> This console output stuff is read-only.
[09:09] <twb> OK, let me get even weirder
[09:09] <soren> Using a pty sounds like adding another layer of problems.
[09:09] <twb> Turn that off, and instead use netconsole.ko in the guest
[09:10] <soren> Can't.
[09:10] <soren> It's not ours to control.
[09:10] <twb> So the VM could be windows or something?
[09:10] <soren> This is the cloud. WE provide infrastrucute, random nutjobs run their guests.
[09:10] <twb> OK
[09:11] <soren> Right now, the contract is: Shove the stuff into the virtual serial port, and it'll magically turn up in GetConsoleOuput's response.
[09:11] <Daviey> soren: sorry, 64K
[09:11] <soren> ..and we can't really change that contract.
[09:11] <soren> Well, we can, but we need much better arguments than have been made here.
[09:11] <soren> Since that's what all the existing EC2 images do and losing compatibility with them over something like this sounds like a really bad idea.
[09:12] <rbasak> soren: what do you think of my plan (which is just my understanding of your plan with the details filled out)? Will it work? Would you accept it?
[09:12] <Daviey> soren: Hah, smoser has been pushing for ages that openstack should just provide the hardware.. not this poking, reaching in lark that has been punted.
[09:12] <soren> rbasak: It sounds good, yes.
[09:13] <soren> Daviey: And I completely agree with that.
[09:13] <rbasak> the only catch I can see is that the ringbuffer management is in memory so if nova dies then the stored console log is no longer readable (it'll wrap at an unknown location)
[09:13] <soren> Yeah, just put it in the file.
[09:13] <soren> Like you suggested a couple of days ago.
[09:13] <rbasak> and of course if nova does then we'll lose the fifo fd and that'll mess things up
[09:14] <rbasak> Oh, OK :)
[09:14] <rbasak> So that'll make the ringbuffer robust
[09:14] <rbasak> but what about re-opening the fifo if nova is restarted?
[09:14] <soren> Just make the first line something like: <max_length> <current tail position>
[09:15] <soren> rbasak: What about it?
[09:15] <Daviey> soren: Do you have time to discuss this, shouldn't you be arguing about what code hosting to use? :)
[09:15] <rbasak> soren: I presume I'll need to implement that, but I don't know where or how
[09:15] <twb> OK, REALLY leaving now
[09:15] <soren> Daviey: Everyone else is asleep. I can do whatever I want. When they turn up, I'll get back to bitching about tooling.
[09:15] <Daviey> twb: have fun o/
[09:15] <rbasak> soren: the first line would have to be a fixed length
[09:15] <Daviey> soren: cool :)
[09:16] <soren> rbasak: Just open it again when nova starts. It has code already to pick up where it left off.
[09:16] <soren> rbasak: That would certainly make it easier.
[09:16] <rbasak> OK so that's a step 6 in my plan
[09:17] <soren> rbasak: "<0-padded 64 bit hex max_length> <0-padded 64 bit hex current_tail>\n"
[09:17] <rbasak> I suppose I could pad <current tail position> with zeroes to the same length as max_length
[09:17] <rbasak> why hex?
[09:18] <soren> Because 64 bit is a predictable lenght in hex.
[09:18] <soren> Not so much in decimal.
[09:18] <rbasak> oh, one other thing about the disk ringbuffer format
[09:18] <soren> You can go with octal if you want.
[09:18] <soren> Or binary. Just not decimal :)
[09:18] <rbasak> If I use ascii then it'll corrupt if killed
[09:18] <soren> Why?
[09:19] <rbasak> I can't update it atomically
[09:19] <soren> What *can* you update atomically?
[09:19] <rbasak> If I use a binary int then it will
[09:19] <soren> eh?
[09:20] <rbasak> Maybe it's overkill
[09:20] <soren> Why is it less atomic if you encode it in ASCII?
[09:20] <Daviey> can be improved over time, having /something/ would be nice for first fix :)
[09:20] <rbasak> ascii will cover many more bytes
[09:21] <soren> Still *well* within a filesystem block.
[09:21] <rbasak> too many layers before it gets to a filesystem block. is everything aligned before then?
[09:21] <rbasak> anyway, as Daviey says
[09:21] <rbasak> I'll find or implement a ringbuffer class
[09:22] <rbasak> We could fix that later :)
[09:22] <soren> You're arguing that writing 8 bytes rather than 16 bytes makes it more atomic?
[09:22] <lifeless> yay memory tears.
[09:22] <lifeless> :P
[09:22] <rbasak> I'm arguing that atomic_t is atomic and nothing else is
[09:22] <rbasak> (guaranteed)
[09:23] <soren> Not sure what relevance atomic_t's have here?
[09:23] <soren> What am I missing?
[09:23] <soren> atomic_t sounds like wicked name for a rapper, though.
[09:33] <rbasak> https://bugs.launchpad.net/nova/+bug/832507/comments/4
[09:33] <rbasak> Is this OK with everyone?
[09:34] <soren> Yes.
[09:34] <Daviey> looks good
[09:34] <rbasak> cool
[09:35] <rbasak> Implementation details that are a bit unclear to me: doing periodic jobs, and the restart logic
[09:35] <rbasak> I might need some pointers on the way if that's OK soren?
[09:35] <koolhead11> does it makes sense to do system lockdown with gconf anymore?
[09:35] <soren> Sure.
[09:35] <Daviey> rbasak: restart logic might be for free.
[09:36] <Daviey> koolhead11: gconf? wat?
[09:36] <rbasak> Daviey: I think I see what you mean
[09:36]  * rbasak isn't familiar with the Nova source yet
[09:36] <Daviey> rbasak: You ca use my bzr branch for inspiration where the entry point is to mk the fifo
[09:37] <rbasak> I didn't test what happens if the fifo reader blocks so the fifo buffer fills (but still has it open). qemu will either block or it won't. I don't think it matters as we'll suck at a reasonable periodic rate and if the guest wants to dump tons to the console it's reasonable for it to be slowed down
[09:37] <rbasak> thanks Daviey
[09:38] <koolhead11> Daviey: for ubuntu desktop in  lockdown enviornment
[09:38] <Daviey> soren: What would be a good example of a scheduled task to use?
[09:38] <Daviey> rbasak: you should be able to first just read directly from the fifo with minor changes to my branch..
[09:39] <Daviey> difference between a mknod and mkfifo is small :)
[09:40] <soren> Daviey: nova.compute.manager has a period_tasks that does various things.
[09:40] <soren> err...
[09:40] <soren> periodic_tasks
[09:40] <soren> Daviey: We should extend it to call driver specific periodic_tasks.
[09:40] <Daviey> rbasak: Get that working first, then think about changing the behaviour to schedule a task to read and put to txt file.. Then make get-console read from the txt file, rather than the fifo (the branch i have edits the read log part.)
[09:40] <rbasak> Daviey: I see it, thanks
[09:40] <rbasak> Daviey: why do we need root to create the fifo?
[09:41] <soren> rbasak: Only root can mknod.
[09:41] <soren> rbasak: So Daviey needed it.
[09:41] <soren> We might not need it.
[09:41] <Daviey> yeah, drop that
[09:41] <soren> (since we're only mkfifo'ing, but I'm not completely sure we're in the clear anyway)
[09:41] <soren> It depends who the guest is going to run as.
[09:42] <Daviey> you can still chown a fifo :/
[09:42] <jamespage> negronjl: hey - please give me a ping re tomcat formulas when you start your day
[09:43] <rbasak> Daviey: I'm only seeing the fifo being opened, don't see anything about reading it
[09:43] <rbasak> Daviey: have you pushed?
[09:43] <soren> Daviey: Sure, sure. I'm just pondering who needs to own it.
[09:43] <rbasak> http://bazaar.launchpad.net/~davewalker/nova/832507_with_emlog/revision/1509 is the only revision I can see
[09:43] <Daviey> rbasak: def _dump_file(self, fpath):
[09:43] <soren> Daviey: Have you used the dashboard package, by the way?
[09:43] <rbasak> Daviey: oh, right
[09:44] <Daviey> soren: no, but i believe it is known to be broken
[09:44] <Daviey> rbasak: I had to make changes to that to make it not block, i don't know if a fifo will need that like my nod did.
[09:45] <soren> Daviey: Ah.
[09:45] <rbasak> Daviey: yeah I think it will, otherwise it'll just hang there waiting for all console output ever
[09:45] <Daviey> soren: Have you tried it the dash?
[09:45] <soren> A long, long time ago.
[09:45] <rbasak> Daviey: non-blocking using Python file objects is a bit dodgy actually, it might not work and I might need to use os.read instead
[09:46] <Daviey> soren: from the package?
[09:46] <soren> Daviey: Nope.
[09:46] <Daviey> soren: I don't believe we can currently get ec2 creds from it, which is a pain.
[09:46] <soren> Daviey: WEll,...
[09:46] <soren> Daviey: Yes, I've *tried* trying it.
[09:46] <Daviey> rbasak: So it isn't using normal os, it's overidden by greenlet
[09:47] <Daviey> err, eventlet
[09:50] <rbasak> Daviey: I'm not familiar with that, what are the implications?
[09:51] <soren> Oh, dear.
[09:52] <soren> rbasak: Pretend like you don't know the first thing about blocking and non-blocking I/O.
[09:52] <soren> rbasak: Because eventlet will screw with all of that.
[09:52] <rbasak> So how should I suck all available data out of the fifo but not block?
[09:52] <soren> Do it like someone who hasn't a clue there's such a thing a blocking or non-blocking I/O.
[09:52] <rbasak> will what Daviey has already work?
[09:53] <soren> Just read.
[09:53] <soren> Haven't looked at it.
[09:53] <koolhead11> soren: is there something like nova --version ?
[09:53] <soren> koolhead11: There ought to be, but there probably isn't. We log the version in the logfiles, but I don't think we expose it that way.
[09:54] <rbasak> soren: I don't see how I can code in ignorance like that. Will my call block or won't it?
[09:54] <koolhead11> soren  dpkg -l says 2011.3. something
[09:54] <Daviey> rbasak: So what i have in that for reading, doesn't block
[09:54] <Daviey> if it blocks without it, add it in :)
[09:54] <soren> rbasak: Yes.
[09:55] <Daviey> rbasak: just opening it with NONBLOCK did infact block, thanks to eventlet
[09:55] <koolhead11> jamespage: hellos
[09:55] <soren> rbasak: I'm not trying to be more of an arse to you than eventlet is to everyone else. You just read and cross your fingers hoping that eventlet does something vaguely reasonable on the backend.
[09:56] <Daviey> rbasak: soren is right.. I couldn't work out why the hell it was blocking when it was os.open'd NONBLOCK.. it was only when i started inspecting the object that i realised it wasn't normal os
[09:56]  * jamespage waves at  koolhead11
[09:57] <Daviey> frustrating()
[09:57] <rbasak> Presumably someone chose to use eventlet here, and presumably whoever that is knows it well. Can we ask him how we should read from a FIFO?
[09:57]  * koolhead11 bows to TeTeT: 
[09:57] <koolhead11> jamespage: wassup?
[09:57] <soren> while fifo.read():
[09:57] <soren>     do_stuff_with_it()
[09:58] <rbasak> and that won't block when the fifo is starved?
[09:58] <soren> Depends on what "that" is.
[09:58] <soren> Nova, no. Your loop, yes. PRobably.
[09:58] <Daviey> rbasak: probably worth just trying it to see what happens?
[09:58] <Daviey> suck it and see. :)
[09:59] <soren> rbasak: You'd think that whoever chose it knew it really well. You'd be wrong.
[09:59] <soren> rbasak: It was chosen because it made the code easier to read for people who didn't understand async code and non-blocking i/o and whatnot.
[09:59] <rbasak> Daviey: I hate coding like that, can you tell? Otherwise it may break for an unknown unfathomable reason later, especially with stuff like this where we're heavily dependant on what's going on with the underlying fifo
[09:59] <soren> Why they're more important than my sanity, I don't know.
[10:00] <soren> This was a chance to argue endlessly about something, so of course we did.
[10:01] <Daviey> rbasak: Hey, if the unit tests pass, ship it.. that is the upstream workflow :P
[10:01]  * rbasak wonders how many race conditions there are in nova at the moment
[10:02] <soren> Daviey: Luckily, our test suite is awesome....
[10:02] <soren> Daviey: ....ly slow.
[10:02] <soren> But I digress.
[10:03] <Daviey> soren: well if rbasak makes a change that breaks a unit test, we just SKIP the unit test, right?
[10:03] <soren> Daviey: Too obvious. Just remove it.
[10:03] <rbasak> I can always "adjust" the test
[10:03] <Daviey> heh
[10:04] <Daviey> rbasak: Sorry, i was commenting on the fact that nova has history of disabling tests which are annoying and fail.
[10:04] <Daviey> This was yet another area that upstream were able to argue the moral toss about.
[10:04] <soren> And I have a history of crying myself to sleep.
[10:04] <soren> These facts are related.
[10:27] <koolhead11> soren: http://paste.ubuntu.com/685140/  nova version :-> http://paste.ubuntu.com/685142/
[10:29] <soren> koolhead11: Zones are mysterious to me. I suggest you ask someone like sandywalsh.
[10:31] <koolhead11> Daviey: i have my fingures crossed, we will be testing new oneiric images  via cobbler :)
[10:33] <Daviey> koolhead11: cool
[10:36] <rbasak> https://bugs.launchpad.net/ubuntu/+source/asterisk/+bug/831228
[10:36] <rbasak> This is a problem with the libc-client.so build
[10:37] <rbasak> Even if I rebuild the debian binary, it's not import needed symbols
[10:37] <rbasak> (but the debian binary has it right)
[10:37] <Daviey> rbasak: Have you come across ld --as-needed?
[10:37] <rbasak> no
[10:38] <Daviey> rbasak: http://wiki.debian.org/ToolChain/DSOLinking
[10:39] <Daviey> rbasak: which implies that uw-imap needs an extra LDFLAG?
[10:39] <rbasak> yeah, maybe
[10:40] <rbasak> is there a reason for debian and ubuntu to behave dfferently?
[10:40] <rbasak> I haven't tested to see if debian rebuilds it right or not
[10:40] <Daviey> rbasak: Debian will move to ld --as-needed one day it's suspected.
[10:41] <Daviey> Some people feel it's easier for us to make the transsiiton first, then Debian.
[10:42] <soren> Man, if we had to wait for Debian before we tried interesting toolchain changes...
[10:42] <rbasak> So what's the current status?
[10:45] <rbasak> It doesn't seem to just be new libraries
[10:45] <rbasak> It's _all_ libraries
[10:48] <Daviey> rbasak: yup!  Which is the main reason we have so many FTBFS's
[10:48] <Daviey> fwiw, this was tried last cycle - and was reverted before release.
[10:48] <Daviey> It could happen again, but i'm doubtful.
[10:56] <rbasak> yeah -Wl,--no-as-needed fixes it
[10:56] <rbasak> so now I just have to find a proper solution
[10:59] <Daviey> rbasak: well that will be ok for a last resort, just need to find out what the missing lib is.
[11:00] <rbasak> Daviey: well, all of them are missing
[11:00] <rbasak> that's the odd thing
[11:00] <rbasak> even though they are specified on that same line
[11:03] <Daviey> rbasak: it culd be an ordering issue then
[11:03] <Daviey> could*
[11:03] <rbasak> the current line is: gcc --shared -Wl,--no-as-needed,-soname,libc-client.so.$(VERSION) $(PAMLDFLAGS) -lc $(EXTRALDFLAGS) $(SSLLDFLAGS) -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -o
[11:03] <rbasak> (I added --no-as-needed)
[11:04] <rbasak> without --no-as-needed, that results in ldd reporting only libc and ld.so, linux-gate etc
[11:04] <Daviey> -o needs to be before LDFLAGS
[11:04] <Daviey> 12:04 -!- doko_ [~doko@dslb-088-073-086-171.pools.arcor-ip.net] has quit [DISGUSTED]
[11:04] <rbasak> that makes a difference?
[11:04] <Daviey> yah!
[11:05] <rbasak> OK then the build system has  problem
[11:05] <Daviey> rbasak: For example, http://launchpadlibrarian.net/79186443/monkey_0.9.3-1_0.9.3-1ubuntu1.diff.gz
[11:05] <rbasak> It's really a variable that's being set, it assumes that -o is at the end
[11:05] <Daviey> Yeah, that is the bug
[11:07] <rbasak> http://paste.ubuntu.com/685163/ look at line 180
[11:07] <rbasak> To fix this, the build system is going to need a radical change
[11:07] <rbasak> echo "$(ARRC) $(ARCHIVE) $(BINARIES);$(RANLIB) $(ARCHIVE)" > ARCHIVE
[11:08] <rbasak> looks like that's how it works
[11:09] <rbasak> I suppose I could add an ARRC_AFTER_ARCHIVE or something
[11:10] <Daviey> :/
[11:11] <rbasak> It does look like a rather insane build system
[11:12] <Daviey> rbasak: :(
[11:25] <koolhead17> TeTeT: need your help. ping me back when your around.
[11:34] <rbasak> it does feel a bit odd that to fix an asterisk ftbfs I'm hacking an imap library makefile
[11:34] <rbasak> wtf does asterisk do with imap anyway?
[11:35]  * rbasak resists running configure with --without-imap
[11:36] <Daviey> rbasak: stores voicemail in imap as an option
[11:36] <Daviey> pretty nifty feature to sync it to a mail account TBH.. when you mark the email as read, asterisk knows the voicemail has been listened to :)
[11:36] <rbasak> Ah I see
[11:37] <rbasak> I was going to suggest that if only there were some kind of common protocol to trasmit IMAP messages over the the Internet :-P
[11:37] <Daviey> heh
[11:37] <rbasak> (but I suppose for the mark as seen case I'll accept it :-)
[11:43] <TeTeT> koolhead17: sure, what's the problem?
[11:45] <rbasak> Daviey: In http://launchpadlibrarian.net/79186443/monkey_0.9.3-1_0.9.3-1ubuntu1.diff.gz you've moved two things - $OBJ to the front and $LDFLAGS to the back. Do I need both?
[11:45] <rbasak> Or does the location of $OBJ not matter?
[11:48] <Daviey> OBJ is a weird thing that only that has, unless i am mistaken
[11:49] <rbasak> I assume it's the list of source object files
[11:49] <rbasak> Needs to go somewhere
[11:50] <rbasak> Oh, or is that $mod_obj in your case?
[11:51]  * Daviey can think of somewhere to put it.
[11:51] <Daviey> ah yes
[11:51] <rbasak> lol
[11:51] <rbasak> OK, that worked
[11:53] <Daviey> rocking!
[11:53] <Daviey> rbasak: BTW, did you add your DEBEMAIL to your launchpad account?
[11:56] <koolhead17> TeTeT: curently am using lucid LTS for deployments and we had to lockdown the desktop apps so we used gconf there, now when i will upgarde to new LTS it will have Unity in it. It means all configurations woll break?
[11:57] <TeTeT> koolhead17: I suspect so
[11:57] <TeTeT> koolhead17: you might need to have a custom package that installs lock downs for unity / gnome 3 based on the old gconf settings
[11:58] <koolhead17> TeTeT: so how will i achive that :(
[11:59] <koolhead17> also from now on for coming deployments i have to read/tweak Unity for the same lockdown approach
[12:00] <TeTeT> koolhead17: I guess finding the counter for the old lockdown settings in the new distro will do the job. Though I suspect there will not always be 1:1 replacement
[12:00] <koolhead17> TeTeT: and also suggest me some alternative i should work on for the same
[12:01] <TeTeT> koolhead17: I'd recommend to grab an oneiric desktop in a vm and see if you can lock it down in a similar way as 10.04 LTS. If some pieces are missing, file bugs or talk to the devels directly and see if it can be integrated
[12:02] <koolhead17> TeTeT: hmm. Some tough work lined up it seems for me. :D
[12:03] <TeTeT> koolhead17: not only for you, there are lots of deployments that face massive changes with the move to gnome 3 and unity
[12:04] <koolhead17> TeTeT: so we are on same side!! :D
[12:17] <rbasak> Daviey: just done it now. It needed me to merge my @ubuntu.com address into my main launchpad account, now done
[12:18] <Daviey> rbasak: cool, otherwise launchpad doesn't know it was your upload
[12:26] <Sander^work> Hi. I get: make: cc: Command not found
[12:26] <Sander^work> Tried to install gcc, but it says: Package gcc is not available, but is referred to by another package.
[12:27] <rbasak> Try installing build-essential
[12:28] <Sander^work> Can't find it
[12:28] <pmatulis> !info build-essential
[12:29] <Pici> Sander^work: What release of Ubuntu are you using?
[12:29] <Sander^work> 10.04
[12:29] <Sander^work> newly installed
[12:29] <Pici> Sander^work: Please run apt-get update and then see if you can install build-essential again.
[12:30] <Sander^work> Pici, thanks :-)
[12:36] <rbasak> Daviey: please pull and upload lp:~racb/+junk/uw-imap. LP #831228. Hope the fix isn't too hacky, it's the cleanest i could come up with without being too intrusive.
[12:37] <rbasak> Daviey: After that do we need to do a no-change update for asterisk or will it rebuild by itself?
[12:38] <Daviey> rbasak: can you propose it for merging?
[12:38] <Daviey> you could even try bzr lp-propose-merge ? :)
[12:39] <Daviey> rbasak: Does asterisk actually need rebuilding?  There is a binary in the archive, it just wouldn't rebuild..
[12:39] <Daviey> It needs rebuilding if the binary is currently broken tho
[12:39] <rbasak> Daviey: I'd love to...how?
[12:39] <rbasak> oh
[12:39] <koolhead17> TeTeT: http://developer.gnome.org/gio/stable/ch28.html But i doubt it will work for next  LTS :P
[12:40] <rbasak> bzr: ERROR: File exists: '/srv/bazaar.launchpad.net/mirrors/00/07/cf/61'
[12:40] <rbasak> Ah OK, then it doesn't need rebuilding
[12:40] <rbasak> It would be nice to know if it rebuilds to close the bug though
[12:41] <rbasak> Is there a way to propose a merge through the web app?
[12:42] <Daviey> rbasak: yes
[12:42] <Daviey> bzr lp-open ; click "propose merge"
[12:42] <Daviey> ah dammit
[12:42] <Daviey> it's in +junk
[12:42] <rbasak> where should I have pushed it o?
[12:42] <rbasak> to?
[12:43] <Daviey> needs to be in lp:~racb/ubuntu/uw-imap/BRANCH-NAME
[12:43] <Daviey> (maybe BRACH-NAME = bug number)
[12:44] <soren> Will that work?
[12:44] <soren> I thought it needed to be ~XXX/ubuntu/uw-imapd/oneiric/BRANCH-NAME
[12:44] <soren> With ~XXX/ubuntu/uw-imap just being an alias.
[12:45] <rbasak> bzr push lp:~racb/ubuntu/oneiric/uw-imap/831228 worked
[12:45] <rbasak> bzr push lp:~racb/ubuntu/uw-imap/831228 did not
[12:46] <zul> holy crap uw-imap...old school
[12:46] <soren> right, of course.
[12:46] <zul> uw-imap was the first imap server i used....then i growed up
[12:47] <Daviey> soren: Ah, yes - that was an intentional mistake to see who was paying attention]
[12:47] <Daviey> zul: You are a big boy now!
[12:47] <nigelb> Rule #284058: When wrong, always claim it was a test.
[12:48] <zul> Daviey: yes i took the pacifier out
[12:49] <rbasak> OK merge proposed
[12:54] <rbasak> zul: well asterisk depends uw-imap. what fun!
[12:57] <zul> rbasak: heh..
[13:42] <smoser> that was a fun conversation up above, regarding console
[13:43] <smoser> the thing that i'm concerned about is that you're going to have to be polling that buffer fifo fairly often, aren't you?
[13:43] <smoser> rbasak, ^ ? soren ^?
[13:43] <soren> smoser: Why?
[13:44] <smoser> because on kernel boot (or potentially at any point) you will have a fairly quick 30k set of writes
[13:44] <soren> smoser: The kernel has a 64k buffer for named pipes.
[13:44] <smoser> so if i turn kernel messages to verbose then i just block boot until the daemon comes trhough and reads.
[13:45] <soren> MAybe.
[13:45] <smoser> or (actual real life) if i use a kexec loader, and get 2 kernels worth of output
[13:45] <soren> I haven't actually checked that.
[13:45] <soren> I woulnd't be overly surprised if it just discarded anything exceeding those 64k.
[13:45] <soren> "it" == qemu.
[13:45] <smoser> i happen to know that 2 kernels worth of boot will not fit in 64k thanks to bug 566793
[13:46] <soren> If you think we need to run it more frequently than once a minute, that's not a problem. I just didn't think it would be.
[13:46] <smoser> i'd be surprised if qemu is doing anything more intelligent than writing to the target.
[13:46] <soren> Well, for one it doesn't hang if nothing is listening.
[13:47] <smoser> listening to the fifo ?
[13:47] <smoser> it doesn't ?
[13:47] <soren> Yes.
[13:47] <soren> No.
[13:47] <smoser> ah.
[13:47] <smoser> well then clearly i'm wrong
[13:47] <soren> So it's certainly smarter than something htat just assumes writing to a pipe works.
[13:47] <soren> It even buffers the stuff that isn't read (Because nothing is connected) and flushes it to whoever comes along and starts reading.
[13:48] <smoser> so how will this solution play withthe other uses of that console ?
[13:48] <soren> Which other uses?
[13:48] <smoser> i think there was disicussion at least of some way to multiplex that output
[13:48] <smoser> so that a user could get an ajaxterm or something and not break the console output
[13:48] <smoser> putting something in between.
[13:48] <soren> We'll deal with that when it starts to happen.
[13:48] <smoser> i dont know .... just thinking.
[13:48] <smoser> :)
[13:48] <soren> No point in engineering a whole bunch of stuff nothing will need/use.
[13:48] <smoser> true
[13:49] <smoser> i'm really curious what kvm does with the writes  if no one is listening.
[13:49] <soren> It's not like something is going to come along from the side wanting to look at this. These are Nova's VM's.
[13:49] <soren> I don't know for sure, but I see a couple of options.
[13:49] <soren> Doing it in a thread.
[13:49] <smoser> i wonder where my laptop writes when i'm not here to look at the screen and read its buffer
[13:49] <soren> ...will just hang, but who cares? It's a separate thread..
[13:50] <soren> or doing non-blocking I/O, using select() to wait for something that's willing to listen.
[13:50] <smoser> yeah, i think its easonable.
[13:50] <smoser> reasonable
[13:50] <soren> smoser: Your laptop is a different story.
[13:50] <soren> I could tell you, but that would ruin the fun.
[13:51] <smoser> i just assumed  that amazon had the 4 minute delay just to stop polling on the console
[13:51] <smoser> but i suspect now that it is more complex, and possibly related to this :)
[13:51] <soren> I'm having troube figuring out why they do that.
[13:51] <soren> It makes no real sense to me.
[13:52] <soren> It seems incredibly arbitrary and inelegant.
[13:52] <soren> Any problem I can envision that would be solved by something like that would be so much more elegantly solved by different means.
[13:52] <smoser> because they only have to have their fifo-emptier come through every 4 minutes then.
[13:52] <smoser> and can even schedule it to happen 4 minutes after hypervisor detects reboot
[13:52] <soren> And that's only a problem if it's one, great big fifo-emptier for all of EC2.
[13:52] <smoser> i agree.
[13:53] <soren> OUrs just runs on the compute node. It would at the most have to scale to... I don't know, maybe 64 guests.
[13:53] <smoser> its a interesting problem.
[13:53] <soren> Hardly a major problem.
[13:53] <soren> Now, if they run on each node, but send the stuff they collect to a central node or shove it in a database, that would also explain it.
[13:54] <soren> But why do that? To save a roundtrip to the compute nodes from their frontends if someone asks for this?
[13:54] <smoser> i really doubt that a call to GetConsoleOutput hits the node
[13:54] <smoser> anyway.
[13:55] <smoser> i just find it surprising that this is so complicated and there wasn't an existing solution for this sort of thing.
[13:57] <LinuxAdmin> hi guys
[13:57] <soren> If a GetConsoleOutput call doesn't hit the node, they're storing these things centrally. That sounds like a fascinating way to scale somthing like this.
[13:57] <soren> But meh.
[13:57] <LinuxAdmin> I forgot the password of a KVM virtual machine. how can I recover it?
[13:58] <LinuxAdmin> I've got virt-manager installed
[13:58] <LinuxAdmin> on a different host
[13:59] <soren> LinuxAdmin: Same way you'd recover it from any other box.
[13:59] <soren> LinuxAdmin: Boot into resuce mode.
[13:59] <soren> LinuxAdmin: Or take its disk somewhere else, mount it, reset the password, put it back into the machine.
[13:59] <_ruben> bah .. keepalived doesn't seem to work well when using unnumbered interfaces :(
[13:59] <LinuxAdmin> how can I boot a VM into rescue mode?
[13:59] <soren> LinuxAdmin: How d oyou boot any other machine into rescue mode?
[14:00] <soren> (The astute reader will notice a pattern here)
[14:00]  * ersi facepalms
[14:00] <soren> _ruben: Unnumbered interfaces?
[14:00] <soren> _ruben: What does that mean?
[14:00] <ersi> LinuxAdmin: By rebooting the machine and haxxing it up in the "boot prompt"
[14:00] <_ruben> soren: interface without ip address configured
[14:00] <soren> _ruben: Oh.
[14:01] <LinuxAdmin> soren, I type "single" in the boot process
[14:01] <LinuxAdmin> but I don't do it a long time
[14:02] <jasonmsp> hey all.  My server was used last night in a DOS attack in an attempt to take out some DNS servers.  The hole that it used was due to the fact it is coming up as open resolver.  I've been reading up all day and can not find how to close it.  Any love?
[14:03] <LinuxAdmin> I tryed restart the machine and then open immediately the virt-manager window, so I can see the boot process, but I could not interact with it.
[14:03] <LinuxAdmin> how can I do it?
[14:03] <soren> LinuxAdmin: Or take its disk somewhere else, mount it, reset the password, put it back into the machine.
[14:04] <LinuxAdmin> soren, I'm talking about a virtual machine, its disk are virtual
[14:04] <soren> Yes.
[14:05] <soren> I realise.
[14:05] <soren> That makes it even easier.
[14:05] <soren> No need for screwdrivers.
[14:05]  * soren needs to take a break
[14:07] <rbasak> soren, smoser: just catching up with that
[14:07] <rbasak> AFAICT, qemu ignores failed writes
[14:07] <smoser> but writes to a file dont fail
[14:07] <rbasak> I left a machine running outputting junk and timestamps to the console overnight
[14:07] <smoser> :)
[14:07] <rbasak> Left it without connecting a reader to the fifo
[14:08] <rbasak> the timestamps started approx when I resumed reading the fifo
[14:09] <rbasak> Nova is a persistent process, right? Can we hook into the eventing system to have an event when the fifo is available for reading?
[14:09] <rbasak> (rather than trying to poll)
[14:12] <paljas> Which setting on apache 2.2 on ubuntu makes that php pages are interpreted
[14:12] <paljas> without the extension being given in the request? Can't find it in the
[14:12] <paljas> config. It looks like /etc/apache2/mods-enabled/php5.conf only has some
[14:12] <paljas> lines to matches .php file to the php handler, but apache also  matches
[14:12] <paljas> things like /path/file instead of /path/file.php to the php handler.
[14:15] <patdk-wk> there is no magic way to do that
[14:16] <patdk-wk> cause how is apache suppost to know you wanted to use php for it? instead of like perl? or ruby?
[14:16] <patdk-wk> the only magic you can do, is attempt a rewrite, to see that when someone asks for /path/file, to check if /path/file.php exists, if it does, use it
[14:17] <just-a-visitor> It sounds like a rewrite rule.
[14:17] <just-a-visitor> Oh.
[14:17] <paljas> It's not that I want it, but it does.
[14:17] <paljas> Could be done by: RewriteRule !\. - [H=application/x-httpd-php]
[14:17] <paljas> but that's not in the config
[14:18] <patdk-wk> heh? why would ubuntu have that in the config at all?
[14:18] <patdk-wk> that isn't *normal*
[14:19] <norc> I have an OVH server with Ubuntu Server 10.04 running. When I install a standard kernel, remove the OVH kernel and run update-grub and then reboot - the server doesn't come back up again and needs reformating
[14:19] <patdk-wk> why would it need reformatting?
[14:19] <norc> patdk-wk: because it doesnt boot back
[14:19] <norc> For which reason I cannot determine
[14:19] <patdk-wk> not booting and reformatting are two totally unrelated things
[14:20] <norc> patdk-wk: point is, I dont have serial console access
[14:20] <norc> If it doesnt come back up, the only thing I can do is hit the reformat button
[14:20] <patdk-wk> what version of grub is on it?
[14:20] <patdk-wk> oh? it's some kind of vps thing
[14:20] <patdk-wk> likely they had some customizations, that normal ubuntu grub killed
[14:20] <_ruben> which might require special initrd stuff which is now missing
[14:21] <norc> patdk-wk: grub-setup (GRUB) 1.98-1ubuntu10
[14:21] <norc> no its a dedicated root serer
[14:21] <norc> but it does have a custom kernel
[14:21] <norc> mhmmm
[14:21] <_ruben> physical hardware with a reformat button? interesting
[14:21] <norc> _ruben: no, a web interface with a reformat button.
[14:22] <_ruben> doesn't make it less interesting .. it's a typical feature for virtualized stuff
[14:22] <norc> Well its rather common for server providers these days
[14:22] <paljas> patdk-wk: xxx.xxx.220.152 - - [08/Sep/2011:16:20:15 +0200] "GET /aap.php?=SUHO8567F54-D428-14d2-A769-00DA302A5F18 HTTP/1.1" 200 3042 "http://dateldb.net.xxx.nl/aap" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.22) Gecko/20110905 Ubuntu/10.04 (lucid) Firefox/3.6.22"
[14:22] <norc> patdk-wk: how would I go about verifying grub integrity?
[14:23] <patdk-wk> paljas, random logs don't make sense, without a referense as to why it's posted
[14:23] <paljas> to show you ubuntu _does_ the rewrite
[14:24] <patdk-wk> rewrites aren't logged like that
[14:24] <patdk-wk> that was a redirect
[14:24] <patdk-wk> probably cause of multiviews being on
[14:24] <patdk-wk> so the user got a page saying, you asked for app, but it doesn't exist, but app.php does, did you really mean that?
[14:24] <paljas> I'll have a look at multiviews.
[14:25] <rbasak> smoser: so what do you suggest?
[14:25] <smoser> i dont suggest that i have a solution.
[14:26] <smoser> i'm just honestly surprised that this is this complicated
[14:26] <smoser> :)
[14:26] <smoser> i dont know about polling, does inotify support fifo having data ?
[14:27] <paljas> patdk-wk: it is indeed due to multiviews. Thanx a lot!
[14:27] <rbasak> smoser: no, but any sensible eventing framework woiuld support it, eg. a select or poll loop
[14:28] <rbasak> smoser: I don't know eventlet though
[14:28] <zul> nova is suppose to be getting an event based framework but not what you guys are talking about
[14:29] <rbasak> I suppose I need to study the code more, and do whatever the rest of the code does when it wants to listen for events on a fd
[14:29] <rbasak> (assuming it is capable of that
[14:29] <rbasak> )
[14:34] <Daviey> smoser: ideally qemu would provide a ringbuffer interface, i think - that get_console could poll and flush to file, and append fresh output, keeping the text file at 64K i think.
[14:35] <Daviey> But a generic solution, seems to be why emlog was invented.  Just a kicker the author didn't seem to try and get it mainlined.
[14:36] <Daviey> it is starting to taste like getting emlog-dkms into the archive might be less pain.. but hardly fair as an upsteam solution.
[14:36] <rbasak> I'd prefer a generic solution in userspace
[14:36] <rbasak> libringbuffer
[14:37] <Daviey> Anyway, lets see what rbasak churns out :)
[14:37] <rbasak> trouble about putting ringbuffer support in qemu is that support would be needed all the way through the stack
[14:37] <rbasak> libvirt would need to support that option
[14:37] <rbasak> nova would need to ask for it
[14:37] <rbasak> etc
[14:38] <Daviey> ture
[14:38] <rbasak> the fifo answer is really a hack which is useful because qemu happens to be able to write to it without* issues
[14:39] <rbasak> also with qemu writing to the ringbuffer and nova get_console_output reading from it, there would have to be a standard ringbuffer format and there isn't an obvious file-based one, hence the need for a libringbuffer
[14:39] <rbasak> that's the ideal solution but I don't think the time it would take can be justified
[14:39] <Daviey> yah
[14:40] <rbasak> (also I'd want to have agreement from upstream)
[14:40] <soren> rbasak: You can just spawn a thread that reads from that fifo, really.
[14:40] <soren> rbasak: In a tight loop.
[14:41] <rbasak> soren: that's really horrible but I suppose I'll do it if I have to. I'll need a lock on the ringbuffer then too :-/
[14:42] <soren> rbasak: You will anyway?
[14:43] <rbasak> soren: maybe. I haven't looked at eventlet, remember? And you said to ignore its presence, and nobody told me that it used threads :)
[14:44] <Daviey> rbasak: interesting, http://laforge.gnumonks.org/weblog/2004/08/18/#20040818-syslog-performance
[14:46] <soren> rbasak: It doesn't.
[14:46] <soren> rbasak: :)
[14:46] <soren> rbasak: Well, it uses greenthreads.
[14:46] <rbasak> Daviey: yeah that's the kind of thing I have in mind for the ideal solution
[14:46] <rbasak> Daviey: I'd use a disk-based format, the library would mmap it and use mutexes
[14:47] <rbasak> (not really for performance, but just for the shared ringbuffer really)
[14:49] <smoser> i think the ideal solutoin actually modifies kvm
[14:49] <smoser> to support this.
[14:49] <rbasak> smoser: and thus modifies libvirt and nova too?
[14:49] <smoser> well, yeah.
[14:49] <smoser> obviously kvm is the right place, its the closest to it.
[14:49] <smoser> supporting writing content to a ring buffer.
[14:49] <smoser> no?
[14:50] <Daviey> smoser: The problem is, there is no such thing as a userspace or mainline kernel generic ringbuffer
[14:50] <Daviey> If one is provided as named pipe, kvm already supports it :)
[14:51] <rbasak> well, qemu already supports writing to a socket
[14:51] <rbasak> and indeed a pair of named pipes
[14:51] <Daviey> incidently, screen works against connecting to a unix pipe.  I like that.
[14:52] <rbasak> so if we are always going to have a qemu monitoring process, then I don't think ringbuffer _has_ to be in qemu for the result to be clean
[14:53] <rbasak> in fact I think it would be reasonable if qemu upstream rejected a ringbuffer patch for this reason
[14:53] <rbasak> trouble is that I don't think libvirt allows for a monitoring process, and it sits in the middle of our stack
[14:54] <rbasak> although I suppose it would be reasonable for libvirt to support a socket destination for console logging and to pass that through from nova to qemu
[14:55] <rbasak> but we're coming back to nova supporting events on a file descriptor again
[14:55] <soren> It does.
[14:56] <rbasak> OK so I think the only ugliness really is the fifo without telling qemu
[14:57] <rbasak> so if we implement as planned (with nova monitoring the fifo better than a poll if possible) and there are issues then that would need to be fixed by actually using qemu's pipe type and modifying libvirt to pass that request through
[14:57] <soren> Sorry, what?
[14:58] <soren> Qemu's pipe type is what you'll use as a fallback? What's plan A?
[14:59] <rbasak> Plan A is using qemu's file type which is actually a fifo
[14:59] <rbasak> (as in, we make the fifo and then tell qemu to write to it as a file)
[15:00] <rbasak> I thought that was the plan all along, otherwise how do we do it without modifying libvirt?
[15:44] <ScatterBrain> Is AppArmor worth keeping on a Web Server?
[15:45] <jpds> ScatterBrain: Possibly.
[15:46] <jdstrand> ScatterBrain: sure, especially if you are using mysql and/or confine specific applications via http://wiki.apparmor.net/index.php/Mod_apparmor_example
[15:46] <jdstrand> s/specific/specific web/
[15:46]  * ScatterBrain looking at link.
[15:47] <norc_> Ok let me approach this from another angle. I installed a kernel via apt, then moved the ovh kernel and /etc/grub/06_OVH to a temporary dir, hit update-grub and then reboot - this should technically boot directly into the new kernel then right?
[15:48] <patdk-wk> depends on grub's settings
[16:21] <zul> lunches
[16:30] <garymc> I still cant get server to load dhcp3 after server reboot. Any ideas anyone?
[16:41] <jamespage> RoAkSoAx: around? need a bit of help pxebooting my panda
[16:46] <RoAkSoAx> jamespage: i'm here
[16:46] <jamespage> hey RoAkSoAx
[16:46] <jamespage> I'm trying to get my pandaboard to pxe boot and install using cobbler (oneiric)
[16:47] <RoAkSoAx> jamespage: ok, where are you stuck?
[16:47] <jamespage> its booting OK from the sd card - but I got an error trying to add the sysem
[16:47] <RoAkSoAx> jamespage: pastebinit ;)
[16:48] <jamespage> <type 'exceptions.IOError'>:[Errno 2] No such file or directory: '/etc/cobbler/pxe/pxesystem_arm.template'
[16:48] <RoAkSoAx> jamespage: interesting, let me test
[16:49] <jamespage> RoAkSoAx: the message is right i.e. the file does not exist
[16:49] <RoAkSoAx> jamespage: alright, so it might just be a packaging error
[16:49] <RoAkSoAx> let me check
[16:54] <RoAkSoAx> jamespage: btw.. I think you'll have to trick the preseed to get manual partitioning cause in my case, everything installed successfully however, it failed to come up
[16:54] <jamespage> RoAkSoAx: right-oh
[16:55] <jamespage> I was going to try installing to a usb disk to try an improve performance
[16:55] <RoAkSoAx> jamespage: ok.. anyways I'm building the apckage now
[16:55] <jamespage> RoAkSoAx: OK - so I hacked in the pxe*_arm.templates from the source pckage
[16:55] <RoAkSoAx> should be able to get it fixed soon
[16:55] <jamespage> for the time being
[16:55] <jamespage> :-)
[16:56] <jamespage> its doing somehting
[16:56] <jamespage> installing I think
[16:56] <jamespage> but nothing on the serial line - maybe I need to set some more kernel options
[16:56] <RoAkSoAx> jamespage: yes you do
[16:58] <RoAkSoAx> sudo cobbler system edit --name=balblabla --kopts="blabla" I believe it is
[17:01] <jamespage> yep - picked then from here http://rsalveti.wordpress.com/2011/07/11/net-booting-with-tftp-and-pxe-with-pandaboard/
[17:03] <utlemming> smoser: around?
[17:04] <RoAkSoAx> jamespage: cool, alright, I';ll be uploading the fix in a bit
[17:04] <RoAkSoAx> forgot to install them in cobbler-common
[17:04] <jamespage> np
[17:05] <RoAkSoAx> jamespage: good catch though ;)
[17:05] <jamespage> I did not need todo your hack with the pxelinux.cfg/* file - that worked fine
[17:05] <RoAkSoAx> jamespage: yeah they fixed it already AFAIk
[17:11] <GrueMaster> I am interested in this cobbler for panda pxe boot.  I already have pxeboot working here (quite well, actually), except for doing custom preseeds & custom pxelinux.cfg.
[17:12] <GrueMaster> (and the occasional issue of not being able to install to the advertently unplugged usb drive - oops).
[17:19] <jamespage> GrueMaster: I'll let you know how I get on - I want to network install to USB drive so may need a little help there
[17:20] <GrueMaster> Only real issues I have seen lately are related to bug 806751, and recently something has changed to where it now prompts me for keyboard detection again.
[17:27] <koolhead17> hey all
[17:31] <sms_> a noob me just installed ubuntu server, is there a tutorial somewhere how I can control the server from my win7 pc?
[17:34] <sms_> By control I mean see and use the terminal on the win7 pc basically..
[17:34] <jamespage> sms_: try putty
[17:34] <sms_> will do
[17:42] <sms_> oh wow
[17:42] <sms_> Didn't think it would be this simple
[17:42] <sms_> Yess :)
[18:03] <RoAkSoAx> jamespage: how's the installation going?
[18:18] <jamespage> RoAkSoAx: worked like a dream - had to work my own preseed for install to usb disk
[18:26] <GrueMaster> jamespage: I could have sent you mine to work from.
[18:27] <jamespage> well it did not need to much alteration TBH
[18:27] <jamespage> just fixing the disk to install to
[18:28] <GrueMaster> I'm curious, can a preseed file use an include statement to pull a sub-section for preseed parameters?
[18:29] <GrueMaster> (I haven't worked with preseeding that much).
[18:29] <RoAkSoAx> jamespage: is it booting?
[18:29] <jamespage> RoAkSoAx: yep
[18:29] <jamespage> can figure out how to make it netboot again tho
[18:29] <jamespage> can't that is
[18:30] <RoAkSoAx> jamespage: did you create the two partitions on the USB stick?
[18:30] <jamespage> yes
[18:30] <RoAkSoAx> jamespage: would be could if you could write down your steps so I can play aroudn wtih what you did too
[18:31] <GrueMaster> jamespage: The easiest way I have found is to reformat the SD and copy /usr/lib/u-boot/omap4_panda/[MLO|u-boot.bin] (in that specific order) to the SD.
[18:32] <GrueMaster> Then u-boot will look to the bootp server for netboot.scr first.
[18:54] <ScatterBrain> Ok, really showing my noobness here.  If am installing a LAMP Server (for development purposes) and I'm ONLY going to be talking to MySQL on the local machine (via bash scripts, and PHP), the default setting in /etc/mysql/my.conf to bind to the loopback address is correct, right?
[18:55] <kirkland> RoAkSoAx: ping
[18:56] <RoAkSoAx> kirkland: pong
[18:56] <kirkland> RoAkSoAx: are you getting bzr crashes when running release-build?
[18:56] <kirkland> RoAkSoAx: looks like something in bzr build-deb broke very recently
[18:56] <kirkland> lifeless: ^
[18:56] <RoAkSoAx> kirkland: yes
[18:56] <kirkland> RoAkSoAx: have you triaged it?
[18:56] <ScatterBrain> The server will be accesible from the Internet, but I only want commands being sent to MySQL from my web applications and the local backup scripts.
[18:56] <RoAkSoAx> kirkland: i got it fixed by commenting this:
[18:57] <RoAkSoAx> kirkland: http://paste.ubuntu.com/685506
[18:57] <RoAkSoAx> kirkland: lines 54,55
[18:57] <kirkland> RoAkSoAx: ugh
[18:57] <kirkland> RoAkSoAx: that sucks
[18:57] <kirkland> RoAkSoAx: i guess we can sign it later
[18:58] <kirkland> RoAkSoAx: but i think that's a bug in bzr build-deb
[18:58] <lifeless> kirkland: hi; tell jelmer / james_w / poolie :P
[18:58] <lifeless> kirkland: I'm on leave atm
[18:59] <kirkland> lickalott: thanks
[18:59] <kirkland> lifeless: ah, right, congrads ;-)
[18:59] <RoAkSoAx> kirkland: yeah it wont hurt for now
[19:01] <lifeless> kirkland: thanks :P - if I wasn't about to hop in the car and go pick lynne up (again :P) I would poke at it for you, but I am, so I won't.
[19:02] <kirkland> lifeless: np, later
[19:16] <kev009> any opinions on zenoss vs opsview for monitoring?
[19:27] <jamespage> RoAkSoAx: will document somewhere handy tommorow
[19:29] <RoAkSoAx> jamespage: thanks
[19:51] <RoAkSoAx> jamespage: what partiotioning layout did you use?
[19:53] <lborda> kim0, hi
[19:53] <kim0> lborda: hey there
[19:53] <lborda> kim0, I'm filling a merge proposal for the serverguide. Should I add you as a reviewer?
[19:54] <kim0> lborda: not really, whoever can review will .. I don't like becoming a bottleneck :)
[19:54] <kim0> lborda: congrats on the great work though
[19:54] <kim0> lborda: keep em coming
[19:55] <lborda> kim0, not a problem... it's a pleasure...
[19:57] <kim0> great thanks!
[19:58] <RoAkSoAx> jamespage: are you using today's images though?
[20:37] <sms_> two computers connected to the same router, what featurs must be on in the router so I can ftp into one computer to the other?
[20:38] <sms_> from the other*
[20:38] <patdk-lap> how are they connected to the router?
[20:38] <patdk-lap> using different nics? vlans? ...
[20:38] <sms_> rj45 cable?
[20:38] <sms_> I'm pretty confused with this networking stuff
[20:39] <patdk-lap> the same rj45 cable?
[20:39] <sms_> No, the router has 4 slots
[20:39] <patdk-lap> sounds scary
[20:39] <patdk-lap> what kind of *router* is this?
[20:39] <sms_> hehe
[20:39] <sms_> Uhh
[20:40] <sms_> a-link roadrunner 44
[20:40] <Pici> Normally you don't need to do anything on a router for two clients under it to communicate.
[20:40] <sms_> so adsl router? I guess
[20:40] <patdk-lap> ya, sounds like nothing needs to happen
[20:40] <patdk-lap> as long as the router is providing dhcp info, or you used static ip's
[20:41] <Pici> sms_: have you installed an ftp service on the server you'd like to ftp into?
[20:41] <sms_> I can ftp into my server, but apparently the connection went over the internet
[20:42] <sms_> So I guess I used the external ip
[20:42] <sms_> and this is where I'm lost
[20:42] <Pici> So use the internal IP.
[20:42] <sms_> Where can I find it
[20:42] <patdk-lap> on the computer running the ftp server
[20:43] <sms_> what command?
[20:43] <patdk-lap> dunno, what os is running on it?
[20:44] <sms_> ubuntu
[20:44] <patdk-lap> ifconfig
[20:44] <sms_> so uhh
[20:45] <sms_> eth0: info
[20:45] <sms_> lo: info
[20:46] <sms_> which one is the internal ip :S
[20:46] <patdk-lap> all of them :)
[20:46] <patdk-lap> but your looking for the eth0 one
[20:46] <sms_> umm
[20:47] <sms_> inet addr:... is my external ip I think
[20:47] <sms_> Because I can access it from anywhere..
[20:47] <patdk-lap> post it
[20:47] <sms_> 88.115.187.159
[20:47] <patdk-lap> hmm, odd
[20:48] <patdk-lap> what ip is on the other machine?
[20:48] <sms_> 91.155.133.247
[20:48] <sms_> the one I'm on now
[20:48] <Pici> weird.
[20:48] <patdk-lap> both plugged into that same router?
[20:48] <sms_> yes
[20:49] <Pici> Those are both publically addressable ips.
[20:49] <patdk-lap> must have got multichannel from the isp then
[20:49] <patdk-lap> or whatever they call that on dsl
[20:49] <sms_> yeah? :D
[20:50] <sms_> So is it a setting in the router or something
[20:50] <sms_> I mean
[20:50] <patdk-lap> you have to have fun with routes to get that to work nicely :)
[20:50] <sms_> I keep reading 192.168.0.0 from everywhere
[20:50] <sms_> so I'm guessing that's if I had an internal ip
[20:50] <patdk-lap> normally that is how it works, but your getting multible connections
[20:51] <sms_> Wait so
[20:51] <sms_> My router is in bridged mode?
[20:52] <patdk-lap> nope
[20:52] <patdk-lap> or, I don't believe so
[20:53] <patdk-lap> try this on your machines
[20:53] <patdk-lap> route add -host 91.155.133.247 dev eth0 (on the server)
[20:53] <patdk-lap> and, route add -host 88.115.187.159 dev eth0 (on the client)
[20:54] <patdk-lap> it's been awhile since I have done something like this
[20:54] <sms_> well the client is win7
[20:54] <patdk-lap> hmm
[20:54] <KrayZ> hi all, howto mount the floppy drive on the ubuntu server livecd? I need to load some raid drivers in order to see my raid.. (rocketraid 620)
[21:06] <sms_> hmm, I can't configure my router without changing my ip to 10.0.0.2, subnet mask 255.255.255.0 and gateway 10.0.0.3
[21:07] <toddnine> Hi guys.  Does anyone have any experience with either Ubuntu Cloud or a server with virtualization?  I'm looking to replace our old vmware server (still running on cent os 5.0) with an updated ubuntu version.  We're also growing on our internal system needs, so we need the ability to expand as well.
[21:10] <toddnine> I also need an interface that's relatively easy to use like the vmware server 2.0 web ui.  Any suggestions in the Ubuntu sphere of products since this is our preferred OS now
[21:11] <koolhead17> toddnine: better you speak to canonical in case you need support :D
[21:12] <jamespage> RoAkSoAx: this is the preseed I'm using - http://paste.ubuntu.com/685589/
[21:12] <toddnine> koolhead17: We're not a big shop.  The hosts are all used for development work, so we should be ok for support
[21:12] <jamespage> and yes I am working from todays images - although I have had to hard power off/on a couple of time
[21:12] <RoAkSoAx> jamespage: i get a weird error about not finding the root filesystem, but I don't get the partitioner at all
[21:13] <jamespage> RoAkSoAx: so did you do the 2 partition setup on your sd card?
[21:14] <koolhead17> RoAkSoAx: even am not able to get custom partitioning working via preseed
[21:14] <RoAkSoAx> jamespage: yeah, its the same setup as I had installing before
[21:14] <RoAkSoAx> jamespage: this time it complainst about not finding the root filesystem so might be the installer
[21:14] <jamespage> so is this ARM or x86?
[21:14] <jamespage> sorry - its a bit late so brain a little fuddled
[21:15] <RoAkSoAx> jamespage: ARM
[21:15] <jamespage> right-oh
[21:15] <koolhead17> toddnine: i am not aware who all works for canonical here and who can help you with it. :(
[21:15] <toddnine> koolhead17: Do they have a dedicated channel?
[21:15] <RoAkSoAx> koolhead17: yeah you cant, but the thing here is that I'm using a rpeseed that is supposed to wipe out the root partition and use it
[21:15] <RoAkSoAx> and it used to do it
[21:15] <RoAkSoAx> not anymore
[21:16] <koolhead17> :(
[21:16] <koolhead17> pressed is crazy :D
[21:16] <koolhead17> toddnine: where are you based in
[21:17] <toddnine> New Zealand
[21:17] <toddnine> www.spidertracks.com
[21:19] <koolhead17> toddnine: check pm
[21:20] <sms_> patdk-lap, my router is in bridged mode..
[21:20] <GrueMaster> jamespage: Here's my preseed. http://paste.ubuntu.com/685594/  So far it works quite well, although recently I have been getting prompted for keyboard info prior to dchp.
[21:21] <koolhead17> jamespage: but this should work
[21:21] <koolhead17> its auto partitining
[21:21] <koolhead17> i suppose
[21:21] <koolhead17> ?
[21:21] <koolhead17> partitioning
[21:22] <sms_> lan, nat, natp, dhcp I'm just so lost with the relation of these :|
[21:23] <jamespage> GrueMaster: well I did not get any prompts
[21:24] <GrueMaster> And my pxelinux.cfg  http://paste.ubuntu.com/685598/
[21:24] <GrueMaster> Might be a difference in kernel boot params.
[21:24] <RoAkSoAx> jamespage: yeah what are your kernel boot params
[21:26] <jamespage> RoAkSoAx: console=ttyO2,115200n8 earlyprintk=ttyO2 locale=en_GB text priority=critical fixrtc omapfb.vram=0:24M vram=48M
[21:27] <koolhead17> RoAkSoAx: i don`t provide kernel parameter like that. i simply import the ISO and then when am creating profile i provide it with path
[21:28] <koolhead17> and yes if you are using eth1 as default on dhcp, you have to provide it in kernel options or your profile or system
[21:28]  * koolhead17 spent 7 days kicking himself with preseed
[21:28] <koolhead17> Daviey: hey
[21:28] <jamespage> so I set those kernel params on the profile - but I should prob have done them on the system as they are for serial console access (i,e, specifi)
[21:28] <RoAkSoAx> jamespage: have the same plus root=/dev/mmcetcetc
[21:28] <GrueMaster> Guess I can delete the mem= lines.  They are only needed for accelerated encoding.
[21:28] <jamespage> I dropped root=
[21:29] <jamespage> as I'm not installing to the SD card but to a USB connected SATA drive
[21:29] <jamespage> hence the /dev/sda in my preseed
[21:29] <RoAkSoAx> jamespage: right, I see the failure with both, adding root= or removing it
[21:29] <RoAkSoAx> jamespage: which I didn't see before
[21:29] <jamespage> hrm
[21:29] <RoAkSoAx> jamespage: so if you have the chance to give it a try with a SD card
[21:29] <RoAkSoAx> let me know your results
[21:30] <jamespage> I will do
[21:30] <jamespage> GrueMaster: I just raised bug 845158
[21:30] <jamespage> keep bumping into it
[21:30] <GrueMaster> I don't think netinstall will install properly to SD due to the partitioning.
[21:31]  * GrueMaster looks
[21:31] <jamespage> have you seen anything similar in the testing you have been doing?
[21:31] <GrueMaster> No, but I am only installing the jenkins slave on my systems.
[21:33] <GrueMaster> Which java are you using?  I have seen issues with openjdk-7-jre-headless.
[21:33] <jamespage> yeah - I was more poking at any other java based stuff you had been trying?
[21:33] <jamespage> thats running on openjdk-6
[21:35] <jamespage> -jre-headless
[21:36] <GrueMaster> According to https://wiki.ubuntu.com/ARM/QA/Server, Phoronix-test-suite was run against openjdk-6 (if I read this correctly).  It should have found something if there was an issue.
[21:36] <GrueMaster> Although it was a slightly older version.
[22:14] <sms_> well I made progress
[22:17] <sms_> Now I have internal IP adresses
[22:17] <sms_> What answers my external ip now?
[22:20] <sms_> aaaa that's where the "virtual server" thingy in my router comes along