| rick_h_ | jrwren: hah cool | 00:54 |
|---|---|---|
| rick_h_ | jrwren: congrats | 00:54 |
| cmaloney | Good morning | 13:43 |
| jrwren | Good Morning. | 13:43 |
| daemoneye | moin cmaloney and jrwren | 13:46 |
| cmaloney | Are we having fun yet? | 13:50 |
| daemoneye | I am. Dunno about anyone else. | 13:52 |
| * daemoneye is happily digging into the code of ceph | 13:52 | |
| cmaloney | any particular reason? | 13:53 |
| daemoneye | Yeah. $job is paying me to do so... figuring out how ceph deals with permissions. | 13:54 |
| waldo323 | hopefully better morning to everyone else :) oi, digging through log files and zenoss events | 14:06 |
| waldo323 | trying to figure out why they NFS server is periodically dreaming of being an airplane or trying to rev its motorcycle alter ego | 14:08 |
| daemoneye | waldo323: I suspect it is just tired of being a NFS server | 14:09 |
| waldo323 | perhaps it needs an intervention | 14:09 |
| daemoneye | I think it might. | 14:10 |
| daemoneye | Anyone in the A2 area want to grab lunch around 11:30? | 14:10 |
| cmaloney | THanks for the offer, but not in A2. :) | 14:11 |
| daemoneye | Ahh okay. :-) | 14:11 |
| waldo323 | I'm not seeing any errors, and actually thought the disk group was connected with a different machine and saw some other messages which didn't look good on the wrong machine | 14:12 |
| daemoneye | the machines are going rouge! RUN AWAY! | 14:12 |
| waldo323 | "kernel: hpilo" ... "Open could not dequeue a packet" | 14:12 |
| daemoneye | :O | 14:13 |
| waldo323 | google hasn't been helpful so far | 14:13 |
| waldo323 | i have only found others with that in their logs from years ago but no solutions yet | 14:14 |
| * daemoneye nods | 14:14 | |
| waldo323 | ...oh someone is saying those are informational and can be ignored | 14:14 |
| daemoneye | ohh | 14:15 |
| jrwren | daemoneye: that sounds FUN! But... I didn't think ceph did anything with permissions. :) | 14:18 |
| daemoneye | jrwren: It does! You can specify that only one UID can work with a given key. So if I state that UID 1001 can read/write to the ceph server, I cannot let any other UID write there, including root. | 14:22 |
| jrwren | daemoneye: they aren't unix uids are they? they are ceph uids? are we talking ceph rbd, object, or FS? | 14:25 |
| daemoneye | They are, from what I can tell, unix uids | 14:26 |
| jrwren | Is there an idmap? | 14:26 |
| cmaloney | are they internally unix uids or what jrwren said? :) | 14:27 |
| daemoneye | I'm not sure. This is what I am going off of. http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-February/016173.html | 14:27 |
| jrwren | ah, ok. I found some answers to my question in the docs: "A Ceph Storage Cluster user is not the same as a Ceph Object Storage user or a Ceph Filesystem user. The Ceph Object Gateway uses a Ceph Storage Cluster user to communicate between the gateway daemon and the storage cluster, but the gateway has its own user management functionality for end users. The Ceph Filesystem uses POSIX semantics. The | 14:28 |
| jrwren | user space associated with the Ceph Filesystem is not the same as a Ceph Storage Cluster user." | 14:28 |
| cmaloney | So there is a mapping | 14:28 |
| daemoneye | Yep | 14:28 |
| cmaloney | per storage cluster | 14:29 |
| jrwren | its CephFS only. | 14:29 |
| jrwren | Makes sense. | 14:29 |
| jrwren | how big is your ceph cluster? | 14:29 |
| daemoneye | We have about 15PB. | 14:29 |
| cmaloney | curious how the storage cluster users map across different machines | 14:29 |
| cmaloney | or is it one of those "you are UID 1001 across each of these machines that you're mapped to" | 14:30 |
| daemoneye | cmaloney: there is a metadata server that manages that for you. | 14:30 |
| cmaloney | Ah, OK | 14:30 |
| cmaloney | so a special driver on the mounting machine then? | 14:30 |
| daemoneye | Yeah. There is some code that wraps around fuse. | 14:31 |
| daemoneye | There are three types of servers for ceph. One for the storage of data, one for metadata, and one for monitoring the system. | 14:32 |
| jrwren | daemoneye: 15PB is a lot. What kind of network is it on? | 14:36 |
| jrwren | cmaloney: MDS not necessarily on the mounting machine, other than the in-kernel cephfs driver, AFAIK | 14:37 |
| jrwren | Ceph is awesome, but the network gear to run it reasonably is expensive AFAICT | 14:37 |
| daemoneye | jrwren: It is split between UMich, Wayne State, and MSU. | 14:39 |
| daemoneye | We have a 100GB link, a couple 40GB links, and a couple 80GB links | 14:40 |
| brousch | You can't keep UM and MSU data in the same cluster. It will destroy itself. | 14:40 |
| jrwren | daemoneye: ugh, WAN ceph? isn't that not recommended? | 14:40 |
| jrwren | daemoneye: I meant more like: what is the local net like for local cluster. | 14:40 |
| daemoneye | Dunno if it's not recommended, but I could see why it wouldn't be. | 14:41 |
| jrwren | daemoneye: I guess I mean each server with ceph storage nodes in them. What are their connectivity. | 14:41 |
| * daemoneye *cough*MSU*cough* | 14:41 | |
| jrwren | yeah, the 100GB, 40GB and 2-80GB aren't dedicated for the ceph, are they? | 14:41 |
| daemoneye | jrwren: Ohh, I believe they are 100GB on the LAN | 14:42 |
| jrwren | WOW! | 14:42 |
| jrwren | that is awesome. | 14:42 |
| daemoneye | yeah | 14:43 |
| daemoneye | It's fun seeing 40TB go across a 100GB link | 14:43 |
| jrwren | not even AWS uses that. | 14:43 |
| jrwren | oh, i assume we were just writing B when we meant b for all of those network speeds. | 14:43 |
| daemoneye | Yep. Well, UMich lawyers like things fast and stable. And we don't want to piss them off. | 14:43 |
| daemoneye | and yes | 14:44 |
| cmaloney | That's still pretty damn fast | 14:44 |
| daemoneye | I love how the drivers for a portable DVD writer comes on a CD... | 14:45 |
| cmaloney | So this is one humongous Ceph cluster between all of the schools? What sorts of data are on this thing? | 14:45 |
| daemoneye | That it is. And it is being designed for physics computation that various companies want to do. | 14:45 |
| daemoneye | It is a shame I am only here for another six months or so... Got a 1 year term limited gig working on it, so I am already looking for new work. | 14:47 |
| jrwren | sounds pretty darned awesome. | 14:47 |
| daemoneye | yeah, it's fun | 14:48 |
| jrwren | with that much ceph experience, sounds like you could just go to redhat and say, "give me a job, K?" | 14:48 |
| daemoneye | lol | 14:48 |
| daemoneye | I would, but the code is awful IMO. | 14:48 |
| cmaloney | heh | 14:49 |
| daemoneye | Documentation is lacking, for about three months I was finding a bug a week, and the comments within the code were marginal at best. | 14:50 |
| daemoneye | And this is all on the stable release, not the repo head. | 14:50 |
| daemoneye | If I had the time and energy to do it, I'd probably spend my time cleaning it all up. | 14:52 |
| daemoneye | :o | 14:53 |
| daemoneye | welcome back cmaloney | 14:54 |
| cmaloney | tx | 14:54 |
| jrwren | its amazing RH doesn't invest in cleaning it up, isn't it? | 14:54 |
| daemoneye | jrwren: yeah. | 14:54 |
| cmaloney | Since RH bought it they may not be aware of the code quality. :) | 14:54 |
| daemoneye | That's possible | 14:54 |
| jrwren | um, they are aware. | 14:55 |
| jrwren | heck, RH has a track record of buying things with exceptionally poor code quality. | 14:55 |
| daemoneye | And the line count for some of the files makes some folks I know gag. | 14:56 |
| jrwren | jboss, and that other FS. | 14:56 |
| jrwren | well, that is just personal prerence and they need to get over that. | 14:56 |
| jrwren | style is not necessarily a code quality issue. | 14:56 |
| daemoneye | Yep. One file is almost 12KLOC | 14:56 |
| jrwren | no big deal. | 14:57 |
| daemoneye | Only issue I have with the code are comments like this: "/* -- FIXME -- */" | 14:57 |
| jrwren | now, if that is 1 function OTOH :) | 14:57 |
| daemoneye | jrwren: naw, it's about 6 classes and a couple dozen functions. | 14:58 |
| daemoneye | And I can see the reason behind multiple pairs of public and private sections of a class. | 14:59 |
| jrwren | a couple dozen functions? so... 500line function average? ok, that is bad. | 15:00 |
| daemoneye | Yep | 15:00 |
| daemoneye | I know it can be improved greatly. | 15:00 |
| daemoneye | And that is coming from someone who hasn't touched C++ in five or six years | 15:01 |
| jrwren | were you looking specifically at cephfs code or other areas? | 15:03 |
| jrwren | rbd seems to be their core. i'm not surprised cephfs is secondary. | 15:03 |
| daemoneye | I was looking at the ceph fuse client code | 15:03 |
| jrwren | oh, is that even core project code? | 15:06 |
| daemoneye | I believe so, as it is the common way to mount the fs on a client machine. | 15:06 |
| daemoneye | Or at least one of the common ways | 15:06 |
| jrwren | it is? I thought there was a kernel module | 15:07 |
| daemoneye | There is if you want to do a kernel mount. There is also a userland mount version | 15:07 |
| jrwren | I guess both would be important depending on the client. | 15:08 |
| daemoneye | Mostly for those poor souls who don't get the joys of having sudo rights. | 15:08 |
| cmaloney | Looks like the code is a combination of C++ and Python? | 15:08 |
| daemoneye | Yep. | 15:08 |
| daemoneye | And I have found minor hints of java as well placed here and there | 15:08 |
| cmaloney | I did not expect to see PYthon in there. :) | 15:08 |
| daemoneye | Ahh. Python is there as a way for folks to manage the cluster. | 15:09 |
| cmaloney | right | 15:09 |
| cmaloney | Man, I do not know C++ at all | 15:10 |
| jrwren | cmaloney: no one does, not even bjarn :p | 15:10 |
| daemoneye | I am the same way | 15:10 |
| daemoneye | lol jrwren | 15:10 |
| cmaloney | Of course I'm just picking random files to see what's going on so it's all line noise | 15:11 |
| cmaloney | I took a look at Firefox just to see if I could read the code and see what it does. Blender as well. I wept. | 15:11 |
| jrwren | cmaloney: well, like daemoneye said, that is *bad* C++ code. | 15:11 |
| jrwren | cmaloney: there is good c++ code out there from which you can learn. | 15:12 |
| daemoneye | cmaloney: this is probably some of the better code. https://github.com/ceph/ceph/blob/master/src/ceph_fuse.cc | 15:12 |
| cmaloney | It looks like it's mostly warning messages about known kernel issues. :) | 15:13 |
| * cmaloney packs up the computers and becomes a farmer | 15:14 | |
| daemoneye | aww. | 15:14 |
| * daemoneye offers cmaloney a pint | 15:14 | |
| cmaloney | heh | 15:14 |
| daemoneye | It is the only thing that gets me through sometimes | 15:14 |
| daemoneye | That and the occasional nice cigar or pipe | 15:14 |
| cmaloney | I think the major problen is my eyes don't like C++ syntax at all | 15:14 |
| cmaloney | it's too busy | 15:15 |
| daemoneye | Yeah. I can agree with that | 15:15 |
| cmaloney | I think Python has spoiled me a little too much | 15:17 |
| cmaloney | I mean, I can still read C code, but that's pretty much a HLA anyway. ;) | 15:18 |
| daemoneye | lol | 15:18 |
| cmaloney | But C++ objects never clicked with me | 15:18 |
| daemoneye | Hmm... how much do I want to punish myself tonight? Have to do a reinstall of my OS. Do I stay with Debian? Or do I try to install slackware or Gentoo on the machine? | 15:19 |
| cmaloney | heh | 15:19 |
| cmaloney | I mean, C++ always starts with "OK, we're going to do 'Hello world'". And it starts easy enough. | 15:19 |
| cmaloney | Then they add objects and it's like someone kicked over a bunch of alphabet blocks into the page | 15:20 |
| daemoneye | lol, that's actually quite true | 15:20 |
| daemoneye | And probably went with "fsck this noise!" | 15:20 |
| daemoneye | Well, time to go forge for food. later everyone! | 15:21 |
| cmaloney | have fun! | 15:22 |
| daemoneye | will do! | 15:22 |
| jrwren | C++ objects should be simple if you think of them like a C struct with a vtable. | 15:22 |
| jrwren | its the C++ metaprogramming that makes my head hurt. | 15:22 |
| cmaloney | I think the other problem with C++ was I was learning it at the same time as Java | 16:22 |
| cmaloney | so I've kinda conflated the two as a bundle of hot mess | 16:23 |
| jrwren | oh yeah, that would be TOUGH | 16:35 |
| cmaloney | It wasn't until Python that I really understood OO | 16:36 |
| cmaloney | just in ime for everyone to start on again about functional programming | 16:37 |
| cmaloney | ;) | 16:37 |
| jrwren | lol, well not the Go people. | 16:41 |
| cmaloney | Is Go not particularly functional? | 16:43 |
| jrwren | its not. | 16:47 |
| jrwren | in fact, it is anti functional. | 16:47 |
| jrwren | "use a for loop" is the go way instead of passing a function to map. | 16:47 |
| jrwren | "use a for loop with an if statement" is the go way instead of passing a function to filter. | 16:47 |
| cmaloney | Interesting | 16:49 |
| Scary_Guy | wow, busy morning | 16:53 |
| jrwren | someone with a real world ceph implementation, not the crap I used in teh past :p | 16:54 |
| daemoneye | Yep. We are social today | 16:54 |
| Scary_Guy | apparently, also that you on the bridge? | 16:55 |
| daemoneye | Yeah, that was me. I'll get back on | 16:55 |
| Scary_Guy | I'll call in once I find my phone | 16:55 |
| daemoneye | cool | 16:55 |
| Scary_Guy | 248-724-2600 for anyone else that wants to join us on the #mi2600 (ofdoom.org) partyline | 16:56 |
| jrwren | is there a discord? | 16:58 |
| daemoneye | not that I know of | 16:58 |
| Scary_Guy | old school PBX as far as I know | 17:00 |
| cmaloney | heh | 17:01 |
| jrwren | classic | 17:02 |
| jrwren | maybe there is an discord asterisk bridge. | 17:03 |
| Scary_Guy | https://github.com/42wim/matterbridge | 17:07 |
| Scary_Guy | so, no but maybe you can bridge something else to it? | 17:08 |
| cmaloney | Nice. Apparently good ol' Amazon Music isn't allowing folks to upload their music anymore | 17:10 |
| cmaloney | probably because of licensing nonsense | 17:11 |
| Scary_Guy | Amazon what? ;_ | 17:11 |
| Scary_Guy | ;) | 17:11 |
| cmaloney | Apparently I have uploaded music to there in the past but I'll be damned if I know what that is. | 17:11 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!