| === psivaa-afk is now known as psivaa | ||
| === psivaa is now known as psivaa-afk | ||
| === psivaa-afk is now known as psivaa | ||
| Yaannnn | Hi | 17:03 |
|---|---|---|
| Yaannnn | I'm looking for help with high performance 1-to-1 NAT | 17:04 |
| apw | ask your question you never know we might know, or know who to point you at | 17:04 |
| Yaannnn | I have trouble NATing 10gbits with 1000 1-to-1 rules | 17:05 |
| apw | and the trouble is? throughput? | 17:05 |
| Yaannnn | Yep + packetloss | 17:06 |
| Yaannnn | the server never reaches 10gbit/s when they are many flows | 17:06 |
| apw | yeah that is somewhat beyond out experience i would say, #ubuntu-server may have done something like that | 17:06 |
| Yaannnn | Ok, thx, i'm gonna look for help there then :) | 17:07 |
| Yaannnn | The only thing I can say is that with normal iptables the CPU is used by Hardware IRQ and with Xtables it's softirq which are using the CPU | 17:08 |
| trippeh_ | Yaannnn: Do you have any stateful netfilter modules loaded? As soon any of those conntrack thingies are loaded into the kernel throughput can often plummet when many flows is involved. | 18:09 |
| === chuck__ is now known as zul | ||
| trippeh_ | 1-1 nat should be able to run statelessly, without conntrack, IIRC | 18:09 |
| trippeh_ | Yaannnn: also rules get evaluated in order, you should look at moving the most used rules to the front, maybe even grouping into chains to limit the amount of searching done per packet | 18:11 |
| Yaannnn | trippeh_: The problem is that by default iptables loads conntrack | 18:12 |
| trippeh_ | what iptables target are you using to do the natting? | 18:12 |
| Yaannnn | trippeh_: We tried the default DNAT/SNAT | 18:13 |
| trippeh_ | iptables only loads the modules needed for its targets and matches to work, conntrack is not really default unless you use a match or target requiring it | 18:13 |
| Yaannnn | trippeh_: but we also tried with rawpost from X-tables | 18:13 |
| trippeh_ | SNAT/DNAT would load conntrack indeed | 18:13 |
| trippeh_ | You might want to try out SNPT and DNPT | 18:15 |
| Yaannnn | trippeh_: is it mainstream ? | 18:15 |
| trippeh_ | Yaannnn: it should exist in 14.04 at least | 18:15 |
| trippeh_ | its pretty recent, but mainstream | 18:15 |
| Yaannnn | trippeh_: according to the man it seems to be IPV6-specific | 18:16 |
| trippeh_ | oh, hum *tries* | 18:17 |
| trippeh_ | Granted, I've only used it on ipv6 myself :) | 18:17 |
| Yaannnn | trippeh_: did you try NETMAP ? | 18:18 |
| Yaannnn | trippeh_: I tried DNETMAP but without sucess, the server was not able to reach 10gbit with lots of flows | 18:18 |
| trippeh_ | if you cant get rid of conntrack, you might want to try out kernel 3.15, it has significant performance boosts for conntrack on multicore systems | 18:19 |
| Yaannnn | trippeh_: do you know about any kernel tweaks which could help ? | 18:20 |
| Yaannnn | trippeh_: did you try out nftables ? We are trying this but it doesn't seems to behave but we didn't activate the rbtree module | 18:22 |
| trippeh_ | Yaannnn: there is a conntrack table size that can be tuned, but other than that there is not that many tunables | 18:23 |
| trippeh_ | maybe pinning ethernet ports to cpu's could help | 18:24 |
| trippeh_ | have not gotten around to nftables yet no | 18:24 |
| trippeh_ | yeah looks like SNPT/DNPT fails for ipv4 | 18:25 |
| Yaannnn | trippeh_: I was thinking about tweaks on interrupt handling | 18:25 |
| Yaannnn | trippeh_: But I don't know what is possible on this side | 18:25 |
| trippeh_ | and netmap seems to load conntrack too, hmm | 18:26 |
| trippeh_ | tweaks on interrupt handling = typically cpu pinning :) | 18:26 |
| trippeh_ | I wonder if SNAT/DNAT/NETMAP would still work with -j CT --notrack in raw table | 18:28 |
| trippeh_ | if its just address to address mapping | 18:28 |
| Yaannnn | trippeh_: nop, already tried that :P | 18:29 |
| Yaannnn | trippeh_: the traffic doesn't flow anymore | 18:29 |
| trippeh_ | bummer | 18:29 |
| trippeh_ | maybe try out 3.15 ;) | 18:29 |
| Yaannnn | trippeh_: are conntrack entries created on each packet processed for UDP packets ? | 18:29 |
| apw | there is a 3.15 final kernel in utopic as of like yesterday | 18:29 |
| apw | i believe they are, bacuase a udp pairing is a flow | 18:30 |
| trippeh_ | Yaannnn: no, it will try to match existing conntrack entry | 18:30 |
| trippeh_ | Yaannnn: switch to ipv6 :-) | 18:31 |
| trippeh_ | ahem | 18:31 |
| Yaannnn | trippeh_: Haha, I whish I could :), gonna try the 3.15 | 18:31 |
| apw | ok, so i am suggesting there is a contrack entry made for the flow | 18:31 |
| apw | which means on the first one, and those get reused | 18:32 |
| apw | bah i'll butt out | 18:32 |
| trippeh_ | :) | 18:32 |
| trippeh_ | I find it odd that there is appearantly no stateless address rewriting facility for ipv4 | 18:33 |
| Yaannnn | trippeh_: Yep, they are some but in X-tables | 18:33 |
| Yaannnn | trippeh_: And it doesn't perform well | 18:34 |
| trippeh_ | right | 18:34 |
| trippeh_ | did you make sure conntrack modules were not loaded into kernel memory when you tried it? :) | 18:34 |
| trippeh_ | say, from earlier playing around | 18:35 |
| Yaannnn | trippeh_: what is the low latency version of the kernel ? | 18:35 |
| apw | lowlatency is a slight config change, it is a preempt enabled, and irq via threads | 18:36 |
| Yaannnn | apw: ok, thx, it which situation is it supposed to help ? | 18:37 |
| Yaannnn | in * | 18:37 |
| apw | it exists for the audio folk who will burn overall performance for latency | 18:38 |
| Yaannnn | trippeh_: yep, we blacklisted the conntrack modules | 18:40 |
| Yaannnn | apw: trippeh_ : Doesn't seem to be a lot better with the 3.15, maybe more level on the rules could improve the results | 18:49 |
| Yaannnn | I'm stopping my tests for today | 18:49 |
| Yaannnn | Thanks for your help :) | 18:50 |
| ekarlso | will 3.16 be in the july update for the kernel ? | 18:52 |
| trippeh_ | hm. the NPT v6 module seems very simple. I wonder how well a straight over port would work on v4 | 18:58 |
| === retoad is now known as retoaded | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!