10GbE on Sun SPARC Enterprise T5120/T5220

The Sun SPARC Enterprise T5120/T5220 servers have two ways to provide 10 GbE ports.
One way is to plug in a dual--port PCI-e Network Interface Unit (NIC) X1027A-z (right).
Another way is to use the on-chip dual 10GbE Network Interface Unit (NIU). You connect to the on-chip by using a PCI slot to hold the XAUI (X (roman 10) Attachement Unit Interface), which is a standard for connecting 10GbE devices.
There are important tuning parameter for both the NUI and PCI-e 10 GbE options. For example, the tunable parameter for the number of kernel threads that offload processing of received packets from interrupt CPU should be set in /etc/system:

set ip:ip_soft_rings_cnt=16

What is the difference in performance? Here's what "pure see" says:

NIU wins in all micro-benchmarks except one - single 10GbE port transmitting UDP small packets.

Sun Multi-threaded 10GbE PCI-e NIC can transmit an impressive 2.1 million 64 byte UDP packets per second, 50% more than NIU (1.4 million pps) out of one port, due to the fact that it has 50% more transmit DMA channels than NIU (12 vs. 8 per port).

So if your workload is mostly sending small UDP packets, Sun Multi-threaded 10GbE PCI-e NIC may deliver higher packet rate than NIU on UltraSPARC T2 systems.

In all other scenarios, NIU gives higher throughput or lower CPU utilization at similar throughput. On 2 10GbE port throughput test, 2 NIU can achieve an impressive 14.6 Gbits/s on TCP transmit, or 18.2 Gbits/s on TCP receive using 8Kbytes messages and 145 connections.

For TCP transmit, the CPU efficiency (measured by Gbps/GHz) of NIU is 23% higher than PCI-e NIC at maximum throughput for 2 ports, and 46% higher at maximum throughput for 1 port.

(read more)