Tuning TCP Performance on nvOS

System administrators can tune nvOS to reduce TCP packet latency.

Standard nvOS settings favor throughput and many connections. If latency is more important to your enterprise, consider tuning the settings below.

Always validate the results of tuning empirically. Before tuning, determine the empirical tests and evaluation criteria you will use to validate the results.

These values are a starting point for latency tuning. Empirical testing with reference to the specific criteria of your enterprise can often improve performance even further.

For more information see performance tuning guides for Solaris and OpenSolaris.

Prerequisites

Ensure that the appropriate version of nvOS is installed before tuning.

Procedure

  1. Tune the system parameters.
    Add these settings to the configuration file /etc/system: 
    set ip:ip_squeue_fanout = 1
set ip:tcp_squeue_wput = 1
set ip:ip_squeue_wait = 0
set ddi_msix_alloc_limit = 8

    (These settings take effect after hardware restart.)

  2. Tune the TCP device parameters.
    Use the ndd command. 
    ndd -set /dev/tcp tcp_max_buf 16777216
ndd -set /dev/tcp tcp_cwnd_max 8388608
ndd -set /dev/tcp tcp_xmit_hiwat 4194304 
ndd -set /dev/tcp tcp_recv_hiwat 4194304
ndd -set /dev/tcp tcp_conn_req_max_q 512
ndd -set /dev/tcp tcp_conn_req_max_q0 4096
  3. Restart the hardware.
  4. Test the results, and iterate if needed.