SUMMARY: Increasing tcp_sendspace buffers under Solaris 2.x

From: Earl Zmijewski (eez@fluent.com)
Date: Tue Apr 04 1995 - 04:55:15 CDT


No need for much of a summary, since I only got 3 responses. First the
question, then the responses. Many thanks to those who wrote.

-----------------
Original Question
-----------------

Under SunOS, the tcp_sendspace and tcp_recvspace buffer sizes are both set to
a wimpy 4K. Increasing them to 16K is easy and can dramatically increase
network performance. The change is simple:

  edit /usr/kvm/sys/netinet/in_proto.c
  rebuild the kernel.

Ok, so how do you do the same thing under Solaris, which has 8K buffers?
ndd doesn't seem to have the necessary parameters and there isn't anyway to
rebuild the kernel.

For the curious, there is a good discussion of all of this in "TCP/IP
Illustrated Volume 1" by W. Richard Stevens. As an added bonus, I've also
enclosed the announced widow sizes and MSS (via tcpdump) for various OSes to
off-site machines.

I'd be interested in hearing about any experiences with modifying these
parameters and how you might go about it for anything other than SunOS.
I will summarize.

Thanks,
Earl

--

SunOS 4.1.3 win 4096 Solaris 5.3&4 win 8760 <mss 1460> (DF) IRIX 4.0.5 win 61440 <mss 512> IRIX 5.2&3 win 61440 <mss 512> IRIX 6.0.1 win 49152 <mss 512> SPP-UX V3.0.2 win 4096 <mss 512> HP-UX 9.0.5 win 8192 HP-UX 10.0.0 win 32768 <mss 1460> (DF) AIX 3.2.5 win 16384 <mss 512> OpenVMS V6.1 win 4096 <mss 1460,wscale 0,eol> ULTRIX V4.4 win 16384 OSF/1 V2&3 win 32768 <mss 536,nop,wscale 0> [tos 0x10] unicos8.0.2.1A win 32768 [tos 0x10] PC-NFS win 512 <mss 512> Windows-NT 3.5 win 8192 <mss 1460> (DF) Windows 3.11 win 61440 <mss 512>

--------- Responses ---------

From: stern@sunrise.East.Sun.COM (Hal Stern - NE Area Systems Engineer) To: eez@fluent.com

i wouldn't increase the sendspace blindly like that. when you go to 16k, you allocate 16k for every socket -- which chews up your kernel memory pretty quickly. sunos has a limited amount of kernel address space (like 6M, usually, out of a 16M total), so you go throught that much more quickly with a larger default send space. your better bet is to use setsockopt() to crank up the send space on a per-socket basis.

the ndd variables for 2.x are:

udp_xmit_hiwat udp_recv_hiwat tcp_xmit_hiwat tcp_recv_hiwat

--hal

------------- Editor's Note ------------- (6*1020*1024)/(16*1024) = 382.5.

I don't think I'm in much danger of running out of kernel memory, and I don't have access to source code to alter calls to setsockopt. Thanks for the ndd variables. They are just what I needed. ------------------------------------------------------------------------------

From: diekema@linus.si.com (Jon Diekema) To: eez@fluent.com

#!/sbin/sh # # WARNING: This script tunes the TCP/UDP high and low # water marks for Solaris 2.3 only. # This script generates messages on the console. If you # don't like them, just comment them out.

# # Tuning TCP (bytes) #

echo "TCP xmit water-marks: High = 32Kbytes, Low = 24Kbytes" /usr/sbin/ndd -set /dev/tcp tcp_xmit_hiwat 32768 /usr/sbin/ndd -set /dev/tcp tcp_xmit_lowat 24576

echo "TCP recv water-marks: High = 32Kbytes " /usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 32768

# # Tuning UDP (bytes) #

echo "UDP xmit water-marks: High = 32Kbytes, Low = 24Kbytes " /usr/sbin/ndd -set /dev/udp udp_xmit_hiwat 32768 /usr/sbin/ndd -set /dev/udp udp_xmit_lowat 24576

echo "UDP recv water-marks: High = 32Kbytes, Low = 24Kbytes " /usr/sbin/ndd -set /dev/udp udp_xmit_hiwat 32768 /usr/sbin/ndd -set /dev/udp udp_xmit_lowat 24576

-----------------------------------------------------------------------------

From: perryh@pluto.rain.com (Perry Hutchison) To: eez@fluent.com

Solaris may be different, but under generic SVR4 you could see if /etc/conf/bin/idtune knows about the relevant parameters. If not, you would edit something in /etc/conf/pack.d/*/space.c (where the * is the package name, probably something like tcp or ip), then use /etc/conf/bin/idbuild to rebuild the kernel.

-- Earl E. Zmijewski -- Unix Systems Administrator, Fluent Inc. Centerra Resource Park; 10 Cavendish Court; Lebanon, NH 03766-1442 eez@Fluent.COM http://www.Fluent.COM voice@603.643.2600 fax@603.643.3967



This archive was generated by hypermail 2.1.2 : Fri Sep 28 2001 - 23:10:21 CDT