Well I was thinking "no" options for time-outs and buffers - in particular "tcp_finwait2" sounds appropriate - I have 1200 set, although no explanation in the manual as to what that means
Check your adapter settings for queue and pool sizes (to the max allowed) and for the interface use "tcp_nodelay=1"- those settings must be done before configuring any Ethernet Aggregation or EtherChannel.
Check the interface MTU is correct and matches on both ends to avoid windowing delay problems as well as fragmentation.
From “smit tuning”, the following command can be built from the Tuning Network Option Parameters menu, to set options for next boot – you may find some of these are set already as a default, this is what I run with these days. In particular I've tried various settings for tcp_timewait, tcp_keepidle and tcp__keepinit when faced with mail download hangs and cut offs over wide area connections.
no -r -o fasttimo='50' -o nbc_limit='65536' \
-o nbc_pseg_limit='20480' -o tcp_timewait='1' \
-o clean_partial_conns='1' -o rfc1323='1' \
-o tcp_keepidle='1200' -o tcp_keepinit='60' \
-o tcp_keepintvl='20' \
-o bcastping='0' -o directed_broadcast='0' \
-o ipignoreredirects='1' -o ipqmaxlen='150' \
-o ipsendredirects='0' -o ipsrcrouteforward='0' \
-o ipsrcrouterecv='0' -o ipsrcroutesend='0' \
-o tcp_pmtu_discover='0' -o udp_pmtu_discover='0'
In addition, for best tuning with TCP/IP over a fibre fabric, the following options apply:
no -r -o -o tcp_recvspace='393216 \
-o tcp_sendspace='393216 -o sb_max='1310720' \
-o udp_recvspace='655360' -o udp_sendspace='65536'
Whereas for best performance over a basic Ethernet network, in preference to a disk fabric, the following options would instead apply:
no -r -o -o tcp_recvspace='65536' \
-o tcp_sendspace='65536' -o sb_max='262144' \
-o udp_recvspace='65536' -o udp_sendspace='32768'
For web servers, the following lines would also be needed:
no -r -o -o sack='1' -o delayack='3' delayackports='{80}'
With AIX AIX 5.3 ML6, there was a fault with the default value for “ipfragttl” – the manuals say it should be “60”, but “2” is the apparently wrong default.
no -r -o -o ipfragttl='60' # To fix the fault.