-- CarstenAulbert - 31 Dec 2008

After adding extra nodes in December 2008 we discovered that the SUN servers sent out a continuous stream of ARP requests into the network and some nodes were not able to access the servers at all. When investigating this it turned out that there seemed to be a magic barrier in the Solaris ARP cache size since all hosts showed only 1480-1518 known entries in their ARP cache ($arp -an|wc -l$).

With a little bit of searching on the web we found only very few hints but no real solution thus the case was opened.

Update: While waiting for the call back from Sun, we found that setting one arp timeout value too a much larger value also increased the possibility of having more entries in the arp table (not sure if before the entries were pushed out faster then new ones were acquired):

ndd -set /dev/arp arp_cleanup_interval 2000000

Update2: In this PDF document (Solaris IP Duplicate Address Detection by James Carlson) we found some hints that the SunOS kernel starts pushing out probes to the network to validate its arp entries. The amount of traffic seems to grow linear with the number of entries in the arp table and thus could also be responsible for the effect.

Still missing: Detailed explanation for these parameters:

arp_cleanup_interval
arp_publish_interval
arp_publish_count
arp_probe_delay
arp_probe_interval
arp_probe_count
arp_fastprobe_delay
arp_fastprobe_interval
arp_fastprobe_count
arp_defend_interval
arp_defend_rate
arp_broadcast_interval
arp_defend_period