[PLUG] Solaris syslog behavior when overloaded

Aaron Burt aaron at bavariati.org
Tue Jul 13 18:48:56 UTC 2010


On Tue, Jul 13, 2010 at 08:13:19AM -0700, Michael wrote:
> 
> We're looking into a problem with VOIP phone freezing and have several
> hundred phones doing debug level logging to a Solaris box.  Analysis
> shows between 1000 and 1500 messages a second being logged.

Cool problem!  I used to have fun working on VoIP equipment.

> However after a few hours we see a backwards drift in timestamps - as if
> the syslog server is falling behind.   I also see that syslog is
> resolving IP addresses and writing FQDN to the log.

So timestamps in logged messages are falling behind current-time?

Do you suppose the UDP receive queue is filling up with messages before
they hit syslogd, or syslogd is backing up behind the disk I/O?

At your message rate, you're receiving a lot of bits.

> The man pages only reference to buffering is that if syslog is sent a HUP
> it will attempt to flush all pending messages.   There is no reference
> about controlling the DNS lookup.

Ah, the joy of Solaris manpages.  You might find other settings in
/etc/default/syslogd or /etc/netconfig or /etc/net/transport/ but I think
Solaris syslogd behaviour is pretty much set in stone.

Have you considered using syslog-ng and/or a more powerful (Linux- or
OpenSolaris-based) log-server?

I <3 CSW,
  Aaron



More information about the PLUG mailing list