log analysis

[MelRay]

I have syslog-ng running on a RHEL4 box logging Cisco traffic, finally. I found a nice example on the syslog-ng mailing list and modified it accordingly. I have it set to log to file instead of MySQL. I chose a flat file to be able to grep/search the logs at any time with out the assistance of a web front or other front and it's also cross platform. We are required to save logs for one year at least, I did not want to be responsible for such a large MySQL database and all the maintenance that comes with that. :)

 

I am desperately looking for a log analysis tool that will correlate all the logs and run reports, identify patterns, you know...all the bells and whistles. I looked at OSSIM at [/url]http://www.ossim.net/home.php but it does way more than I want it to. Php-syslog-ng( [url=http://www.vermeer.org/]http://www.vermeer.org/ which was last updated 2004) requires syslog-ng to use a MySQL DB. SWATCH does not do what I require so far as I can tell, nor does octopussy (8pussy.org).

 

My goal is to have most/all of our Windows domain controllers/member servers, all the Linux systems and all the network gear log to a central server and use a tool to process that data and generate results of emerging patters, warning signs and other things. Hopefully I can accomplish this with files instead of a MySQL DB but if I have to a DB would be very acceptable.

 

Any advise on this?

[eschoeller]

I have syslog-ng running on a RHEL4 box logging Cisco traffic, finally. I found a nice example on the syslog-ng mailing list and modified it accordingly. I have it set to log to file instead of MySQL. I chose a flat file to be able to grep/search the logs at any time with out the assistance of a web front or other front and it's also cross platform. We are required to save logs for one year at least, I did not want to be responsible for such a large MySQL database and all the maintenance that comes with that. :) 

.....

 

Any advise on this?

 

I am looking for a similar tool. Something very simple that just creates a nice email summary for all hosts that are logging to a central logserver.

 

php-syslog-ng is not acceptable because it runs on php / sql etc. This analysis is occuring on a secured box where running semi insecure code (php) is not an option.

 

I have found a closed-source product called 'sawmill' that I am testing now, but it looks like it is overly complex as well.

 

logwatch does not handle multiple hosts well, but it can send you a different email for *every* host if you want. This is excessive when dealing with over a hundred syslog devices.

 

Logcheck / logsentry may be another option that I'm also looking at

[/url][url=http://sourceforge.net/projects/sentrytools/]http://sourceforge.net/projects/sentrytools/

 

 

Please let me know if you come up with anything interesting!

[MelRay]

I think I