Possible Anomaly Detection Algorithms
Some possibilities include:
- Profiling short sequences of system calls (pH, Sana Security's Primary-Response)
- System call flow graphs (PAID)
- Semanitc constraint violations at the system call level (Systrace)
- Various combinations of supervised and unsupervised learning algorithms on network or application input data (i.e. Stefano Zanero's work)
- "Loose" signatures (most current IPS products)
Each method has its strengths and weaknesses, but these methods are new enough to most people that the positive and negative attributes are generally mis-understood at best.