Node-Bell

Node-Bell is a real-time anomalies detection system for periodic time series, built to be able to monitor thousands of metrics. It collects metrics from clients like statsd, analyzes them with the 3-sigma rule and visualizes results on the web. Once enough anomalies were found in a short time, it alerts you via alerters like hipchat.

Latest version: v0.4.3

We(Eleme) have blogged how we created it.

Requirements

• node.js 0.11+ (bell is written in nodejs)
• ssdb 1.6.8.8+ (disk based datastore)
• beanstalkd (job queue between listeners and analyzes)

Installation

$ npm install node-bell -g

to add node-bell to statsd's backends, edit statsd's config.js:

{
, backends: ["node-bell/clients/statsd"]
}

Getting Start

1. Start ssdb & beanstalkd & clients.

2. Generate sample config and edit it, default: config/configs.toml:

   $ bell -s
   $ mv sample.configs.toml configs.toml
   $ vi configs.toml

3. Start services: listener, analyzers, webapp, alerter, cleaner

   $ bell analyzer -c configs.toml
   $ bell listener -c configs.toml
   $ bell webapp -c configs.toml
   $ bell alerter -c configs.toml
   $ bell cleaner -c configs.toml

Tutorial

• 中文: tutorial-zh.md

Services

1. listener

   Receive incoming metrics from clients over tcp, then put them to job queue (beanstalkd), default port: 8889.

2. analyzer(s)

   Get jobs from queue, analyze if current datapoint is an anomaly or not via 3-sigma rule. The results and all metrics are stored in ssdb. We can start multiple analyzer processes, see analyzer-cluster.

3. webapp

   Visualize analyzation results and metrics on web, default port: 8989.

4. alerter

   Alert once enough anomalies were detected. It receives anomalous datapoints from all analyzers over tcp, default port: 8789

5. cleaner

   Check the last time of a metric hitting node-bell every certain time interval, if the age exceeds the threshold, clean it.

Questions

Search faq.md or open an issue.

• Write a client
• Analyzers Cluster
• Custom Alerters
• Listener Net Protocol
• Week Analyzation Ability

Configs

Configuration Manual: conf.md.

Inside

1. algorithm

   Node-bell use the 3-sigma rule(similar to z-score) to detect if a datapoint is an anomaly:

   States that nearly all values(99.7%) lie within 3 standard deviations of the mean in a normal distribution.

2. storage schema

   Metrics are stored in ssdb, using zset, and the schema is:

   key       |  score
   ------------------------------------------------
   timestamp | value:anomalous multiples:timestamp
   

3. data flow

   [clients]->[listener]->[beanstalkd]
                              |
                              v
               --------> [analyzers] ------> [alerter]
               |              |
       history |         save v    visualize
               ------------ [ssdb] --------> [webapp]
   

Changes

See changes.md.

License

MIT Copyright (c) 2014 Eleme, Inc.