Network Weather Forecasting

Domain:

The analysis of data from High Energy Physics is increasingly dependant on high speed networks to share the data across world-wide collaborations. High speed networks often suffer with sudden drops in performance due to various reasons. The current approach to network measurement and diagnosis is generally reactive by manually viewing graphs and tables of analyzed measures and results. We assume that an automated approach to forecast and event detection can serve network engineers and administrators better to identify network problems.

Problem:

To forecast values of network performance into future with reasonable confidence so that preventive measures can be adapted before hand.

Related work:

Significant work is being done for two some what related projects i.e., Network anomaly detection and Network event isolation. In Network anomaly detection the approach is to find out unwanted events in networks through rigorous analysis of data collected using different measuring tools. Event diagnosis is an extension to Network anomaly detection and goal is to classify unwanted events on the basis of their cause, effect and other differences.

Network weather forecasting is similar to above two projects in a sense that it applies similar rigorous statistical analysis and can be used for anomaly detection but it is different on a ground that it is proactive and both other approaches are reactive so in a way it can prove more efficient than other.

Previous work:

Some work has already been carried out for network weather forecasting. That work includes forecasting of active and passive data and its comparison. Main algorithm tested for this purpose was Holt-Winters. Although the results are encouraging, yet we believe some more sophisticated scheme is required to improve the results.    

Our Goals

1-      We have to get a set of data that can be used as basic data set for testing.

2-      We have to apply regularization where needed.

3-      Implementation of main stream ARMA/ARIMA forecasting algorithm

4-      Apply algorithm on previous dataset

5-      Apply algorithm on new dataset (if any)

6-      Results analysis

 

Task division and dependency

Following table shows a rough distribution of tasks identified in this project, their dependence on other tasks.

 

 

PX = PhaseX

TX = TaskX

Network Weather Forecasting

Phase

Task

Description

Dependence

P1

T1

Previous work study

Nil

P1

T2

Study of ARMA/ARIMA

Nil

P1

T3

Getting previously tested data set/results

Nil

P2

T4

Design of software

T2

P2

T5

Implementation

T4

P3

T6

Regularization

T3,T5

P4

T7

Apply on previously tested data set

T3,T5,T6

P4

T8

Analysis/Comparison with previous results

T7, T3

P5

T9

Apply on any new dataset

T5,T6

P5

T10

Analysis

T9

P6

T11

Report Results

T8,T10