STREAM

Description

Want to process all credit card transactions in Europe in real time? No luck, today’s computers cannot do it. Unless, of course, you use technology from STREAM project.
A growing number of applications require the ability to analyze massive amounts of streaming data in real time. A few examples: stock market data processing, anti-spam and anti-virus filters for e-mail, network security systems for incoming IP traffic in organisation-wide networks, automatic trading, fraud detection for cellular telephony to analyze and correlate phone calls, fraud detection for credit cards, and e-services for verifying the respect of service level agreements.

Typically, such applications require strong analysis and processing capabilities, i.e., data mining, to discover facts of interest.

Data analysis happens today on clusters of workstations using specialized middleware and applications.
Although solutions for real-time processing of information flows already exist, current platforms and infrastructures face three main limitations:

(a) scalability: the number of processing nodes is upwards limited

(b) autonomy: support from IT experts is needed to keep the system working over the time

(c) performance: some problems cannot be managed, because the raw amount of data to be processed is too large.
STREAM aims at scaling system size by an order of magnitude, to 100s of nodes, achieving real-time processing of information flows, and providing unsupervised and autonomous operation. This will allow for much broader deployment of such products and services to new areas that need to manipulate large information flows in a cost-effective manner, and in particular, the Telecom, Financial, and E-services sectors.