Introduction to Lambda Architecture
With the rapid growth of communication, the handling of data has become a problem. The batch processing was introduced as a solution where a batch of data is collected and processed at once. The problem with this approach was the outdated result. To overcome this issue the Lambda Architecture was introduced which is a combination of batch processing and stream processing. In stream processing the real time results.
Lets see what is the purpose of each layer in Lambda architecture.
Batch layer
This layer is responsible for storing all the incoming events and create batch views. The generation of the results in this layer takes a considerable time as it processes a large set of historical data.
Speed layer
This layer will catch up the data which are missed from the batch layer, at the time the user queries for a output. This will process a small set of data and gives a real time output.
Service layer
This layer will aggregate the both results from batch layer and speed layer to give a final result.
Using WSO2 CEP and BAM for Lambda Architecture
Lets discuss how to use the WSO2 CEP and BAM to produce a real time result using Lambda Architecture.