-
Effective Tests for Spark Streaming
At OVO we are heavily relying on Spark in both it’s batch and stream processing capability to successfully master the challenge of smart meter and real time energy usage data and provide increasing insights for our customers and business.
Streaming solutions come with their own challenges, the biggest of them being able to continuously operate them 24/7 and still maintaining the ability continuously update them. While we can (and will) reason about message delivery patterns, failure handling, state recovery and more sophisticated aspects of building a resilient stream based solution, let’s start with the most important one: How can we structure our Spark application in a way, so it’s easily testable?
subscribe via RSS