At the recent sold-out Spark & Machine Learning Meetup in Brussels, Pierre Borckmans of Real Impact Analytics delivered a lightning talk called Writing Spark applications, the easy way.

As Pierre explained, even though Apache Spark™ offers intuitive and high-level APIs, writing production-ready Spark data pipelines involves non-trivial challenges for data scientists without expert background in software development and devops matters. In this short talk, he shows how his team tackled these issues at Real Impact Analytics, by developing an intuitive framework for writing dataflows, offering convenient data exploration and testing facilities, while hiding devops-related complexity.

See a video of the talk on YouTube

See the slides on SlideShare

Join The Discussion

Your email address will not be published. Required fields are marked *