“I am learning from the data scientist, and the data scientist is learning from the developer, constantly”
In Episode 3 of the The New Builders, David Taieb, a developer, is joined by Jorge Castañón, a data scientist, to explain how getting a drink together at Datapalooza led to a partnership in which they refined a sample application, built by Taieb, that predicts how much a flight will be delayed, based on weather conditions. Thanks to the rise of cloud computing, collaborative technology and complementary skill sets, Taieb and Castañón were able to achieve 60 percent accuracy with the app – far exceeding their expectations.
We get into the details behind their collaboration, including how open source tools like Simple Data Pipe helped Taieb move on-premises data sources into a cloud-based Apache Spark environment for analysis (8:32), the role of intuition in data science (13:23), how a common language of Spark and IPython notebooks enabled them to collaboratively execute and experiment with data (14:30), how deep learning could further improve the sample app (27:03), and why they think it’s never too late for developers and data scientists to start working together on their projects (29:49).
Check out the flight predictor sample app on Github.
You can find new episodes of The New Builders on iTunes, SoundCloud and developerWorks TV. Find out more about IBM Cloud Data Services at IBM.biz/forbuilders. Contact hosts Doug Flora and Jim Young on Twitter (@DSFlora, @JW_Young) or email (email@example.com, firstname.lastname@example.org).
The show’s music is provided by School for Robots. Check them out at schoolforrobots.bandcamp.com!
[Free eBook]: A Field Guide to the World of Modern Data Stores
Navigating today’s cloud databases and analytics options can be challenging. But it doesn’t have to be intimidating. Start (or level-set) your journey today with A Field Guide to the World of Modern Data Stores – read this FREE eBook to learn:
- What are the defining characteristics and strengths of today’s different cloud databases?
- How can data in NoSQL stores be analyzed to learn more about your customers?
- How are different open source databases used together to achieve polyglot persistence?
Get A Field Guide to the World of Modern Data Stores from IBM Cloud Data Services.