Bryan Cutler is a software engineer at IBM's Center for Open-Source Data & AI Technologies (CODAIT), where he works on big data analytics and machine learning systems. He is a committer for Apache Spark and Apache Arrow, with a focus on Python and ML development. His interests are in pushing the boundaries of software to build high performance tools that are also a snap to use.
Better utilize all available cluster resources when running a large Spark ML job by enabling model parallelism. Beginning with Apache Spark 2.3.0, it will be...