Accelerating Apache Spark with RDMA
by Rich Brueckner from Inside HPC & AI News | High-Performance Computing & Artificial Intelligence on (#2HXRV)
Yuval Degani from Mellanox presented this talk at the OpenFabrics Workshop. "In this talk, we present a Java-based, RDMA network layer for Apache Spark. The implementation optimized both the RPC and the Shuffle mechanisms for RDMA. Initial benchmarking shows up to 25% improvement for Spark Applications."
The post Accelerating Apache Spark with RDMA appeared first on insideHPC.