About the role
Senior Spark QA Engineer – Functional & Performance Testing
Location: Remote Experience: 5+ Years
Job Description - We are looking for a Senior Spark QA Engineer with strong expertise in functional and performance testing of Apache Spark applications and distributed data platforms.
Key Responsibilities - * Perform manual and automated testing of Spark jobs, Spark SQL queries, and ETL pipelines. * Execute functional, scalability, and performance testing for Spark workloads. * Set up and manage Spark clusters on Standalone, YARN, Kubernetes, Mesos, Databricks, EMR, and Dataproc. * Conduct benchmarking, validation, and performance analysis of Spark applications. * Identify bottlenecks and troubleshoot distributed system issues. * Lead QA initiatives and mentor team members.
Required Skills -
* 5+ years of QA/testing experience with strong hands-on expertise in Apache Spark. * Experience in functional and performance testing of distributed systems. * Strong understanding of Spark architecture and optimization techniques. * Experience with Kubernetes and cloud-based Spark environments. * Proficiency in Python or Java. * Experience with automation frameworks and CI/CD pipelines.
Originally posted on Himalayas