Big Data Engineer - Spark/Scala

As a Spark Scala Developer, you will be responsible for designing and developing scalable data pipelines using Apache Spark and Scala. Your key responsibilities will include: - Optimizing and troubleshooting Spark jobs for performance, such as memory management, shuffles, and skew - Working with massive datasets in on-prem Hadoop clusters or cloud platforms like AWS, GCP, or Azure - Writing clean and modular Scala code using functional programming principles - Collaborating with data teams to integrate with platforms like Snowflake, Databricks, or data lakes - Ensuring code quality, documentation, and CI/CD practices are followed Qualifications required for this role include: - 3+ years of experience with Apache Spark in Scala - Deep understanding of Spark internals including DAG, stages, tasks, caching, joins, and partitioning - Hands-on experience with performance tuning in production Spark jobs - Proficiency in Scala functional programming, including immutability, higher-order functions, and Option/Either - Proficiency in SQL - Experience with any major cloud platform such as AWS, Azure, or GCP Nice-to-have skills for this position include: - Experience working with Databricks, Snowflake, or Delta Lake - Exposure to data pipeline tools like Airflow, Kafka, Glue, or BigQuery - Familiarity with CI/CD pipelines and Git-based workflows - Comfort with SQL optimization and schema design in distributed environments Please note that the company details were not provided in the job description.,