Data Engineer - Scala

Role Overview: You will be working as a Data Engineer - Scala at the forefront of Experian's cloud modernization within the Credit & Verification Services. This role is a hybrid one, requiring you to travel to the Hyderabad office 40% of the month. You will have the opportunity to be part of a collaborative team that uses Agile DevSecOps principles to deliver business value. Key Responsibilities: - Develop and maintain batch data processing pipelines using Scala and Apache Spark. - Optimize Spark jobs for performance and scalability in a cloud-native environment (AWS). - Collaborate with data engineers and analysts to understand data requirements and deliver high-quality solutions. - Ensure data integrity, reliability, and consistency across systems. - Participate in code reviews, design discussions, and agile development practices. - Monitor and troubleshoot production jobs, ensuring the resolution of issues. Qualifications Required: - 3-6 years of software development experience, with at least 2 years in Scala. - Write clean, scalable, and re-usable code that implements SOLID principles, common design patterns where applicable, and adheres to published coding standards. - Strong understanding of functional programming and distributed computing. - Strong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins. - Hands-on experience with Apache Spark for batch processing. - Familiarity with AWS services such as S3, AWS Glue, Lambda, Step Functions, AWS Batch, and CloudWatch. - Proficiency in working with structured and semi-structured data formats (e.g., Parquet, JSON). - Experience of the full development lifecycle/application Lifecycle Management Tools - BitBucket, Jira. - Experience with data lake architectures and ETL frameworks. - Exposure to containerization tools (Docker, Kubernetes). - Knowledge of data governance and security best practices. - Relevant certifications in AWS, Spark, or Scala.,