Data Engineer

JOB POSITION

Carelon Global Solutions India is seeking for a Senior Software Engineer who is responsible for understanding the requirements, have to convert the requirements to Java code, test the code, also project in the front end with Angular/any front-end technology tool, test it and make it available for Business users.

EXPERIENCE

5-8 Years of Experience

SKILLS AND COMPETENCIES

• Design and develop efficient and scalable data pipelines using Scala + Spark and Python + Spark
• Optimize and maintain SQL queries and stored procedures for performance and scalability
• Work with Snowflake to manage data warehousing, transformation, and ingestion processes
• Write robust and reusable Shell scripts for automation and deployment tasks
• Leverage key AWS services (S3, Lambda, Glue, EC2, CloudWatch, etc.) to build cloud-native solutions
• Collaborate with cross-functional teams (Product, DevOps, QA) to design and deliver high-quality software
• Participate in code reviews, design discussions, and contribute to technical documentation
• Troubleshoot production issues and ensure system reliability and performance
• Mentor junior developers and contribute to improving engineering best practices

Required Skills and Qualifications:

• Strong expertise in Spark with Scala and Python experience designing and optimizing Spark jobs for large datasets
• SQL ability to write complex SQL queries and analyze large datasets efficiently (Medium to High proficiency)
• Shell Scripting hands-on experience with Bash or other scripting tools for automation (Medium proficiency)
• Snowflake experience with data loading, transformation, and query optimization (Medium proficiency)
• AWS familiarity with key cloud services like S3, Lambda, Glue, EC2, IAM, etc. (Medium proficiency)
• Excellent problem-solving and communication skills

Nice to Have:

• Experience with CI/CD tools (e.g., Jenkins, GitLab)
• Exposure to Airflow or similar orchestration tools
• Understanding of Data Lake architecture and Data Mesh concepts