Job
Description
We are eagerly seeking candidates with 5 to 13 years experience for a Data Engineer / Lead, to join our dynamic team. The ideal candidate will play a pivotal role within the team to who is a skilled professional with exposure to Python, Spark, Hive, AWS. You will collaborate with internal teams to design, develop, deploy, and maintain software applications at scale Role: Data Engineer / Lead Location: PAN India Experience: 5 to 13 years Job type: Full time Work type: Hybrid Data Engineer with minimum 5 years of relevant professional experience • Should have expertise in Python Scripting and Big data technologies like Spark, Hive, Presto etc. • Experience with AWS services – IAM, EC2, S3, EMR, Lambda Functions, Step Functions, CloudWatch, Redshift, Athena, GLUE etc. • Hands-on experience with Databricks. • Proficient writing Spark jobs in Pyspark and Scala. • Experience writing queries with both SQL and NoSQL DB ((Hive, HBase, MongoDB, Elasticsearch, PostgreSQL etc.) • Should have good understanding around python data structures including data frames, datasets, RDD’s etc. • Experience in ML – integration of ML models • Experience with Data profiling, data migration. • Developing Hive UDF and Hive jobs • Proven hands-on Software Development experience • Experience with test-driven development • Preferred experience in the insurance domain • Must have good understanding of Data warehousing concepts. • Experience using CI/CD tools like GitHub Actions, Jenkins, Azure Devops etc. • Experienced working in Agile projects – Sprint planning, grooming and providing estimations. • Experience using JIRA, Confluence, VS Code or similar IDE’s, Jupyter notebooks etc. • Good communication and collaborative skills with internal and external teams • Flexibility and ability to work in onshore/offshore model involving multiple agile teams • Mentor and guide junior developers, review code, familiar with estimation techniques using story points • Strong analytical and problem-solving skills Qualification you must require: Bachelors or master’s with Computer Science or related field
