Job
Description
Experience: 6 to 9 years Location: [Chennai, Bangalore & Hyderabad] Immediate Joiner or serving Notice Period Preferred. Job Summary: We are looking for a skilled Data Engineer to design, build, and optimize data solutions leveraging Big Data, Cloud, and Open-Source technologies on Azure, AWS, GCP, and Snowflake . In this role, you will work with structured and unstructured data , develop scalable data pipelines , and collaborate with cross-functional teams to drive impactful analytics solutions. Key Responsibilities: Design and develop scalable, metadata-driven data ingestion pipelines for batch and streaming datasets. Implement high-performance data processing solutions for data harmonization and transformation. Develop and manage orchestration, scheduling, and validation for data pipelines. Implement exception handling, monitoring, and logging for debugging and performance tuning. Collaborate with Consulting, Data Science, and Application Development teams to deliver analytical solutions. Work with big data technologies such as Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, and Elastic Search. Design and build REST APIs to facilitate seamless data integration. Ensure data governance, security, and accessibility for BI and Data Science teams. Required Qualifications: 6 to 9 years of IT experience, with at least 4+ years in Big Data Engineering on AWS, Azure, GCP, or Snowflake . Hands-on experience in building Data Lakes using: Azure Data Factory (ADF), PySpark, Databricks, ADLS, Azure SQL Database Azure Synapse Analytics, Event Hub, Streaming Analytics, Cosmos DB, Purview Strong programming and debugging skills in SQL, Python, Scala, or Java . Experience working with Delta Lake, Avro, Parquet, JSON, CSV file formats. Knowledge of Data Lake and Lakehouse architectures . Preferred Qualifications (Good to Have): Experience with building REST services and implementing service-oriented architectures . Certifications in AWS, Azure, GCP, or Snowflake Data Engineering . Experience supporting BI and Data Science teams in secure and governed data consumption.
