Senior Data Engineer - cloud data services - Python,Spark, Hadoop, Hive, Kafka

𝐀𝐛𝐨𝐮𝐭 𝐔𝐬:

AutomatonsX is a new-age brand from CtrlS-Cloud4C group, mission-focused on architecting and delivering intelligent automations for large-scale organizations in key industries. We go beyond regular RPA, rendering hyperautomation solutions powered by data transformations and deep analytics, autonomous platforms, and GenAI innovations. From manufacturing, healthcare, energy, BFSI, logistics industries to the BUs under them, we envision helping leaders smartly automate redundant processes at scale and improve productivity multi-fold, freeing up human resources for critical business and creative activities. Backed by CtrlS-Cloud4C group, the leading rated-IV datacenter and application-focused multi-cloud MSP globally, our experts have saved 1.2 million man hours and supported 5X faster GTMs. That's only the tip of the iceberg! Make your enterprise ready for the intelligent wave. Unlock the X-factor in business growth with the powerful integration of Analytics, AI, and Automation.

Job Summary:

Senior Data Engineer

Key Responsibilities:

• Design, build, and maintain scalable data pipelines and systems.
• Develop and optimize ETL/ELT processes for data ingestion, transformation, and loading.
• Collaborate with cross-functional teams to understand data needs and deliver reliable data solutions.
• Ensure data quality, integrity, and security across all platforms.
• Implement and monitor data infrastructure using cloud-based solutions (AWS, Azure, or GCP).
• Work with large datasets using technologies such as Spark, Kafka, and Hadoop.
• Develop data models and maintain data dictionaries.
• Drive automation and performance improvements across the data platform.
• Mentor junior team members and contribute to best practices and technical standards.

Required Skills and Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 3+ years of experience in data engineering or related fields.
• Proficiency in SQL and one or more programming languages (Python, Scala, Java).
• Hands-on experience with big data technologies like Spark, Hadoop, Hive, Kafka.
• Experience with cloud data services (AWS Redshift, Azure Synapse, GCP BigQuery).
• Strong understanding of data modeling, warehousing, and distributed systems.
• Familiarity with CI/CD and DevOps practices in data engineering.
• Excellent problem-solving and communication skills.

Preferred Qualifications:

• Experience with workflow orchestration tools (e.g., Airflow, Luigi).
• Familiarity with data governance and compliance practices.
• Background in real-time data streaming and event-driven architecture