Data Engineering Manager

Company Description

Cloud Destinations, headquartered in Silicon Valley, specializes in digital transformation and enterprise application development for the Healthcare, Retail, and Technology sectors. We excel in cloud migrations, multi-cloud operations, DevOps, security operations, and IoT. Our globally distributed team includes skilled technical experts and effective leaders who thrive on teamwork and respect for diverse viewpoints.

For more details, visit our website: www.clouddestinations.com.

Role Description

Coimbatore

• Team Leadership:

  Lead, mentor, and manage a team of data engineers, ensuring effective collaboration, professional growth, and high-quality output.

• Data Architecture:

  Design, build, and maintain robust data pipelines and architectures that efficiently support data-driven processes and systems.

• Data Strategy:

  Develop and execute data engineering strategies in alignment with business goals and evolving data needs.

• Data Integration:

  Oversee the integration of data from various sources, including third-party APIs, databases, and streaming platforms.

• Optimization & Maintenance:

  Ensure high performance, scalability, and reliability of data processing systems, continuously optimizing workflows and improving data quality.

• Collaboration:

  Work closely with data scientists, analysts, and business stakeholders to understand their data needs and deliver solutions accordingly.

• Cloud Technologies:

  Oversee the adoption and integration of cloud-based solutions (AWS, Azure, Google Cloud) to manage and process large-scale datasets efficiently.

• Governance & Security:

  Implement and maintain best practices for data governance, data privacy, and security in line with industry standards and regulations.

• Stakeholder Communication:

  Regularly communicate progress, challenges, and outcomes to stakeholders across the organization, providing insights and solutions as needed.

• Project Management:

  Plan, prioritize, and oversee data engineering projects, ensuring timely delivery of solutions that meet both business and technical requirements.

Technical Expertise:

• Strong proficiency in programming languages such as

  Python, Java, or Scala.

• Experience with data processing frameworks like

  Apache Spark, Kafka, Hadoop

  , or similar technologies.
• Hands-on experience with

  ETL tools

  and techniques, including workflow orchestration (e.g

  Airflow, Luigi

  ).
• Strong SQL skills and familiarity with

  NoSQL databases

  (e.g., MongoDB, Cassandra).
• Proven experience with cloud platforms (

  AWS, Google Cloud, Azure

  ) and related data storage solutions (

  S3, Redshift, BigQuery, etc.

  ).

• Leadership

  : Proven experience leading and mentoring data engineering teams, fostering a culture of continuous learning and collaboration.

• Data Architecture

  : Expertise in designing scalable and efficient data architectures, pipelines, and workflows.

• Problem Solving

  : Strong analytical and problem-solving skills, with the ability to translate business requirements into technical solutions.

• Communication

  : Excellent communication skills, capable of collaborating with both technical and non-technical stakeholders.

• Project Management

  : Experience managing multiple projects and deadlines in a fast-paced environment.
• Familiarity with machine learning pipelines and big data platforms.
• Certification in cloud platforms (AWS Certified Data Analytics, Google Cloud, Professional Data Engineer, etc.).
• Knowledge of data governance, compliance, and security best practices (e.g., GDPR, CCPA).

kavyaga@clouddestinations.com