Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Jobs

Interviews

Home
>
Jobs in Bengaluru
>
Ai Palatte
>
Lead Data Engineer

Job Expired: This job listing has expired.

Lead Data Engineer

Ai Palatte

8 - 13 years

45 - 90 Lacs

Bengaluru

Posted:6 months ago| Platform:

AI Match Score

Apply

Skills Required

Automation Team management Data modeling Postgresql MySQL Data structures Oracle SQL Python

Work Mode

Work from Office

Job Type

Full Time

Job Description

Responsibilities:

Lead a team of data engineers specialising in data engineering, providing technical guidance, mentoring, and performance feedback.

Optimise and automate processes for regularly refreshing the worlds largest consumer data lake, purpose-built for CPG.

Drive higher levels of automation and faster process (TAT) for weekly, monthly, and quarterly refresh processes.

Drive initiatives for ongoing improvements in the quality and completeness of data on the platform

Design, develop, and maintain spark data pipelines, ensuring efficient and timely data acquisition from various sources.

Develop and enforce data engineering best practices, standards, and processes.

Identify and resolve issues related to data engineering, such as handling complex data structures, mitigating bottlenecks, and addressing scale issues.

Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and develop scalable data engineering solutions.

Collaborate with stakeholders to define data engineering project requirements, timelines, and deliverables.

Perform data extraction, transformation, and loading (ETL) tasks to convert raw data into usable formats for downstream analysis and processing.

Monitor data pipeline performance and implement mechanisms to ensure the reliability and scalability of data pipelines.

Stay up to date with the latest trends and advancements in data engineering techniques and related technologies.

Requirements:

Bachelors or masters degree in computer science, data engineering, or a related field.

Proven experience (8+ years) working as a data engineer, with a specialisation in data engineering.

Strong programming skills in languages such as Python, Java, and Spark Pipelines.

Proficiency in SQL and database technologies (e.g., PostgreSQL, MySQL, or Oracle) for data storage and retrieval.

Familiarity with cloud platforms (e.g., AWS, Azure, or Google Cloud) and related data services for scalable and reliable data crawling.

Knowledge of data modeling, data warehousing, and ETL processes.

Experience with data warehouses such as Snowflake, Redshift, Elastic Search

Strong analytical and problem-solving skills, with a focus on data quality and accuracy.

Excellent leadership and team management abilities, with a proven track record of leading data engineering teams.

Effective communication and collaboration skills, with the ability to explain complex technical concepts to non-technical stakeholders

https://lh3.googleusercontent.com/p/AF1QipOIjL5OIM8MdjmVJtS8lCwx04dEdCbQj5DtCRA=s451-k-no

https://lh3.googleusercontent.com/p/AF1QipNMgF8SFQvffdZB_op2MmWeeDh4-VP43Hgjwug=s677-k-no

https://lh3.googleusercontent.com/p/AF1QipPB43Xn61Lg60QAB_9Pd7DmvzzvFCSecaOj-n4=s387-k-no

https://lh3.googleusercontent.com/p/AF1QipPVipjWtOFXEy5dEGfN_nUJLt76W3Hsc6vVBYc=s406-k-no

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.