7 12 years of experience in managing end-to-end data systems, including data collection, API-driven data delivery, and AI model integrations.
Proven expertise in Python-based data engineering, data scraping, and pipeline orchestration for large-scale analytics and automation projects.
Strong background in building scalable APIs, microservices, and data pipelines using Python, Docker, and Kubernetes.
Experience in customizing open-source language models (LLMs) and integrating them into enterprise solutions.
Proficiency in AWS cloud services, PostgreSQL/MySQL, CI/CD, and DevOps tools for continuous delivery and monitoring.
Ability to lead distributed engineering teams, manage delivery timelines, and drive innovation through automation and AI-driven workflows.

Responsibilities:

Lead and mentor engineering teams responsible for designing, implementing, and maintaining data ingestion, transformation, and delivery pipelines.
Architect data systems combine scraping, big data processing, and AI-based enrichment to support analytics and product intelligence.
Oversee API architecture and system scalability, ensuring high performance and secure data flow across services.
Manage public data scraping and data integration projects that involve large, unstructured datasets and complex relational mappings.
Drive AI model customization using open-source frameworks (LangChain, LlamaIndex, or custom LLM integrations) to improve automation and search intelligence.
Collaborate with product managers, AI teams, and data analysts to translate requirements into actionable data engineering tasks.
Implement and monitor data quality, governance, and compliance standards throughout the data lifecycle.
Optimize deployment processes using Docker, Kubernetes, AWS Lambda, and CI/CD pipelines.
Conduct regular technical reviews, mentor developers, and establish best practices for Python, cloud deployment, and automation.

Qualifications:

Bachelor s or master s degree in computer science, Engineering, or related field.
Hands-on expertise in Python, Django, FastAPI, SQL, and API-driven system design.
Strong understanding of data pipelines, scraping frameworks (BeautifulSoup, Scrapy), and distributed data systems.
Experience with AI/LLM integration, automation frameworks, and data-driven architecture design.
Proficiency in AWS cloud ecosystem, Docker, Kubernetes, and Git-based CI/CD workflows.
Demonstrated leadership in managing multi-member engineering teams and delivering enterprise-grade products.
Excellent analytical, communication, and problem-solving abilities.
Familiarity with Agile methodologies, Jira/ClickUp, and project management best practices.

More Jobs at Aimleap

Data Pipelines Engineering Manager Python

bengaluru

7.0 - 12.0 yrs

INR 15 - 19 Lacs

Technical Lead Manager

bengaluru

7.0 - 12.0 yrs

INR 13 - 18 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.