AGM - Lead Data Engineer

Job Purpose

We are looking for a hands-on Data Engineer with 3 to 5 years of experience to build, manage, and scale data pipelines, deploy ML solutions, and enable advanced data visualizations and dashboards for business consumption. The ideal candidate will have a strong engineering mindset, deep understanding of data infrastructure, and prior experience working on self-managed or private cloud (VM-based) deployments.

Key Result Areas/Accountabilities

• Design and build robust, scalable, and secure data pipelines (batch and real-time) to support AI/ML workloads and BI dashboards.
• Collaborate with data scientists to operationalize ML models, including containerization (Docker), CI/CD pipelines, model serving (FastAPI/Flask), and monitoring.
• Develop and maintain interactive dashboards using tools such as Plotly Dash, Power BI, or Streamlit to visualize key insights for business stakeholders.
• Manage deployments and orchestration on Vis local private cloud infrastructure (VM-based setups).
• Work closely with analytics, business, and DevOps teams to ensure reliable data availability and system health.
• Optimize ETL/ELT workflows for performance and scale across large telecom datasets.
• Implement data quality checks, governance, and logging/monitoring solutions for all production workloads.

Core Competencies, Knowledge,Experience

• Strong proficiency in Python, SQL, and data pipeline frameworks (Airflow, Luigi, or similar).
• Solid experience with containerization (Docker), scripting, and deploying production-grade ML or analytics services.
• Hands-on experience with dashboarding and visualization tools such as:
• Power BI / Tableau / Streamlit
• Custom front-end dashboards (nice to have)
• Experience working on self-managed VMs, bare-metal servers, or local private clouds (not just public cloud services).
• Familiarity with ML deployment architectures, REST APIs, and performance tuning.
• 58 years of experience in data engineering, platform development, and/or ML deployment.

Must have technical / professional qualifications

B.Tech/M.Tech from Tier-1 or Tier-2 institutes (IITs, NITs, IIITs, BITS, etc.)