Data Engineer

Working Hours: 3:00 PM IST – 12:00 AM IST

Primary Responsibilities

• Build, scale, and maintain robust data solutions to support business objectives.
• Design, implement, and optimize high-performance data pipelines (extraction, loading, transformation, and orchestration) with scalability, reliability, and speed.
• Lead end-to-end software development projects involving large language models (LLMs), retrieval-augmented generation (RAG) frameworks, and other AI technologies.
• Champion modern software engineering practices such as CI/CD, infrastructure-as-code, containerization, and cloud-native deployments.
• Collaborate with business stakeholders to transform use cases into production-ready services and solutions, owning systems from concept to production.
• Implement rigorous testing and monitoring practices to ensure superior data quality and integrity.
• Mentor and guide junior team members, fostering a culture of excellence and continuous learning.

Requirements

• Bachelor’s degree or higher in a STEM field (Computer Science, Math, Physics, Engineering, or related), required.
• Advanced degree in Computer Science or related field, preferred.
• 5–7 years of total professional experience.
• 3–5 years of experience in Data Engineering or a related discipline with a proven track record of success.
• Experience in financial services or product development environments is a strong plus.
• Expertise in Python and SQL with strong data manipulation and analysis skills.
• Proficiency with Databricks/PySpark and dbt for data warehousing and transformation.
• Hands-on experience with workflow orchestration tools (e.g., Airflow, Temporal).
• Experience working with LLMs, prompt engineering, RAG frameworks, and vector databases.
• Knowledge of machine learning fundamentals, feature engineering, and knowledge Graphs are an advantage.
• Strong experience in designing and implementing complex data systems from scratch.
• Ability to handle large-scale data projects including ETL, data cleaning, and information retrieval.
• Strong communication skills (verbal and written).