Job
Description
Role Description Role Overview We are looking for a highly skilled Senior Data Engineer with strong hands-on experience in Azure Databricks, PySpark, and SQL , who can creatively design and develop high-performance data applications. The ideal candidate should be capable of optimizing solutions, mentoring team members, and collaborating with stakeholders to deliver quality software aligned with business goals. Roles & Responsibilities Technical Development Develop application components based on design specifications (HLD/LLD). Code, debug, test, and document the development lifecycle. Ensure adherence to coding standards and reuse best practices and design patterns. Optimize for performance, scalability, and cost efficiency. Testing & Quality Create and review unit test cases, test scenarios, and execution plans. Collaborate with QA teams to review test plans and provide technical clarifications. Conduct root cause analysis (RCA) for defects and implement mitigation strategies. Design & Architecture Contribute to the design and architecture of components/features (HLD, LLD, SAD). Participate in interface and data model design discussions. Evaluate and suggest technical options including reuse, reconfiguration, or new implementation. Project & Delivery Manage and deliver assigned modules/user stories within agile sprints. Provide effort estimates and assist in sprint planning. Execute and monitor the release and deployment process. Documentation & Configuration Create/review development checklists, design templates, and technical documentation. Define and enforce configuration management practices. Domain Understanding Understand business needs and translate them into technical requirements. Advise developers by applying domain knowledge to product features. Pursue domain certifications to enhance project contributions. Team & Stakeholder Management Set and review FAST goals for self and team members. Provide mentorship and address people-related issues in the team. Interface with customers to gather requirements, present solutions, and conduct demos. Maintain motivation and positive dynamics within the team. Knowledge Management Contribute to internal documentation repositories and client-specific knowledge bases. Review and promote reusable components and best practices. Must-Have Skills Azure Databricks – Hands-on experience with Spark-based analytics on Azure. PySpark – Proficient in developing and optimizing large-scale ETL pipelines. SQL – Strong command of writing, optimizing, and troubleshooting complex queries. Agile Methodology – Experience working in Scrum/Kanban environments. Software Development Life Cycle (SDLC) – Solid understanding from requirement to release. Experience in effort estimation, defect triage, and client communication. Good-to-Have Skills Azure Data Lake, Azure Data Factory, Synapse Analytics CI/CD pipelines using Azure DevOps or GitHub Actions Data modeling and Interface definition techniques Domain experience in healthcare, finance, or other data-heavy industries Experience with unit testing frameworks (e.g., PyTest) Familiarity with business intelligence tools (Power BI, Tableau) Certifications (Preferred) Microsoft Certified: Azure Data Engineer Associate Databricks Certified Developer – Apache Spark using Python Agile/Scrum Certification (CSM, PSM, etc.) Soft Skills Strong analytical and problem-solving abilities Ability to work under pressure and handle multiple tasks Excellent communication and client-facing skills Proactive attitude and team mentorship capabilities Ability to drive technical discussions and solution demos Skills Adb,Adf,Mongodb,Pyspark Show more Show less
