Senior Data Engineer

Roles & Responsibilities:

• Design and implement end-to-end data engineering solutions by leveraging the full suite of Databricks, Fabric tools, including data ingestion, transformation, and modeling.
• Design, develop and maintain end-to-end data pipelines by using spark, ensuring scalability, reliability, and cost optimized solutions.
• Collaborate with internal teams and clients to understand business requirements and translate them into robust technical solutions.
• Conduct performance tuning and troubleshooting to identify and resolve any issues.
• Implement data governance and security best practices, including role-based access control, encryption, and auditing.
• Translate Business requirements into high-quality technical documents including, data mapping, data processes, and operational support guides.
• Work closely with architects, product managers and reporting team to collect functional and system requirements.
• Work in fast-paced environment and perform effectively in an agile development environment.

REQUIREMENTS

• 8+ years of experience in designing and implementing data solutions with at least 4+ years of experience in data engineering.
• Extensive experience with Databricks, Fabric, including a deep understanding of its architecture, data modeling, and real-time analytics.
• Minimum 6+ years of experience in Spark, PySpark and Python.
• Must have strong experience in SQL, Spark SQL, data modeling & RDBMS concepts.
• Strong knowledge of Data Fabric services, particularly Data engineering, Data warehouse, Data factory, and Real- time intelligence.
• Strong problem-solving skills, with ability to perform multi-tasking.
• Familiarity with security best practices in cloud environments, Active Directory, encryption, and data privacy compliance.
• Communicate effectively in both oral and written.
• Experience in AGILE development, SCRUM and Application Lifecycle Management (ALM).
• Preference given to current or former Labcorp employees.

EDUCATION

• Bachelors in engineering, MCA or equivalent.