Data Engineering Lead

Data Engineering Lead:

Design and implement scalable data pipelines using technologies such as Apache Spark, Kafka, and AWS Glue.

Develop and maintain ETL processes to ingest, transform, and store structured and unstructured data.

Optimize data storage solutions, including data lakes, data warehouses, and relational databases.

Collaborate with cross functional teams to ensure data availability and quality.

Work with big data technologies such as Hadoop, Snowflake, or Big Query for large-scale data processing.

Monitor and troubleshoot data pipeline failures, optimizing performance and reliability.

Implement security best practices for data storage, access control, and encryption.

Stay up to date with the latest data engineering trends, tools, and frameworks to enhance the data ecosystem.

Required skills:

SQL & Database Management

Data Warehousing

ETL (Extract, Transform, Load)

Big Data Technologies

Programming (Python, R languages, Scala)

Cloud Platforms (AWS, Azure)

Data Governance & Security

Nice to Have:

Machine learning algorithms, deep learning, data handling and analysis.

Experience in BI tools for data visualization.

Domain experience in Insurance / Health care industry.