Senior Data Engineer

Linkfields Innovations (Pty) Ltd

7 years

0 Lacs

hyderabad telangana india

Posted:3 hours ago| Platform:

Apply

Skills Required

data analytics power ai design etl support learning reporting kafka spark azure iot consistency reliability indexing query aws governance management automate orchestration airflow engineering sql relational nosql server mysql mongodb processing integration hadoop redshift gcp optimization python scala java containerization docker compliance certifications

Work Mode

On-site

Job Type

Contractual

Job Description

Are you passionate about building scalable data pipelines and enabling real-time analytics? Do you enjoy designing and optimizing data architectures that power business intelligence, AI, and advanced analytics? If so, Linkfields invites you to be part of its Data & AI Center of Excellence (CoE) as a Senior Data Engineer.

Responsibilities and Duties

Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
Build and optimize data architectures to support analytics, machine learning, and reporting workloads.
Work with real-time streaming data using tools like Kafka, Spark Streaming, or Azure Event Hubs.
Integrate data from multiple sources (databases, APIs, cloud platforms, IoT devices).
Ensure high data quality, consistency, and reliability through validation and monitoring.
Collaborate with Data Scientists, Analysts, and Application Developers to deliver business use cases.
Implement best practices for data modeling, partitioning, indexing, and query optimization.
Manage and optimize cloud-based data platforms (Azure Data Lake, AWS S3, Google BigQuery, Snowflake).
Establish data governance, lineage, and metadata management practices.
Automate workflows using orchestration tools such as Airflow, Azure Data Factory, or AWS Glue.

Desired Experience & Qualification

Bachelor’s or Master’s Degree in Computer Science, Data Engineering, or related field.
7 years of hands-on experience as a Data Engineer, with strong expertise in ETL/ELT design and implementation.
Proficiency in SQL and experience with relational and NoSQL databases (SQL Server, MySQL, MongoDB, Cassandra).
Strong experience with real-time data processing frameworks (Kafka, Spark Streaming, Flink).
Hands-on experience with data integration and orchestration tools (Airflow, Data Factory, AWS Glue, dbt).
Experience with big data platforms (Hadoop, Spark, Databricks).
Strong knowledge of cloud data services (Azure Synapse, AWS Redshift, GCP BigQuery, Snowflake).
Excellent problem-solving and optimization skills for handling large-scale datasets.
Proficiency in Python/Scala/Java for building data pipelines.

Preferred Qualifications:

Experience in designing event-driven data architectures and microservices.
Knowledge of containerization and orchestration (Docker, Kubernetes).
Exposure to machine learning workflows and MLOps for data preparation.

Familiarity with data governance frameworks and compliance standards (GDPR, HIPAA).
Cloud certifications in Azure Data Engineer, AWS Big Data, or GCP Data Engineer tracks.

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.