Data Engineering Sr. Analyst

Teamware Solutions

2 - 5 years

4 - 7 Lacs

Bengaluru Karnataka India

Posted:4 weeks ago| Platform: Foundit logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Key Responsibilities:
Data Pipeline Development:
Design, develop, and implement scalable, efficient
data pipelines
to ingest, transform, and load data from various sources (e.g., databases, flat files, APIs, cloud services).
Work with
ETL tools
(e.g.,
Apache Spark
,
Talend
,
Informatica
) to extract, transform, and load data into the organization's data warehouse or data lake.
Optimize data pipelines for speed, scalability, and cost-efficiency, ensuring that they can handle large volumes of data.
Data Architecture & Design:
Collaborate with data architects and engineers to design and implement
data storage solutions
(e.g.,
data lakes
,
data warehouses
,
NoSQL databases
) that support business intelligence and data analytics.
Create and maintain data
models
,
schemas
, and
data dictionaries
to ensure data consistency and integrity across different systems.
Ensure the application of best practices for
data governance
, including
data quality
,
metadata management
, and
data security
.
Data Integration:
Integrate diverse datasets from internal and external sources (e.g., third-party APIs, cloud-based systems) into the data environment, ensuring compatibility and consistency.
Develop and maintain automated
data integration workflows
, ensuring smooth and reliable movement of data between systems.
Perform data
cleansing
and
transformation
to ensure high-quality data for reporting, analysis, and decision-making.
Big Data & Cloud Solutions:
Work with
cloud data platforms
(e.g.,
AWS Redshift
,
Azure Synapse Analytics
,
Google BigQuery
) to build scalable and efficient data storage solutions.
Implement
big data technologies
such as
Hadoop
,
Spark
, and
Kafka
to handle large datasets and enable real-time data processing.
Optimize and manage cloud-based data services for optimal performance, scalability, and cost-effectiveness.
Collaboration & Cross-functional Support:
Collaborate with data scientists, business analysts, and other stakeholders to understand their data requirements and deliver solutions that meet business needs.
Provide technical support to data analysts and data scientists, ensuring they have the data and tools needed to perform their tasks efficiently.
Work closely with stakeholders to translate business requirements into technical specifications and data solutions.
Data Quality & Performance Monitoring:
Monitor and maintain data pipeline performance, troubleshoot issues, and resolve any data quality issues to ensure high data accuracy and reliability.
Implement monitoring and alerting solutions to track pipeline performance and data quality, ensuring proactive issue resolution.
Conduct regular performance tuning and optimization of data systems and queries to enhance speed and efficiency.
Documentation & Reporting:
Create and maintain
technical documentation
for data pipelines, architecture, and processes, ensuring that they are clear, concise, and accessible to the team.
Develop reports and dashboards to communicate data quality, pipeline status, and other key metrics to stakeholders and management.
Ensure that all data work is thoroughly documented and adheres to compliance and data governance policies.
Required Qualifications:
Bachelor's degree
in
Computer Science
,
Engineering
,
Information Systems
, or a related field.
3-5 years
of experience in
data engineering
,
data architecture
, or related fields.
Proficiency in
SQL
, including the ability to write complex queries and optimize query performance.
Experience with
ETL tools
such as
Apache Spark
,
Talend
,
Informatica
, or custom ETL frameworks.
Strong knowledge of
big data technologies
like
Hadoop
,
Kafka
,
Spark
, or
Flume
.
Familiarity with
cloud platforms
such as
AWS
,
Azure
, or
Google Cloud Platform
, particularly for data storage and processing solutions.
Experience with
data warehousing
solutions such as
Redshift
,
Snowflake
, or
BigQuery
.
Understanding of
data integration
,
data modeling
, and
data governance
best practices.
Strong problem-solving skills with the ability to troubleshoot data issues and optimize systems for performance.

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.