Senior Data Engineer

Sonata Software

5 - 8 years

10 - 20 Lacs

hyderabad

Posted:1 month ago| Platform:

Apply

Skills Required

streamsets spark informatica apache nifi sql python

Work Mode

Work from Office

Job Type

Full Time

Job Description

Role & responsibilities

Senior Data Engineer

Key Responsibilities:

Install, configure, and administer data platforms:
StreamSets, Apache Spark, Informatica, Apache NiFi
Develop real-time and batch data pipelines using
PySpark
(Scala is a plus)
Administer
StreamSets Data Collector and Control Hub
for pipeline deployment and monitoring
Automate data flows using
Apache NiFi
, including processor configuration and cluster coordination
Upgrade and migrate data ingestion tools with minimal downtime
Write scalable and efficient
Python
code for data processing and automation
Perform complex
SQL
transformations and optimize database interactions
Work with diverse databases:

MPP
: Teradata, Greenplum
OLTP
: Oracle, SQL Server, PostgreSQL
In-memory
: SAP HANA, MemSQL
Big Data
: Hive, HBase, Kudu

Integrate automated testing using
CI/CD tools
(GitLab preferred, Jenkins acceptable)
Implement deployment pipelines and version control strategies
Ensure platform security, including
TLS hardening
Monitor production environments and perform root cause analysis
Enforce coding standards and data governance practices
Schedule and orchestrate workflows with error handling and retry mechanisms

Mandatory Skills:

Data Ingestion & Transformation Tools
StreamSets
,
Spark (Python preferred)
,
Informatica
,
Apache NiFi
Strong proficiency in
Python
and
SQL

Secondary Skills:

CI/CD tools: GitLab, Jenkins
Database technologies: Teradata, Greenplum, Oracle, SQL Server, PostgreSQL, SAP HANA, MemSQL
Big Data platforms: Hive, HBase, Kudu
Knowledge of data governance, lineage tracking, and validation frameworks

More Jobs at Sonata Software

L2 Support Office 365

Bengaluru

4 - 6 yrs

INR 1 - 6 Lacs

Selenium AT - Pennymac

Bengaluru, Karnataka

7 - 7 yrs

Salary: Not disclosed

Salesforce Architect

Bengaluru

10 - 20 yrs

INR 22 - 37 Lacs

D365 F&O Finance Functional Consultant

Hyderabad, Chennai, Bengaluru

8 - 12 yrs

INR 20 - 35 Lacs

SAP Business Analyst with SAP SD & EDI

Hyderabad, Chennai, Bengaluru

6 - 11 yrs

INR 13 - 23 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.