Data Engineer (Apache Flume & Kafka)

Coforge

4 - 8 years

10 - 20 Lacs

hyderabad pune greater noida

Posted:3 days ago| Platform:

Apply

Skills Required

apache flume airflow data engineering kafka spark aws sql python

Work Mode

Work from Office

Job Type

Full Time

Job Description

Data Engineer (Apache Flume)

SQL

Key Responsibilities

Log Migration:

Design and implement a solution to extract and transfer historical logs from SQL Server to Apache Flume.
Ensure data integrity, completeness, and consistency during migration.
Daily Log Ingestion:
Develop and automate a pipeline to move newly generated logs from SQL Server to Flume on a daily basis.
Monitor and maintain the pipeline for performance and reliability.
System Integration & Optimization:
Collaborate with infrastructure and application teams to ensure seamless integration.
Optimize data flow and troubleshoot issues in real-time.
Documentation & Reporting:
Maintain detailed documentation of architecture, processes, and configurations.
Provide regular updates and reports to stakeholders.

Required Skills & Qualifications:

3+ years of experience in data engineering or ETL development.
Strong expertise in SQL Server and Apache Flume.
Proficiency in scripting languages (e.g., Python, Shell) for automation.
Experience with log management, data pipelines, and system integration.
Familiarity with big data ecosystems (e.g., Hadoop, Kafka) is a plus.
Excellent problem-solving and communication skills.

Aarushi.Shukla@coforge.com

More Jobs at Coforge

ServiceNow ITSM Developer

Pune, Greater Noida, Hyderabad

3 - 8 yrs

INR 15 - 30 Lacs

Snowflake Admin

Hyderabad

5 - 8 yrs

INR 15 - 25 Lacs

Coforge is Hiring Automation Tester(Selenium with Python, Pytest, AWS)

Hyderabad

6 - 10 yrs

INR 15 - 27 Lacs

Full Stack .Net Developer

Greater Noida

4 - 9 yrs

INR 15 - 30 Lacs

Etl Testing Engineer

Gurugram

4 - 9 yrs

INR 10 - 18 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.