As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing streaming pipelines Experience to work with Hadoop / AWS eco system components to implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Apache Spark, Kafka, any Cloud computing etc Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Total 5 - 7+ years of experience in Data Management (DW, DL, Data Platform, Lakehouse) and Data Engineering skills Minimum 4+ years of experience in Big Data technologies with extensive data engineering experience in Spark / Python or Scala. Minimum 3 years of experience on Cloud Data Platforms on AWS; Exposure to streaming solutions and message brokers like Kafka technologies. Experience in AWS EMR / AWS Glue / DataBricks, AWS RedShift, DynamoDB Good to excellent SQL skills Preferred technical and professional experience Certification in AWS and Data Bricks or Cloudera Spark Certified developers AWS S3 , Redshift , and EMR for data storage and distributed processing. AWS Lambda , AWS Step Functions , and AWS Glue to build serverless, event-driven data workflows and orchestrate ETL processes

More Jobs at IBM

IBM - Hiring For Process and Sr. Process Analyst (TM/ AML)

Bengaluru

1 - 3 yrs

INR 4 - 7 Lacs

Backend Developer

Bengaluru

1 - 3 yrs

INR 3 - 7 Lacs

Data Engineer-Data Platforms-AWS

Bengaluru

2 - 6 yrs

INR 12 - 16 Lacs

Data Engineer-Data Platforms

Hyderabad

6 - 9 yrs

INR 14 - 17 Lacs

Advisory AI Engineer

Bengaluru

1 - 5 yrs

INR 10 - 14 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Data Management Interview Now

My Connections IBM

Download Chrome Extension (See your connection in the IBM )

Download Now

IBM

www.ibm.com

Information Technology

Armonk

350,000 Employees

6362 Jobs

Key People

Arvind Krishna

Chairman and Chief Executive Officer
Ginni Rometty

Former Chairman, President and CEO
James Kavanaugh

Chief Financial Officer
David Ferdinand

Senior Vice President, Corporate Strategy

Login to

Please Verify Your Phone or Email

Confirm Action

Search

Profile

Bookmarks

Data Engineer-Data Platforms-AWS