Data Engineer-Data Platforms

Job Description

As a Data Engineer at IBM, you' pay a vita roe in the deveopment, design of appication, provide reguar support/guidance to project teams on compex coding, issue resoution and execution. Your primary responsibiities incude: Lead the design and construction of new soutions using the atest technoogies, aways ooking to add business vaue and meet user requirements. Strive for continuous improvements by testing the buid soution and working under an agie framework. Discover and impement the atest technoogies trends to maximize and buid creative soutions Required education Bacheor's Degree Preferred education Master's Degree Required technica and professiona expertise Experience with Apache Spark (PySpark)In-depth knowedge of Spark’s architecture, core APIs, and PySpark for distributed data processing. Big Data TechnoogiesFamiiarity with Hadoop, HDFS, Kafka, and other big data toos. Data Engineering Skis: Strong understanding of ETL pipeines, data modeing, and data warehousing concepts. Strong proficiency in PythonExpertise in Python programming with a focus on data processing and manipuation. Data Processing FrameworksKnowedge of data processing ibraries such as Pandas, NumPy. SQL ProficiencyExperience writing optimized SQL queries for arge-scae data anaysis and transformation. Coud PatformsExperience working with coud patforms ike AWS, Azure, or GCP, incuding using coud storage systems Preferred technica and professiona experience Define, drive, and impement an architecture strategy and standards for end-to-end monitoring. Partner with the rest of the technoogy teams incuding appication deveopment, enterprise architecture, testing services, network engineering, Good to have detection and prevention toos for Company products and Patform and customer-facing