Data Engineer Spark and Delta Lake

About Client :- Our client is a French multinational information technology (IT) services and consulting company, headquartered in Paris, France. Founded in 1967, It has been a leader in business transformation for over 50 years, leveraging technology to address a wide range of business needs, from strategy and design to managing operations. The company is committed to unleashing human energy through technology for an inclusive and sustainable future, helping organizations accelerate their transition to a digital and sustainable world. They provide a variety of services, including consulting, technology, professional, and outsourcing services. Job Details:- location : Hyderabad Mode Of Work : Hybrid Notice Period : Immediate Joiners Experience : 6-8 yrs Type Of Hire : Contract to Hire JOB DESCRIPTION: • Understanding of Spark core concepts like RDD’s, DataFrames, DataSets, SparkSQL and Spark Streaming. • Experience with Spark optimization techniques. • Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning. • Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer. • Proficiency in Python/Scala/Java for Spark development and integrate with ETL process. • Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database) • Understanding of data quality best practices and data validation techniques. Other Skills: • Understanding of data warehouse concepts, data modelling techniques. • Expertise in Git for code management. • Familiarity with CI/CD pipelines and containerization technologies. • Nice to have experience using data integration tools like DataStage/Prophecy/Informatica/Ab Initio" Show more Show less