Data Engineer Spark and Delta Lake People Prime Worldwide

3.0 - 7.0 years

0 Lacs

hyderabad, telangana

On-site

As a candidate for this role, you would be responsible for the following: - Understanding of Spark core concepts like RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. - Experience with Spark optimization techniques. - Deep knowledge of Delta Lake features like time travel, schema evolution, data partitioning. - Ability to design and implement data pipelines using Spark and Delta Lake as the data storage layer. - Proficiency in Python/Scala/Java for Spark development and integrate with ETL process. - Knowledge of data ingestion techniques from various sources (flat files, CSV, API, database). - Understanding of data quality best practices and data validation techniques. Qualificatio...

Posted 2 weeks ago

AI Match Score

Apply

Senior Data Engineer S M Software Solutions Inc

5.0 - 9.0 years

0 Lacs

hyderabad, telangana

On-site

As an experienced candidate with over 8 years of overall experience and 5+ years of relevant experience, your role will involve the following key responsibilities: - Strong understanding of Spark core including RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming - Proficiency in Delta Lake features such as time travel, schema evolution, and data partitioning - Designing and building data pipelines using Spark and Delta Lake - Solid experience in Python/Scala/Java for Spark development - Knowledge of data ingestion from files, APIs, and databases - Familiarity with data validation and quality best practices - Working knowledge of data warehouse concepts and data modeling - Hands-on expe...

Posted 2 weeks ago

AI Match Score

Apply

Data and Analytics Architect - L1 Wipro

8.0 - 10.0 years

10 - 12 Lacs

Bengaluru

Work from Office

Role Purpose The purpose of the role is to define and develop Enterprise Data Structure along with Data Warehouse, Master Data, Integration and transaction processing with maintaining and strengthening the modelling standards and business information. Do 1. Define and Develop Data Architecture that aids organization and clients in new/ existing deals a. Partnering with business leadership (adopting the rationalization of the data value chain) to provide strategic, information-based recommendations to maximize the value of data and information assets, and protect the organization from disruptions while also embracing innovation b. Assess the benefits and risks of data by using tools such as b...

Posted 3 months ago

AI Match Score

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.