Spark & Delta Lake Specialist

As a Spark & Delta Lake Specialist, you will be joining our technology services client on a contract basis with a strong potential for full-time employment conversion. Your primary responsibilities will include: - Understanding Spark core concepts such as RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. - Implementing Spark optimization techniques to enhance performance. - Demonstrating deep knowledge of Delta Lake features like time travel, schema evolution, and data partitioning. - Designing and implementing data pipelines using Spark and Delta Lake as the data storage layer. - Proficiency in Python, Scala, and Java for Spark development and integration with ETL processes. - Implementing data ingestion techniques from various sources including flat files, CSV, API, and databases. - Applying data quality best practices and data validation techniques. - Understanding data warehouse concepts and data modeling techniques. - Expertise in using Git for code management. - Familiarity with CI/CD pipelines and containerization technologies. You should have the following qualifications: - 8+ years of experience in the technology industry. - Mandatory skills in Python, Spark (PySpark), SQL, and Delta Lake. - Ability to work in Hyderabad location. - Immediate to 15 days notice period. If interested, please share your updated resume with vinod@s3staff.com. As a Spark & Delta Lake Specialist, you will be joining our technology services client on a contract basis with a strong potential for full-time employment conversion. Your primary responsibilities will include: - Understanding Spark core concepts such as RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. - Implementing Spark optimization techniques to enhance performance. - Demonstrating deep knowledge of Delta Lake features like time travel, schema evolution, and data partitioning. - Designing and implementing data pipelines using Spark and Delta Lake as the data storage layer. - Proficiency in Python, Scala, and Java for Spark development and integration with ETL processes. - Implementing data ingestion techniques from various sources including flat files, CSV, API, and databases. - Applying data quality best practices and data validation techniques. - Understanding data warehouse concepts and data modeling techniques. - Expertise in using Git for code management. - Familiarity with CI/CD pipelines and containerization technologies. You should have the following qualifications: - 8+ years of experience in the technology industry. - Mandatory skills in Python, Spark (PySpark), SQL, and Delta Lake. - Ability to work in Hyderabad location. - Immediate to 15 days notice period. If interested, please share your updated resume with vinod@s3staff.com.