Spark, Scala And Python

You are a Bigdata Engineer with over 5 years of experience, located in Hyderabad, currently working on a contract basis with the ability to join immediately. Your expertise lies in Spark, Scala, and Python, with a strong proficiency in Apache Spark and Scala programming language. You possess extensive knowledge of Spark's core concepts, RDDs, DataFrames, and Spark SQL, enabling you to write optimized and scalable Spark applications. In addition to Spark and Scala, you are familiar with big data technologies and have a strong understanding of distributed computing concepts. You have hands-on experience in handling large-scale data processing and are well-versed in Kafka messaging, with the ability to develop Kafka-based applications effectively. Your role involves data engineering, where you demonstrate a solid understanding of data engineering principles, data modeling, and ETL (Extract, Transform, Load) processes. You have experience in designing and developing data pipelines, working with various data formats such as ORC, JSON, CSV, and Parquet. Proficient in SQL for data manipulation, querying, and performance optimization, you also have experience with relational databases like MySQL and PostgreSQL. Your knowledge extends to database design principles, enhancing your ability to work efficiently with SQL and databases in your projects.,