Big Data Engineer

Role Overview: You will work on collecting, storing, processing, and analyzing large sets of data. Your main focus will be on selecting the best solutions for these tasks, as well as maintaining, implementing, and monitoring them. Additionally, you will be responsible for integrating these solutions with the existing company architecture. Key Responsibilities: - Utilize your 6+ years" experience in Big Data technologies such as Hadoop, MapReduce, Hive, Sqoop, and Spark to design and implement high data volume solutions for ETL & Streaming purposes. - Build stream-processing systems using tools like Storm, Spark-Streaming, and Kafka streams. - Work with Big Data tools like Pig, Hive, Athena, Glue, Snowflake, and EMR to handle data effectively. - Utilize NoSQL databases including HBase, Cassandra, and MongoDB. - Implement various ETL techniques and frameworks, such as Flume. - Use messaging systems like Kafka or RabbitMQ for data processing. - Understand Lambda Architecture and its advantages and disadvantages. Qualifications Required: - 6+ years of experience in Big Data technologies (Hadoop, MapReduce, Hive, Sqoop, Spark) - Hands-on expertise in designing and implementing high data volume solutions for ETL & Streaming - Experience with stream-processing systems (Storm, Spark-Streaming, Kafka streams) - Proficiency in working with Big Data tools (Pig, Hive, Athena, Glue, Snowflake, EMR) - Familiarity with NoSQL databases (HBase, Cassandra, MongoDB) - Knowledge of ETL techniques and frameworks (Flume) - Experience with messaging systems (Kafka, RabbitMQ) - Good understanding of Lambda Architecture (Note: No additional details about the company were provided in the job description.),