Big Data Engineer

Job Description

Description We are seeking an experienced Big Data Engineer to join our dynamic team in India. The ideal candidate will have extensive experience in designing and implementing big data solutions that enable data-driven decision-making. You will work closely with cross-functional teams to develop data pipelines, optimize data processing, and ensure data quality and compliance. Responsibilities Design and implement scalable big data solutions using technologies such as Hadoop, Spark, and Kafka. Develop and maintain data pipelines for ingesting, processing, and storing large datasets. Collaborate with data scientists and analysts to understand data requirements and provide necessary data support. Optimize data processing and storage solutions for performance and efficiency. Implement data governance and security best practices to ensure data quality and compliance. Monitor and troubleshoot data systems to ensure high availability and reliability. Stay updated with the latest big data technologies and trends to continuously improve data architecture. Skills and Qualifications 11-16 years of experience in big data technologies and frameworks. Proficiency in programming languages such as Java, Scala, or Python. Extensive experience with Hadoop ecosystem components including HDFS, MapReduce, Hive, and Pig. Hands-on experience with Apache Spark for large-scale data processing. Knowledge of data warehousing solutions and ETL tools. Experience with cloud platforms such as AWS, Azure, or Google Cloud for big data solutions. Strong understanding of data modeling, database design, and data architecture principles. Familiarity with SQL and NoSQL databases like MongoDB, Cassandra, or HBase. Excellent analytical and problem-solving skills, with the ability to work in a fast-paced environment.