Big Data Lead

You will be employed as a Big Data Lead on a full-time basis at our office in Noida. As a Big Data Lead, your primary responsibility will be to lead various data analytics projects, manage data architecture, conduct data modeling, and supervise data management tasks. This role will involve the analysis of extensive datasets to extract valuable insights, ensure data quality and consistency, and collaborate with cross-functional teams to implement data-driven strategies effectively. Your qualifications should include hands-on experience in Big Data Processing tools such as Apache Spark, Trino, Minio/S3, Clickhouse, and SQL for both batch and streaming data pipelines. You should be proficient in designing and developing scalable data pipelines using Apache Spark (PySpark) and Python, as well as implementing batch and real-time data processing solutions on substantial datasets. Additionally, expertise in working with various SQL and NoSQL databases like PostgreSQL, MySQL, MongoDB, Cassandra, and DynamoDB is essential. Furthermore, you should possess knowledge in Data Migration & Warehousing, including metadata management, ETL processes, and cloud-based data solutions. Strong skills in data analysis, pattern recognition, and data visualization using tools like Tableau or Power BI are required. Collaboration with data architects, analysts, and other engineers to understand business requirements and translate them into technical solutions is a key aspect of this role. Moreover, you should be capable of optimizing and troubleshooting Spark jobs for performance, scalability, and reliability. Exposure to containerization technologies such as Docker and orchestration tools like Kubernetes would be beneficial for this position.,