Big Data Engineer

Job Description

We are looking for a Senior Big Data Engineer with deep experience in building scalable, high-performance data processing pipelines using Snowflake (Snowpark) and the Hadoop ecosystem . You'll design and implement batch and streaming data workflows, transform complex datasets, and optimize infrastructure to power analytics and data science solutions. Key Responsibilities: Design, develop, and maintain end-to-end scalable data pipelines for high-volume batch and real-time use cases. Implement advanced data transformations using Spark, Snowpark, Pig , and Sqoop . Process large-scale datasets from varied sources using tools across the Hadoop ecosystem . Optimize data storage and retrieval in HBase , Hive , and other NoSQL stores. Collaborate closely with data scientists, analysts, and business stakeholders to enable data-driven decision-making. Ensure data quality, integrity, and compliance with enterprise security and governance standards. Tune and troubleshoot distributed data applications for performance and efficiency. Must-Have Skills: 5+ years in Data Engineering or Big Data roles Expertise in: Snowflake (Snowpark) Apache Spark MapReduce, Hadoop Sqoop, Pig, HBase Strong knowledge of: ETL/ELT pipeline design Distributed computing principles Big Data architecture & performance tuning Proven experience handling large-scale data ingestion , processing, and transformation Nice-to-Have Skills: Workflow orchestration with Apache Airflow or Oozie Cloud experience: AWS, Azure, or GCP Proficiency in Python or Scala Familiarity with CI/CD pipelines , Git, and DevOps environments Soft Skills: Strong problem-solving and analytical mindset Excellent communication and documentation abilities Ability to work independently and within cross-functional Agile teams