Posted:2 days ago|
Platform:
On-site
Full Time
• Design, develop, and maintain data pipelines using PySpark and related Big Data technologies.
• Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem.
• Write efficient HiveQL and SQL queries to handle large-scale datasets.
• Perform performance tuning and optimization of distributed data systems.
• Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions. • Manage and schedule workflows using Apache Airflow or Oozie.
• Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy.
• Proven experience in Big Data Engineering with a focus on PySpark.
• Strong knowledge of HDFS, Hive, Sqoop, and related tools.
• Proficiency in SQL/HiveQL for large datasets.
• Expertise in performance tuning and optimization of distributed systems.
• Familiarity with Agile methodology and collaborative team practices.
• Experience with workflow orchestration tools (Airflow/Oozie).
• Strong problem-solving, analytical, and communication skills.
• Knowledge of data modeling and data warehousing concepts.
• Exposure to DevOps practices and CI/CD pipelines for data engineering.
• Experience with other Big Data frameworks such as Spark Streaming or Kafka.
Atomic North
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
pune, maharashtra, india
Salary: Not disclosed
pune, maharashtra, india
Salary: Not disclosed
3.0 - 6.0 Lacs P.A.
Chennai, Tamil Nadu, India
Salary: Not disclosed
Pune, Maharashtra, India
Experience: Not specified
Salary: Not disclosed
Bengaluru, Karnataka, India
Salary: Not disclosed
pune, maharashtra, india
Salary: Not disclosed
pune, maharashtra, india
Salary: Not disclosed
3.0 - 6.0 Lacs P.A.
Chennai, Tamil Nadu, India
Salary: Not disclosed