Jobs

Interviews
Job Alerts
Tools

Upskill and Grow with AI

Mock Interview Practice interviews in realistic simulations

Coding Practice Improve your coding skills with challenges

Certification Earn certifications to validate your skills

AI Learning Get trained with AI expert sessions

Career Path AI insights for smarter career decisions

AI Job Match Score AI-Powered Job Match Against Your Resume and Optimize Your Resume

Career Tools and Resources

Resume Builder Build Professional Resume with Ease

ATS Friendliness Check Check Resume Friendliness for Applicant Tracking Systems

Auto Apply Apply to hundreds of jobs on any platform effortlessly

Co-Pilot (Chrome Extension) Your AI Assistant for Seamless Browsing Efficiency

Interview Questions Streamline interviews with ready-to-use questions

Salaries Discover market-driven salary insights across skillsets and geographies

Companies Explore leading companies actively hiring talent
For Employers

Jobs

Interviews

Home
>
Jobs in Delhi
>
Unison Consulting
>
Data Platform Engineer

Data Platform Engineer

Unison Consulting

0 years

0 Lacs

Delhi India

Posted:1 month ago| Platform:

Apply

Skills Required

data spark deployment apache kubernetes scalability reliability monitoring optimization resolve management storage efficiency security authentication authorization encryption backup strategies integrity documentation troubleshooting collaboration devops containerization docker architecture tuning processing analytics communication orchestration helm sql mllib azure certification code terraform scripting python automation

Work Mode

On-site

Job Type

Contractual

Job Description

Spark Cluster Deployment: Deploy, configure, and maintain Apache Spark clusters on Kubernetes, ensuring scalability, reliability, and performance. Application Deployment: Collaborate with data engineers and data scientists to deploy Spark applications and workloads, ensuring they run efficiently. Monitoring and Optimization: Implement monitoring solutions to track cluster performance, resource utilization, and application health. Proactively identify and resolve performance bottlenecks. Resource Management: Manage cluster resources, including CPU, memory, and storage allocation, to ensure optimal utilization and cost efficiency. Security: Implement and maintain security measures, including authentication, authorization, and encryption, to protect sensitive data and Spark clusters. Backup and Recovery: Develop and maintain backup and recovery strategies to ensure data integrity and availability in case of failures. Documentation: Maintain clear and comprehensive documentation of Spark cluster configurations, deployment procedures, and best practices. Troubleshooting: Quickly diagnose and resolve issues related to Spark clusters, applications, and Kubernetes infrastructure. Collaboration: Work closely with cross-functional teams, including data engineers, data scientists, and DevOps, to understand application requirements and optimize Spark clusters accordingly. Requirements Proven experience deploying and managing Apache Spark on Kubernetes in a production environment. Proficiency in containerization technologies, particularly Docker and Kubernetes. Strong knowledge of Spark architecture, including cluster, driver, and worker nodes. Familiarity with Spark tuning, optimization, and performance monitoring. Experience with resource management tools like Kubernetes Resource Quotas and LimitRanges. Understanding of data processing and analytics workflows. Excellent problem-solving and troubleshooting skills. Strong communication and collaboration skills. Experience with Spark cluster orchestration tools like Helm. Knowledge of Spark ecosystem components such as Spark SQL, Spark Streaming, and MLlib. Familiarity with cloud-based solution (Azure). Certification in Kubernetes (e.g., Certified Kubernetes Administrator - CKA). Knowledge of CI/CD pipelines and infrastructure as code (IaC) tools (e.g., Terraform). Scripting skills in languages like Python, Bash, or Shell. Understanding of DevOps practices and automation. Show more Show less

More Jobs at Unison Consulting

Data Engineer

Mumbai Metropolitan Region

2 - 5 yrs

Salary: Not disclosed

IT Recruiter (Middle East Market - Remote)

India

2.0 - 6.0 yrs

Salary: Not disclosed

Temenos Java Extensibility Developer

Hyderabad, Telangana, India

4.0 - 4.0 yrs

Salary: Not disclosed

Temenos Java Extensibility Developer

Hyderabad

4.0 - 9.0 yrs

INR 8 - 12 Lacs

Analytics Data Engineer (Infrastructure as Code)

Mumbai Metropolitan Region

Experience: Not specified

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start DevOps Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

Unison Consulting

Login to

Please Verify Your Phone or Email

Confirm Action

Search

Profile

Upskill and Grow with AI

Data Platform Engineer