Posted:21 hours ago| Platform: Naukri logo

Apply

Work Mode

Work from Office

Job Type

Full Time

Job Description

DETAILED JOB DESCRIPTION

Roles & Responsibilities


  • 17 to 22 years of experience in cloud-based systems and data platforms management with 8 to 10 years of experience as a Data Architect.

  • Design, develop, and implement robust and scalable data architectures on Azure, considering performance, cost-efficiency, and security.

  • Identify performance bottlenecks, analyse system behaviour under load, and develop strategies to improve efficiency and scalability.

  • Provide guidance on system design and architecture to optimize performance, including database configurations, caching mechanisms, and code optimization.

  • Develop and maintain scalable and efficient streaming frameworks.

  • Integrate Salesforce solutions with other system components to meet business requirements.

  • Utilize scripting languages such as Python for automation and data processing tasks.

  • Collaborate with cross-functional teams, including development, operations, and security, to ensure effective cloud architecture and integration.

  • Manage a team of 6 to 8 members with below skills.


Detailed Job Description

MUST HAVE

Azure Cloud Architecture (Expert Level - Required)

Advanced PySpark (Expert Level - Required)

- Mastery of DataFrame APIs and Spark SQL

- Performance tuning: broadcast joins, caching strategies, shuffle reduction

Databricks Expertise (Expert Level - Required)

- Cluster sizing, autoscaling, and spot instance optimization

- Job orchestration using Workflows with retries, alerts, and dependencies

PostgreSQL (Required)

- Query profiling and execution plan analysis

- Indexing strategies for analytical workloads

Performance & Scalability (Expert Level - Required)

- Designing horizontally scalable pipelines

- Spark memory management: executor sizing, GC tuning

GOOD TO HAVE

Kafka (Nice to have)

- Designing high-throughput Kafka consumers with Spark Structured Streaming

- Managing consumer groups and offset strategies

Structured Streaming in Spark (Nice to have)

- Stateful aggregations and windowed operations

- Exactly-once semantics with checkpointing and write-ahead logs

Machine Learning & MLOps (Nice to have)

- MLflow integration for model tracking and deployment

- Deploying Spark ML pipelines

Node.js - (Nice to have)

- Building REST APIs for data access and control

- Integrating with PostgreSQL

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Han Digital Solution logo
Han Digital Solution

Information Technology

Metro City

RecommendedJobs for You

hubli, mangaluru, mysuru, bengaluru, belgaum