Get alerts for new jobs matching your selected skills, preferred locations, and experience range.
2.0 - 5.0 years
2 - 5 Lacs
Bengaluru / Bangalore, Karnataka, India
On-site
Key Responsibilities As a PySpark Data Engineer, you will: Design and develop scalable PySpark data pipelines to ensure efficient processing of large datasets, enabling faster insights and business decision-making. Leverage Databricks notebooks for collaborative data engineering and analytics, improving team productivity and reducing development cycle times. Write clean, modular, and reusable Python code to support data transformation and enrichment, ensuring maintainability and reducing technical debt. Implement data quality checks and validation logic within ETL workflows to ensure trusted data is delivered for downstream analytics and reporting. Optimize Spark jobs for performance and cost-efficiency by tuning partitions, caching strategies, and cluster configurations, resulting in reduced compute costs. Qualifications: Your Skills & Experience Solid understanding of Python programming fundamentals , especially in building modular, efficient, and testable code for data processing. Familiarity with libraries like pandas, NumPy, and SQLAlchemy (for lightweight transformations or metadata management). Proficient in writing and optimizing PySpark code for large-scale distributed data processing. Deep knowledge of Spark internals , partitioning, shuffling, lazy evaluation, and performance tuning. Comfortable using Databricks notebooks, clusters, and Delta Lake . Set Yourself Apart With Familiarity with cloud-native services like AWS S3, EMR, Glue, Lambda, or Azure Data Factory. Experience deploying or integrating pipelines within a cloud environment adds flexibility and scalability. Experience with tools like Great Expectations or custom-built validation logic to ensure data trustworthiness.
Posted 2 weeks ago
3.0 - 5.0 years
0 Lacs
Bengaluru / Bangalore, Karnataka, India
On-site
Big Data Service Team @ OCI OCI is leading the transformation to cloud-native Big Data technologies in our hyper-scale, multi-tenant cloud, deployed in more than 20 regions worldwide. OCI is committed to providing the best in cloud services that meet the needs of our customers, who are tackling some of the world's biggest challenges. The Big Data Service team's charter is to offer a managed, cloud-native Big Data Service focused on large-scale data processing and analytics on unstructured data stored in data lakes and managing the data in data lakes. The service work scope encompasses good integration with OCI's native infrastructure (security, cloud, storage, etc.) and deep integration with other relevant cloud-native services in OCI (like Oracle Kubernetes, Data Catalog, ADW, etc.). It includes doing cloud-native ways of doing service level patching & upgrades and maintaining high availability of the service in the face of random failures & planned downtimes in the underlying infrastructure (e.g., for things like patching the Linux kernels to take care of a security vulnerability). We are interested in senior engineers with expertise and passion for solving difficult problems in distributed systems and highly available services. At Oracle,youcan help, shape, design, and build innovative new systems from the ground up. These are exciting times in our space - we are growing fast, still at an early stage, and working on ambitious new initiatives. Engineers at any level can have significant technical and business impact. Desired Skills and Experience Deep understanding of how distributed resilient software is built and deployed. Prior experience in building, or contributing to, distributed data-intensive systems Experience delivering and operating large-scale, highly available distributed systems. Experience with larger projects (large codebases) Experience with open-source software in the Big Dataecosystem Experience at an organization with strong operational/dev-ops culture Expertise in coding in Java or Scala with emphasis on tuning/optimization Good software engineering skills: know how to write clean, testable, and maintainable code write documentation understanding of simple and robust designs, including designing APIs Bonus Deep understanding of Java and JVM mechanics Interested in speaking about their work, internally and externally, or writing articles BS in Computer Science or a related technical field or equivalent practical experience. Solid foundation in data structures, algorithms, and software design with strong analytical and debugging skills. Passion for learning and always improving yourself and the team around you Career Level - IC2 What to Expect from the Job Working ondistributed data-intensive systems, often as part of open-source communities. Taking ownership of critical parts of the cloud service, including shaping its direction. Code, integrate, andoperationalizeopen and closed source data ecosystem components for Oracle cloud service offerings Becoming an active member of the Apache open-source community when working on open-source components Optionally: Presenting work at conferences, meetups, or via articles. Working with, and supporting customers/users of the cloud service Design, develop, troubleshoot, and debug software programs for databases, applications, tools, networks, etc. Career Level - IC2
Posted 2 weeks ago
10 - 16 years
20 - 35 Lacs
Navi Mumbai
Work from Office
Role & responsibilities Job Description: Minimum 12 years experience in IT / IT Security / Information Security in Banking, Financial Services and Insurance (BFSI)/ Non-Banking Financial Company (NBFC)/ Payment industries/IT MNCs, out of which at least past 5 years should have been in Cloud deployments. Essential: Significant experience with deploying/ managing private and public cloud deployments, virtualized environment, and containerization platforms. Hands-on experience in deployment of cloud infrastructure and cloud security technology Should have experience with Azure / AWS / GCP/ VMware Cloud Foundation/ OCI Should have experience in Tanzu Kubernetes Grid/Openshift/OCI and Devsecops Experience in Inter-technology, Inter-OEM, inter-datacentre and inter-cloud integration from security perspective. Experience in Cloud Security Solutions Proficient with Cloud Security Solutions and Cloud Technologies including CASB/ CSPM/ CWPP/ CNAPP/ Micro segmentation/ Virtualization technologies/ containerization technologies. Thorough in concept of high availability Essential • Any one of OEM certification – (VMware/Azure/GCP/AWS/OCI etc.). •(CCSP/CCSK/GCSA/CompTIA Cloud+/ VCAP etc.). Preferred Additional technical certification like CISA/CISM/CISSP/ GSEC/ CompTIA Security+/ SSCP Preferred candidate profile Looking for Quick Joiners with Cloud Security background Perks and benefits
Posted 2 months ago
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
Accenture
36723 Jobs | Dublin
Wipro
11788 Jobs | Bengaluru
EY
8277 Jobs | London
IBM
6362 Jobs | Armonk
Amazon
6322 Jobs | Seattle,WA
Oracle
5543 Jobs | Redwood City
Capgemini
5131 Jobs | Paris,France
Uplers
4724 Jobs | Ahmedabad
Infosys
4329 Jobs | Bangalore,Karnataka
Accenture in India
4290 Jobs | Dublin 2