23086 Pyspark Jobs - Page 38

Design, develop, and maintain scalable and secure data pipelines using Microsoft Fabric, Azure Synapse Analytics, and Azure Databricks to support critical business processes. Develop curated datasets for clinical, regulatory, and commercial analytics using SQL and PySpark. Create and support dashboards and reports using Power BI (highly preferred). Collaborate with cross-functional stakeholders to understand data needs and translate them into technical solutions. Work closely with ERP teams such as Salesforce.com and SAP S/4HANA to integrate and transform business-critical data into analytic-ready formats. Partner with Data Scientists to enable advanced analytics and machine learning initiat...

Experience At least 5 years of experience in AWS based projects. Technical skills Proficiency in Python and PySpark for data engineering tasks. Big Data Strong knowledge of Big Data technologies and data warehousing concepts. AWS services Experience with AWS Data Engineering stack, including S3, RDS, Athena, Glue, Lambda, and Step Functions. SQL Strong SQL skills for data manipulation and querying. CI CD Experience with CI CD tools like Terraform and Git Actions. Soft skills Good communication skills and ability to work in a multicultural team.

Job Title: Data & MLOps Engineer Python, PySpark, ML Location: [Insert Location] Experience: 5to 8years Employment Type: C2H ( Contract to Hire ) Industry: IT Services / Analytics / AI Functional Area: Data Engineering / Machine Learning Role Category: Data Science & Engineering Role: Data & MLOps Engineer Job Summary We are hiring a Data & MLOps Engineer to design, build, and maintain scalable data pipelines and ML Ops systems. This role involves conducting experiments, performing statistical analysis and EDA, and supporting model training, retraining, and deployment. You’ll help productionize and manage AI/ML models across cloud and on-prem environments. Mandatory Skills Programming : Pyth...

Model Development Individual Contributor - Work independently with team lead(s) Hands-on programming and development Data Analysis Problem Solving - work toward getting the optimal solution of a given problem Tuning & Optimization Conduct risk based tuning of AML scenarios and maintain standards for BTL (Below the Line) ATL (Above the Line) alert testing Support ongoing optimization to the effectiveness of all in scope BAU transaction monitoring systems, scenarios and rules Model Testing & UAT Perform UAT and systems integraton testing of AML model for Significant experience and direct knowledge of AML Transaction Monitoring systems and preferably Oracle Mantas, Prime BSA and Actimize is a b...

Experience with Snowflake and its utilities i. e . SnowSQL, SnowCD etc. Experience with Data Lake, Data Warehouse and Data Sharing concepts. SQL and Unix Shell Scripting Skilled in testing of cloud-based database solutions. Experience in working with Azure Data Engineering project. Strong Development and administration skills for all DB maintenance plan activities including backups, indexing, integrity, and consistency checks, etc. Experience in troubleshooting and resolving database problems performance tuning and optimization. Proactively identify, design, and implement process improvements like automating manual processes, optimizing data flow, increasing scalability etc. knowledge in Sno...

Design and develop robust ETL pipelines using Python, PySpark, and GCP services. Build and optimize data models and queries in BigQuery for analytics and reporting. Ingest, transform, and load structured and semi-structured data from various sources. Collaborate with data analysts, scientists, and business teams to understand data requirements. Ensure data quality, integrity, and security across cloud-based data platforms. Monitor and troubleshoot data workflows and performance issues. Automate data validation and transformation processes using scripting and orchestration tools. Required Skills & Qualifications Hands-on experience with Google Cloud Platform (GCP), especially BigQuery. Strong...

At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions. At PwC - AC, we are looking for an Azure Senior Manager- Data & AI who will have: Passion for technology, in the space...

Summary Position Summary National Consulting Services The National Office, a key Strategic Enabler in our new model, is comprised of professionals with various areas of focus that collectively provide mission critical services to support Deloitte’s overall success. Whether focused on project financial, quality and risk management, methods and tools, sales excellence, talent, leadership support, or other activities these professionals ensure that our Firm operates efficiently, and our people are able to effectively serve clients every day. Position Summary Level : Consultant At Deloitte Consulting LLP, Data Scientists and ML Engineers leverage advanced analytics and machine learning to transf...

Summary Position Summary National Consulting Services The National Office, a key Strategic Enabler in our new model, is comprised of professionals with various areas of focus that collectively provide mission critical services to support Deloitte’s overall success. Whether focused on project financial, quality and risk management, methods and tools, sales excellence, talent, leadership support, or other activities these professionals ensure that our Firm operates efficiently, and our people are able to effectively serve clients every day. Position Summary Level : Consultant At Deloitte Consulting LLP, Data Scientists and ML Engineers leverage advanced analytics and machine learning to transf...

Overview Join the Prodapt team in supporting a unified, scalable, and secure Jupyter-based environment for data science and machine learning. You will help build, maintain, and optimize the platform that empowers analysts, engineers, and scientists to explore data, develop models, and collaborate at scale. Responsibilities Develop, maintain, and enhance the Notebook platform built on JupyterLab, supporting both cloud and on-premises deployments. Integrate and optimize connections to diverse data sources (BigQuery, Hive, Teradata, Hadoop, Spark, etc.). Enable and support distributed data processing and analytics using Spark/PySpark and Dataproc. Implement and maintain platform features for co...

What you'll do As a Data Engineer, youll be looking to simplify our organisation by developing innovative data driven solutions through data pipelines, modelling and ETL design, inspiring to be commercially successful while keeping our customers, and the banks data, safe and secure. Youll drive customer value by understanding complex business problems and requirements to correctly apply the most appropriate and reusable tool to gather and build data solutions. Youll support our strategic direction by engaging with the data engineering community to deliver opportunities, along with carrying out complex data engineering tasks to build a scalable data architecture. Your responsibilities will al...

Role & responsibilities Experience Professional:- Must-Have Skills: AWS Core Services: Amazon S3: Proficiency in using S3 for data storage, organization, and retrieval. AWS Lambda: Experience with serverless computing for data processing and automation. Amazon RDS/Aurora: Understanding of relational databases and their use in AWS environments. Amazon Glue: Familiarity with AWS Glue for ETL (Extract, Transform, Load) operations, data cataloging, and schema management. Data Modeling: Solid understanding of dimensional and relational data modeling, star and snowflake schemas, and normalization/denormalization techniques. ETL Development: Proficiency in designing and building ETL pipelines, both...

Summary Position Summary National Consulting Services The National Office, a key Strategic Enabler in our new model, is comprised of professionals with various areas of focus that collectively provide mission critical services to support Deloitte’s overall success. Whether focused on project financial, quality and risk management, methods and tools, sales excellence, talent, leadership support, or other activities these professionals ensure that our Firm operates efficiently, and our people are able to effectively serve clients every day. Position Summary Level: Senior Consultant or equivalent As a Senior Consultant at Deloitte Consulting, you will design, develop, and deploy enterprise-scal...

Summary Position Summary National Consulting Services The National Office, a key Strategic Enabler in our new model, is comprised of professionals with various areas of focus that collectively provide mission critical services to support Deloitte’s overall success. Whether focused on project financial, quality and risk management, methods and tools, sales excellence, talent, leadership support, or other activities these professionals ensure that our Firm operates efficiently, and our people are able to effectively serve clients every day. Position Summary Level : Consultant At Deloitte Consulting LLP, Data Scientists and ML Engineers leverage advanced analytics and machine learning to transf...

Job Description Data Migration Engineer About Oracle FSGIU - Finergy: The Finergy division within Oracle FSGIU is dedicated to the Banking, Financial Services, and Insurance (BFSI) sector. We offer deep industry knowledge and expertise to address the complex financial needs of our clients. With proven methodologies that accelerate deployment and personalization tools that create loyal customers, Finergy has established itself as a leading provider of end-to-end banking solutions. Our single platform for a wide range of banking services enhances operational efficiency, and our expert consulting services ensure technology aligns with our clients' business goals. Job Summary: We are seeking a s...

We are seeking a skilled GenAI App Developer with expertise in API development , backend logic , machine learning , and NLP to contribute to large-scale GenAI applications . You'll work on API integrations , system performance optimization, and developing multi-agent workflows, all within a dynamic, collaborative environment. Required Skills: Programming Languages: Strong proficiency in Python, with experience in SQL. Machine Learning: Hands-on experience with Scikit-learn, TensorFlow, Keras, PyTorch, or similar ML libraries. Data Analysis: Strong skills in Pandas, NumPy, and Matplotlib for data manipulation and analysis. Statistical Analysis: Experience applying statistical methods to data,...

About the Role: We are seeking a Senior Data Engineer to drive the development of our next-generation Data Lakehouse architecture. You will be responsible for designing, building, and optimizing massive-scale, low-latency data pipelines that support real-time analytics and Machine Learning applications. Key Responsibilities: - Design and build highly optimized, production-grade ETL/ELT pipelines using Apache Spark (PySpark/Scala) to process petabytes of data. - Architect and manage the Data Lakehouse using open-source technologies like Delta Lake or Apache Hudi for ACID transactions and data quality. - Integrate and process real-time data streams using technologies such as Apache Kafka or Ki...

Summary Position Summary Data Analyst (Con) At Deloitte, we do not offer you just a job, but a career in the highly sought-after Risk Management field. We are the business leader in the risk market. We work with a vision to make the world more prosperous, trustworthy, and safe. Our clients, primarily based outside of India, are large, complex organizations that constantly evolve and innovate to build better products and services. In the process, they encounter various risks and the work we do to help them address these risks is increasingly important to their success—and to the strength of the economy and public security. By joining us, you will get to work with diverse teams of professional...

Summary Position Summary Data Analyst (Con) At Deloitte, we do not offer you just a job, but a career in the highly sought-after Risk Management field. We are the business leader in the risk market. We work with a vision to make the world more prosperous, trustworthy, and safe. Our clients, primarily based outside of India, are large, complex organizations that constantly evolve and innovate to build better products and services. In the process, they encounter various risks and the work we do to help them address these risks is increasingly important to their success—and to the strength of the economy and public security. By joining us, you will get to work with diverse teams of professional...

Job description Role :Python Data Engineer Experience: 5 years to 8 years Location: Hyderabad Only Mandatory skills: Python, Data Pipeline, Pyspark, Pandas, Numpy Job Description: Strong Python and SQL knowledge 6 years hands on experience as a data engineer data software architect or advanced BI analyst Expert level competency in Big Data manipulation and transformation both within and outside of a database Strong work ethic collaboration and communication skills 4 Years of experience with written and verbal communication with stakeholders in past roles Some familiarity with cloud architecture 2 years of experience within AWS Azure or Google cloud environments Nice to Have Skills Preferred ...

Join us as a Software Engineer, PySpark This is an opportunity for a technically minded individual to join us as a Software Engineer You’ll be designing, producing, testing and implementing working software, working across the lifecycle of the system Hone your existing software engineering skills and advance your career in this critical role We're offering this role at vice president level What you'll do Working in a permanent feature team, you’ll be developing knowledge of aspects of the associated platform across the disciplines of business, applications, data and infrastructure. You’ll also be liaising with principal engineers, architects in the domain and other key stakeholders to unders...

AI/ML Engineer Location: Bangalore EXP: 5 to 8Yrs Mandatory skills: Python, Pyspark, SQL, Traditional ML, MLOPS Job Summary: The primary purpose of this role is to design, build and maintain the data pipelines & ML ops systems, conducting experiments & performing statistical analysis, EDA & bringing insights aiding fine-tune models, training & retraining ML models as needed that enable the organization to process large volumes of data & also productionize, scale & manage AI/ML models. The main goal includes support to optimizing model training and inference. This role will harness AI/ML platform capabilities and help accelerate the development of Data & AI/ML driven products. Roles & Respons...

Job Description: Responsibilities: Develop & Optimize Data Pipelines Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark). Optimize performance and cost-efficiency of Spark jobs. Ensure data quality through validation, monitoring, and alerting mechanisms. Understand cluster types, configuration, and use-case for serverless Implement Unity Catalog for Data Governance Design and enforce access control policies using Unity Catalog. Manage data lineage, auditing, and metadata governance. Enable secure data sharing across teams and external stakeholders. Integrate with Cloud Data Platforms Work with Azure Data Lake Storage / Azure Blob Storage/ Azure Ev...

Role & responsibilities Position Description : • Databricks Developer to design, build, and optimize scalable data pipelines and analytics solutions. • This role will work with large healthcare datasets to ensure secure, compliant, and efficient data processing that supports business insights and innovation. • Working experience with Agile methodology • Good communication and team player. Required Skills : • 5 - 7 years of experience with Databricks, PySpark, Python, and SQL • Strong knowledge of Delta Lake and modern data lakehouse architectures • Hands-on experience with AWS (Glue, Redshift, S3) • Expertise in ETL development, data modeling, and performance tuning • Familiarity with health...

Join us as a Software Engineer, PySpark This is an opportunity for a driven Software Engineer to take on an exciting new career challenge Day-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutions It’s a chance to hone your existing technical skills and advance your career while building a wide network of stakeholders We're offering this role at associate level What you'll do In your new role, you’ll be working within a feature team to engineer software, scripts and tools, as well as liaising with other engineers, architects and business analysts across the platform. You’ll Also Be Producing complex and critical software rapi...