28509 Pyspark Jobs - Page 7

Role Overview: You will be working as a Senior Azure Data Engineer in an IT services and enterprise data engineering company that delivers cloud-native analytics, ETL/ELT pipelines, and modern data platform implementations on Microsoft Azure for large enterprise customers. This role is on-site and hands-on, supporting strategic data modernization initiatives across business domains. Key Responsibilities: - Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks for ingesting, cleansing, and transforming large datasets. - Develop PySpark and Python-based data processing jobs, optimizing Spark workloads for throughput, latency, and cost on Databrick...

Role Overview: As a Data Engineer at the company, you will play a crucial role in designing and implementing processes to extract insights from data. Your responsibilities will include building and deploying infrastructure for data ingestion, developing data-related code in an automated environment, generating reports for stakeholders, and researching solutions to data-related problems both individually and collaboratively. Key Responsibilities: - Build and deploy infrastructure for ingesting high-volume support data from consumer interactions, devices, and apps - Enable data-driven change by creating effective visualizations and reports for stakeholders - Develop and maintain data-related c...

Role Overview: In this role, you will coordinate the understanding and integration of various datasets and design analytical solutions to support critical business needs, including business opportunity assessments, customer segmentation and targeting, HCP and patient opportunity identification, patient and prospective analytics, smart business experiments (test and learn), market simulations, and patient flow modeling. You will work closely with cross-functional stakeholders including Marketing, Medical, Market Access, Sales, Forecasting, and other Commercial Operations teams. Key Responsibilities: - Coordinate the understanding and integration of various datasets - Design analytical solutio...

Location: Mumbai/ Pune/ Bangalore/ Noida/ Kochi Job Profile Summary: In this role, you will support the Data Engineering team in setting up the Data Lake on Cloud and the implementation of standardized Data Model. You will develop data pipelines for new sources, data transformations within the Data Lake, CI/CD and data delivery as per the business requirements. Job Description: • Build pipelines to bring in wide variety of data from multiple sources within the organization as well as from social media and public data sources. • Collaborate with cross functional teams to source data and make it available for downstream consumption. • Work with the team to provide an effective solution design ...

As a Big Data Engineer at this company, your role involves designing, developing, and maintaining scalable and distributed data architectures capable of processing large volumes of data. Your key responsibilities include: - Designing Big Data Architecture: Develop scalable and distributed data architectures for processing large volumes of data effectively. - Implementing Data Storage Solutions: Utilize technologies such as Hadoop, Spark, and PySpark to implement and optimize data storage solutions. - Developing PySpark ETL Processes: Create and implement efficient ETL processes using PySpark to extract, transform, and load large datasets. - Optimizing Performance: Optimize PySpark applicatio...

Role Overview: You will be responsible for driving strategic initiatives, managing business transformations, and leveraging industry expertise to create value-driven solutions in merchandising analytics. Key Responsibilities: - Provide strategic advisory services and conduct market research to develop data-driven recommendations for enhancing business performance. - Possess relevant experience in the required domain with strong analytical, problem-solving, and communication skills. - Work in a fast-paced, dynamic environment and have knowledge of SQL, R & Python language, at least one cloud-based technology (Azure, AWS, GCP), price/discount elasticity models, non-linear optimization, state s...

You will be joining a worldwide network of smart and driven colleagues experienced in leading statistical tools, methods, and applications as part of the Data & AI practice. From data to analytics and insights to actions, you will provide analytically informed, issue-based insights at scale to help clients improve outcomes and achieve high performance. **Roles & Responsibilities:** - Work through the phases of the project. - Define data requirements for Data-Driven Growth Analytics capability. - Clean, aggregate, analyze, interpret data, and carry out data quality analysis. - Have knowledge of market sizing and lift ratios estimation. - Experience in working with non-linear optimization tech...

Role & responsibilities Senior Databricks Data Engineer About the Role Were looking for a Senior Databricks Data Engineer who enjoys building reliable, scalable data pipelines. You’ll work on everything from ingestion to transformation, helping shape our data processing patterns and ensuring the platform is stable, performant, and easy to operate. You should be comfortable working with PySpark, Spark SQL, Delta Lake, and Databricks features. This role is ideal for someone who loves solving data problems and wants to help design reusable patterns that accelerate onboarding for many teams. What You’ll Do Build and maintain ETL/ELT pipelines across Databricks (Bronze Silver Gold). Implement reu...

About the Role: We are looking for an experienced Data Engineer who will be responsible for designing, building, and maintaining scalable data pipelines and solutions on Microsoft Fabric platforms. The ideal candidate will have solid programming skills, strong knowledge of data processing frameworks, and hands-on capability in optimizing data workflows. Key Responsibilities: Design and implement end-to-end Microsoft Fabric solutions, including data integration, transformation, and analytics. Proficiency in Microsoft Fabrics Notebook for developing Apache Spark jobs. Hands-on experience with Python, PySpark and SparkSQL/TSQL for data transformations. Experience working with File Transfer Serv...

Role & responsibilities : We are seeking a skilled and experienced Data Testing Engineer to join our dynamic team. In this role, you will be responsible for Design and execute comprehensive test plans for ETL processes, ensuring data integrity and accuracy throughout the data lifecycle. Perform functional, integration, and regression testing on ETL workflows and data pipelines. Databricks Advanced SQL queries SOAP, SOA, API testing Analytics and BI Dashboard Testing Cloud platforms (AWS, Azure, GCP) and data testing in cloud Experience with data automation tools like Informatica (PL-SQL), Databricks (Pyspark) Rest APIs & Graphql Git and GItHub and Reporting tools Linux or Windows server Pref...

Dear Candidates, We are hiring Data Architect for our Top IT MNC Client. Job Summary: We are seeking a highly skilled and hands-on Data Architect to join our team. The ideal candidate will have deep expertise in building scalable, high-performance data platforms using modern data architecture principles like Lakehouse and Medallion architecture. You will be responsible for designing and implementing robust ELT pipelines, leveraging metadata-driven approaches, and ensuring adherence to architectural best practices including non-functional requirements, exception handling, and Azure's Well-Architected Framework (WAF). Key Responsibilities: Design and implement ELT pipelines using modern framew...

Job Description 9+ years of overall experience with considerable experience in data engineering Mandatory Skills: 1. Very strong in SQL, PySpark. 2. Good in Databricks, DLT, Medallion Architecture, Unity Catalog, Governance. 3. Good knowledge and experience in Azure Data Services (like Azure Data Factory, Azure Blob, Azure Data Lake, Key Vault, etc.). 4. Excellent written & spoken communication skills, ability to interact with customer & relevant stakeholders. 5. Ability to understand the provided requirements, ability to assist in architecting solutions and ability to translate it into the implementation, following best practices. 6. Proactive, Self-driven, motivated, result oriented, abili...

Job Description Roles & responsibilities Role Overview The Manager – Azure Data Engineer will lead a large team of data engineers , driving strategic initiatives in the Audit Data & Analytics domain. This role demands strong leadership qualities, deep technical expertise in Azure, and ERP knowledge. The Manager will be responsible for delivering complex data solutions, ensuring operational excellence, and fostering a culture of collaboration and innovation. Key Responsibilities Leadership & Team Management Lead and manage a team of 10–16 data engineers, ensuring high performance and engagement. Set clear goals, monitor progress, and provide coaching and mentorship to team members. Foster an ...

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Microsoft Azure Analytics Services Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education , Roles & Responsibilities: -Develop and Optimize Data Pipelines using PySpark and SQL in Databricks to handle large-scale data transformations and batch/streaming workloads. -Design and Implement ETL Processes by leverag...

Preferred candidate profile . Experience using Databricks and proficiency in Microsoft Azure platform (incl. ADF and ADLS). • Experience querying databases and strong programming skills: PySpark, Python, SQL, etc. • Strong knowledge of data concepts and tools. • Ability to be an effective team member with remote colleagues. • Strong communication skills in English language (both written and spoken).

Project Role :Custom Software Engineer Project Role Description :Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills :Microsoft Azure Data Services Good to have skills :NA Minimum 3 year(s) of experience is required Educational Qualification :15 years full time education Summary:As a Software Development Engineer, you will engage in a dynamic work environment where you will analyze, design, code, and test various components of application code across multiple clients. Your day will involve collabor...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing...

Your role and responsibilities As an Application Developer, you will lead IBM into the future by translating system requirements into the design and development of customized systems in an agile environment. The success of IBM is in your hands as you transform vital business needs into code and drive innovation. Your work will power IBM and its clients globally, collaborating and integrating code into enterprise systems. You will have access to the latest education, tools and technology, and a limitless career path with the worlds technology leader. Come to IBM and make a global impact Responsibilities: Responsible to manage end to end feature development and resolve challenges faced in impl...

Job Description Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client qu...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

About the team: Being one of the largest payment platforms in the country, we aim to not only ensure a seamless experience, but also a delightful journey for our customers. We strive to enrich the customer experience by making every transaction transparent, honest, and hassle-free. About the Role: Business analyst focuses on data, statistical analysis and reporting to help investigate and analyze business performance, provide insights, and drive recommendations to improve performance. Expectations/ Requirements: 1. Derive business insights from data with a focus on driving business-level metrics. 2. Ability to interact and convince business stakeholders. 3. Developing insightful analysis abo...

Key Responsibilities: Solution Design & Architecture: Lead the design and architecture of complex software and data engineering solutions, ensuring they are scalable, reliable, and secure. Build and maintain end-to-end data pipelines for efficient data ingestion, processing, and transformation in cloud environments (primarily Google Cloud ). Design and implement cloud-based data solutions using Cloud Run , Cloud Functions , Big Query, Datastore , GCS , Dataplex , Pub/Sub, Cloud Scheduler and Composer . Ensure data quality, consistency, and integrity across data sources and throughout the ETL pipeline. Implement data governance best practices, ensuring compliance, security, and data privacy i...

We are looking for highly skilled Data Engineers to join our team for a long-term offshore position. The ideal candidates will have 5+ years of experience in Data Engineering, with a strong focus on Python and programming. The role requires proficiency in leveraging AWS services to build efficient, cost-effective datasets that support Business Reporting and AI/ML Exploration. Candidates must demonstrate the ability to functionally understand the Client Requirements and deliver Optimized Datasets for multiple Downstream Applications. The selected individuals will work under the guidance of an Lead from Onsite and closely with Client Stakeholders to meet business objectives. Key Responsibiliti...

Immediate Opening!! Skills: AWS, Lambda, Spark, Pyspark,Hive,Python Exp: 4+ years Location: Bangalore, Hyderabad, Pune Notice Period: Immediate-30 days Mode: contract

Job description: About us: At Data Unveil, we believe in delivering the best for our clients (Pharma Companies). We use the latest technology and tools to aggregate and analyze specialty healthcare data received from various data partners. We provide clear and hassle-free business insights to enhance the client’s vision and drive business success. Position Title: Python Developer Experience: 3 Years Location: Hyderabad, Telangana Hire Type : Full Time, On-site Start Date: Immediate Job Summary: We are seeking a Python Developer with 3 years of experience, skilled in Python, PySpark, Oracle, MySQL, and basic AWS services. The candidate will work on data processing, ETL pipelines, and database...