15947 Pyspark Jobs - Page 22

experience in big data development and data engineering. - Proficiency in Java and experience with Apache Spark. - Experience in API development and integration. - Strong understanding of data engineering principles and big data concepts. Required Candidate profile - Familiarity with big data tools such as Hadoop, HDFS, Hive, HBase, and Kafka. - Experience with SQL and NoSQL databases. - Strong communication and collaboration skills

A Candidate has Hands-on exp. working with tech. like Hadoop, Hive, Pig, Oozie, Map Reduce, Spark, Sqoop, Kafka, Flume. Also Know. of selected programming languages e.g. Python, Pyspark .In-depth know. of relational database e.g. PostgreSQL, MySQL . Required Candidate profile A Candidate has Must Hands-on exp. working with tech. like Hadoop, Hive, Pig, Oozie, Map Reduce, Spark, Sqoop, Kafka, Flume. Also Know. Python, Pyspark ,PostgreSQL, MySQL .

Job Title : Azure + Microsoft Fabric + PySpark Experience: 4 to 8 years Notice Period: Immediate joiners to 30 days, Serving notice period Job Description (JD) Azure Data Engineering role focused on Microsoft Fabric (good to have), Azure Databricks with PySpark (must have) capabilities and strong emphasis on Python, SQL, Data Lake, and Data Warehouse: Job Title: Data Engineer – Microsoft Fabric, Azure (Databricks & ADF), PySpark Experience: 4–8 Years Location: Kolkata / Bangalore Employment Type: Full-Time Job Summary: We are looking for a skilled and experienced Data Engineer with 5-8 years of experience in building scalable data solutions on the Microsoft Azure ecosystem. The ideal candida...

Job Description NIQ is looking for a Senior Data Engineer to join our Financial Services Engineering team. At NIQ, the Financial Services team uses alternative datasets to help global public equity investors (hedge funds, mutual funds, pension funds) make better investment decisions. We work with some of the largest hedge funds in the world. As a Senior Data Engineer, you will be at the cutting edge of the alternative data space where you will help maintain and improve our data infrastructure, which enables us to develop market research products and delivery data to our customers. In this role, you would also get the opportunity to work with world-class big data and cloud services, such as: ...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : PySpark Good to have skills : Python (Programming Language) Minimum 7.5 year of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project milestones are met, facilitating discussions to address challenges, and guiding your team through the development process...

About The Role Entity:- Accenture Strategy & Consulting Team:- Strategy & Consulting Global Network Practice:- Talent & organization - Human Capital Analytics Title:- Ind & Func AI Decision Science Consultant Level 9 Job location:- Gurgaon/Bengaluru / Mumbai About Strategy & Consulting Global Network:- Accenture S&C Global Network - Data & AI practice help our clients grow their business in entirely new ways. Analytics enables our clients to achieve high performance through insights from data - insights that inform better decisions and strengthen customer relationships. From strategy to execution, Accenture works with organizations to develop analytic capabilities - from accessing and report...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : PySpark Good to have skills : NA Minimum 5 year of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the solutions align with business objectives. You will also engage in problem-solving discussion...

Role & responsibilities Design and develop RESTful APIs using Python frameworks Implement and manage API proxies, policies, and security configurations using Apigee Hybrid . Collaborate with cross-functional teams to integrate APIs with frontend applications and third-party services. Write optimized SQL queries for data retrieval, transformation, and reporting. Ensure API performance, scalability, and reliability through monitoring and tuning. Maintain documentation for APIs, including specifications, usage guidelines, and versioning. Troubleshoot and resolve issues related to API functionality, performance, and security. Participate in code reviews and contribute to best practices in API de...

About The Role Job Title - S&C Global Network - AI - T&O - Human Capital Analytics Analyst Management Level:11 -Analyst Location:Bengaluru, Gurugram, Hyderabad Must-have skills: Human Capital Analytics ( Data science, ML, Python, Data visualization, Power BI / Tableau, Gen AI, Azure /AWS/ GCP ) Good to have skills: Ability to leverage design thinking, business process optimization, and stakeholder management skills. Job Summary : This role involves driving strategic initiatives, managing business transformations, and leveraging industry expertise to create value-driven solutions. Roles & Responsibilities: Provide strategic advisory services, conduct market research, and develop data-driven r...

About The Role Project Role : Enterprise Technology Architect Project Role Description : Architect complex end-to-end IT solutions across the enterprise. Apply the latest technology and industry expertise to create better products and experiences. Must have skills : Data Consulting Good to have skills : NA Minimum 7.5 year of experience is required Educational Qualification : 15 years full time education Summary :Deliver AI/ML/GenAI solutions in engineering programs with focus on design automation and PLM data.Roles & Responsibilities:Build ML/GenAI solutions for CAD/PLM auto-classification.Implement copilots inside PLM environments.Automate defect/rework prediction.Collaborate with architec...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : MongoDB Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing...

About The Role Project Role : Large Language Model Architect Project Role Description : Architect large language models (LLM) that can process and generate natural language. Design neural network parameters, trained on large quantities of unlabeled text data. Must have skills : Large Language Models Good to have skills : NA Minimum 12 year of experience is required Educational Qualification : 15 years full time education SUMMARY:We are seeking a Senior Technical GenAI Lead with deep expertise in Python, AI/ML concepts, and data engineering to spearhead the design and development of Data Agents powered by Agentic AI. As a senior technical leader, you will drive solution architecture, guide te...

We are seeking a Lead Data Engineer with 7-10 years of experience to join our Data Platform team. This role will report to the Manager of data engineering and be involved in the planning, design, and implementation of our centralized data warehouse solution for ETL, reporting and analytics across all applications within the company. Qualifications: Deep knowledge and experience working with Python/Scala and PySpark Experienced in Azure Data bricks, Azure Blob Storage, Azure Data Lake, Delta lake. Experience working with Spark SQL Experience in creating pipelines and databricks dashboards Experience with Azure cloud environments Experience with acquiring and preparing data from primary and se...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Data Engineering Good to have skills : NA Minimum 5 year of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implem...

Title: Senior Generative AI Engineer (Databricks,Data Lake) Location: 100% Remote Job Type: Fulltime Key Responsibilities • Design and implement GenAI models (LLMs, multimodal, embeddings, and fine-tuning) for enterprise use cases. • Architect and optimize data pipelines and workflows in Databricks for large-scale AI training and inference. • Manage and maintain data lakes on Databricks, ensuring data quality, governance, and scalability for AI workloads. • Collaborate with data scientists, ML engineers, and product teams to translate business problems into AI-driven solutions. • Implement MLOps pipelines for model deployment, monitoring, and lifecycle management in Databricks. • Explore and...

Primary Responsibilities: Collect, clean, and analyze data from various sources Design and maintain data pipelines to ensure data quality and integrity Develop and maintain reports and dashboards in excel and Power BI Utilize SQL and Python and PySpark for data manipulation and analysis Identify trends, patterns, and anomalies in data sets Collaborate with cross-functional teams to understand business requirements and translate them into data solutions Communicate findings and insights to stakeholders through reports and presentations Develop and implement predictive models using machine learning techniques Comply with the terms and conditions of the employment contract, company policies and...

What Youll Do : Take primary ownership in driving both self and team efforts across all phases of the project lifecycle, ensuring alignment with business objectives. Translate business requirements into technical specifications and lead team efforts to design, build, and manage technology solutions that effectively address business problems. Develop and apply advanced statistical models and leverage analytic techniques to utilize data for guiding decision-making for clients and internal teams. Apply appropriate development methodologies (e.g., agile, waterfall) and best practices (e.g., mid-development client reviews, embedded QA procedures, unit testing) to ensure successful and timely proj...

Must-Have Skills: Python (Pandas, PySpark) for data manipulation and pipeline development Strong data engineering skills & workflow optimization Experience with Delta Tables and Parquet file formats Good-to-Have Skills: Experience with Databricks Knowledge of Apache Spark, DBT, Airflow Advanced Pandas optimizations for performance Familiarity with PyTest/DBT testing frameworks

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities An AI Data Scientist at IBM is not just a job title - it’s a mindset. You’ll leverage the watsonx,AWS Sagemaker,Azure Open AI platform to co-create AI value with clients, focusing on technology patterns to enhance repeatability and delight clients. We are seeking an experienced and innovative...

Supply Chain & Manufacturing Team at ZS The Supply Chain & Manufacturing team at ZS is dedicated to optimizing and transforming supply chain operations for our clients across various industries. Out team offers a comprehensive range of services from stragegy and advanced analytics to planning and technology, aimed at optimizing resource deployment while maintaining high customer service levels. We are committed to driving innvotation and efficiency through cutting-edge solutions that address the complex challenges of supply chain and manufacturing operations. What youll do : We are looking for experienced Knowledge Graph developers who have the following set of technical skillsets and experi...

Job Description: We are looking for a skilled Data Engineer with hands-on experience in building scalable data pipelines and analytics solutions. The ideal candidate should have strong expertise in Python, PySpark, SQL, and Databricks. Required Skills: Strong programming skills in Python and PySpark. Hands-on experience with Databricks platform. Excellent knowledge of SQL and query optimization techniques. Experience in ETL development and data pipeline design. Good understanding of data warehousing and data lake concepts. Nice to Have: Experience with cloud platforms (Azure, AWS, or GCP). Familiarity with CI/CD and version control tools.

Locations: Pune, Maharashtra, India Minimum Experience: 8 Maximum Experience: 14 ctc 6-8 -22 lac 8+ -30 lac max Skill Set: Databricks Engineer, Azure Databricks, SQL, Pyspark,Team Handling Skill to Evaluate: Databricks Engineer, Azure Databricks, SQL,Pyspark, Data warehousing, ADF, Team Handling Experience: 8 to 14 Years Location: Pune, Maharashtra, India Job Description: Job Title: Senior Data Engineer Key Responsibilities: - Design, build, and maintain scalable data pipelines and ETL processes. - Collaborate with data scientists and analysts to understand data needs and provide support for their analytics requirements. - Develop and implement data models, database designs, and data managem...

Locations: Pune, Maharashtra, India Minimum Experience: 8 Maximum Experience: 12 Skill to Evaluate: azure/aws,databricks,SQL,PYSPARK, Team Handling Job Title: Technical Lead Company Name: NCSI This role is accountable for running day to day operations of Data Platform in Azure / AWS Databricks NCS. Data Engineer is accountable for ongoing Development, Enhancement support and maintenance data availability and data quality, performance enhancement and stability of the system. 1. Designing and implementing data ingestion pipelines from multiple sources using Azure Databricks 2. Ensure data pipelines run smoothly and efficiently 3. Adherence to security, regulatory and audit control guidelines 4...

Job Title: Lead Data Engineer AWS | Python | Spark | Databricks Location: Pune (Hybrid/Onsite) Experience: 9 – 15 Years Notice Period: Immediate to 15 Days (preferred) CTC: Up to 36 LPA About the Role We’re urgently hiring a Senior Data Engineer to join our high-performing engineering team. You will design, automate, and optimize data pipelines within the AWS ecosystem. This is a critical, fast-moving role with plenty of opportunity to influence architecture and strategy. Key Responsibilities Design, build, and maintain robust ETL/ELT processes using Python, SQL, AWS Glue, Redshift . Automate data flows and integrations using AWS Lambda and other serverless services. Develop scalable and sec...

Data Analyst Location: Bangalore / Pune About the Role: We are looking for an experienced Data Analyst with strong technical expertise and a solution-oriented mindset. The ideal candidate will leverage advanced skills in Python, PySpark, and SQL to manage and transform data, uncover insights, and support complex business use cases. Experience in the banking sector and knowledge of data governance principles will be a strong advantage. Key Responsibilities: Analyze and transform large datasets using PySpark and SQL to solve complex business problems. Identify data anomalies and perform thorough data validation to ensure accuracy and quality. Collaborate effectively with stakeholders, managing...