15947 Pyspark Jobs - Page 20

Summary Position Summary AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. TIn this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. The offering portfolio helps clients ...

Summary Position Summary AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. The offering portfolio helps clients transform their business by architecting organizational intelligence programs and differentiated strategies to win in their chosen markets. AI & Data will work with our clients to: Implement large-scale data ec...

Summary Position Summary AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. The offering portfolio helps clients transform their business by architecting organizational intelligence programs and differentiated strategies to win in their chosen markets. AI & Data will work with our clients to: Implement large-scale data ec...

Summary Position Summary AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. TIn this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. The offering portfolio helps clients ...

Summary Position Summary AI & Data In this age of disruption, organizations need to navigate the future with confidence, embracing decision making with clear, data-driven choices that deliver enterprise value in a dynamic business environment. The AI & Data team leverages the power of data, analytics, robotics, science and cognitive technologies to uncover hidden relationships from vast troves of data, generate insights, and inform decision-making. The offering portfolio helps clients transform their business by architecting organizational intelligence programs and differentiated strategies to win in their chosen markets. AI & Data will work with our clients to: Implement large-scale data ec...

Machine Learning Engineer Location: Bangalore Role & Responsibilities Design, implement, and deploy scalable ML models and pipelines for supervised & unsupervised problems, ensuring robustness and reproducibility. Develop production-quality Python services (APIs/batch jobs) to serve models with low latency and high throughput. Collaborate with data engineers and product owners to define features, data contracts, and ETL flows for reliable model inputs. Build and maintain CI/CD, monitoring, and automated retraining workflows (MLOps) to manage model lifecycle and drift. Optimize model performance and inference cost using quantization, batching, and scalable serving patterns (K8s/containers). D...

Machine Learning Engineer Location: Bangalore Role & Responsibilities Design, implement, and deploy scalable ML models and pipelines for supervised & unsupervised problems, ensuring robustness and reproducibility. Develop production-quality Python services (APIs/batch jobs) to serve models with low latency and high throughput. Collaborate with data engineers and product owners to define features, data contracts, and ETL flows for reliable model inputs. Build and maintain CI/CD, monitoring, and automated retraining workflows (MLOps) to manage model lifecycle and drift. Optimize model performance and inference cost using quantization, batching, and scalable serving patterns (K8s/containers). D...

We are looking for a skilled Data Engineer with 4 to 7 years of experience to join our team in Bangalore, Hyderabad, Chennai, and Pune. The ideal candidate will have expertise in Python, Pyspark, SQL, Hadoop, and NoSQL databases. Roles and Responsibility Design and develop scalable data pipelines using Azure ML, Azure Data Factory, and MLOps. Collaborate with cross-functional teams to integrate data from various sources into a unified platform. Develop and maintain large-scale data warehouses using Hadoop and NoSQL databases. Implement CI/CD pipelines for automated testing and deployment of data engineering projects. Troubleshoot and resolve complex technical issues related to data engineeri...

Title: Senior Generative AI Engineer (Databricks,Data Lake) Location: 100% Remote Job Type: Fulltime Key Responsibilities Design and implement GenAI models (LLMs, multimodal, embeddings, and fine-tuning) for enterprise use cases. Architect and optimize data pipelines and workflows in Databricks for large-scale AI training and inference. Manage and maintain data lakes on Databricks, ensuring data quality, governance, and scalability for AI workloads. Collaborate with data scientists, ML engineers, and product teams to translate business problems into AI-driven solutions. Implement MLOps pipelines for model deployment, monitoring, and lifecycle management in Databricks. Explore and evaluate st...

We are looking for a skilled Sr. Data Engineer with 12 to 17 years of experience to join our team in Bangalore, Hyderabad, Chennai, and Pune. Roles and Responsibility Design and develop scalable data pipelines using Public Cloud and Azure cloud technologies. Implement Snowflake ELT methodologies, including Snowpark, streams, and tasks. Develop and maintain ETL processes on Azure, ensuring data quality and integrity. Collaborate with cross-functional teams to design and implement data management solutions. Automate data processing tasks using Python, Scala, or other programming languages. Ensure compliance with industry standards and best practices for data engineering. Job Requirements Stron...

Job Summary: We are seeking a highly skilled Data Engineer with strong expertise in DBT, Java, Apache Airflow, and DAG (Directed Acyclic Graph) design to join our data platform team. You will be responsible for building robust data pipelines, designing and managing workflow DAGs, and ensuring scalable data transformations to support analytics and business intelligence. Key Responsibilities: Design, implement, and optimize ETL/ELT pipelines using DBT for data modeling and transformation. Develop backend components and data processing logic using Java. Build and maintain DAGs in Apache Airflow for orchestration and automation of data workflows. Ensure the reliability, scalability, and efficien...

We are looking for a skilled Pyspark Data Engineer with 3 to 5 years of experience. The ideal candidate will have expertise in Hadoop, Spark/Pyspark, Hive, YARN, Scala development, and data modeling. This position is based in Pune and Hyderabad. Roles and Responsibility Design and develop scalable data pipelines using Pyspark and Spark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems and architectures. Ensure data quality and integrity by implementing robust testing and validation procedures. Participate in code reviews and contribute to improving overall code quality. Troubleshoot and resolve complex tech...

We are looking for a skilled Azure Data Engineer with 7 to 10 years of experience. The ideal candidate will have expertise in Azure Data Analytics, Synapse, DevOps, and programming languages such as Python, SQL, Py-Spark, Spark-SQL, Azure: ADLS, Synapse, Stream Analytics, SQL DW, Analysis Services, Serverless Architecture, ARM Templates. Roles and Responsibility Design and develop scalable data pipelines using Azure Data Analytics and Synapse. Collaborate with cross-functional teams to implement DevOps practices and ensure smooth deployment of data engineering solutions. Develop and maintain large-scale data architectures using Azure services such as ADLS, Synapse, Stream Analytics, SQL DW, ...

We are looking for a skilled Data Engineer with expertise in Python/Pyspark to join our team at Krazy Mantra HR Solutions Pvt. Ltd, located in Bangalore. The ideal candidate will have 8+ years of experience in a Data Engineer role and be available to join immediately. Roles and Responsibility Design and develop data pipelines using Azure Data Factory, Stream Analytics, and Databricks. Collaborate with cross-functional teams to integrate data from various sources into a unified platform. Develop and maintain large-scale data warehouses using Azure Data Lake Storage and NoSQL databases. Implement serverless architecture using Azure Functions and ARM Templates. Optimize data processing workflow...

We are looking for a skilled professional with 5-10 years of experience to join our team as a Databricks expert in Pune, Chennai, Coimbatore, and Bangalore. Roles and Responsibility Design and develop data pipelines using Azure Data Bricks and Azure Data Factory. Collaborate with cross-functional teams to integrate data from various sources into a unified data warehouse. Develop and maintain ETL processes using PySpark and SQL. Implement data modeling principles to optimize data storage and retrieval. Troubleshoot and resolve issues related to data quality and performance. Ensure compliance with industry standards and best practices for data management. Job Requirements Strong experience in ...

We are looking for a skilled Pyspark Data Engineer with 5 to 7 years of experience, located in Pune and Hyderabad. The ideal candidate will have expertise in Pyspark, Scala, Hive, Airflow, Control M, CICD, Python, Hadoop, Data Modelling, and experience with Jira, Confluence, GitHub, or other similar Agile Scrum technologies. Roles and Responsibility Design and develop scalable data pipelines using Pyspark and Airflow. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems using Hadoop and Data Modelling techniques. Troubleshoot and resolve complex technical issues related to data processing and analysis. Implemen...

Data Engineer 7+ years of experience in cloud data engineering or related roles. Strong proficiency in AWS services including DMS, SCT, S3, Lambda, Glue, Redshift, RDS, EventBridge, API Gateway, IAM. Experience with Python, Spark/PySpark, and PostgreSQL. Hands-on experience with Terraform, Liquibase, and Bitbucket. Familiarity with ancillary AWS services like Secrets Manager, KMS, etc. Excellent problem-solving and communication skills. AWS Certification (e.g., Solutions Architect, Data Analytics) is a plus.Design and implement data migration strategies using AWS DMS and AWS SCT. Develop and maintain ETL pipelines using AWS Glue, Spark/PySpark, and Python. Architect and manage data storage s...

We are looking for a skilled ETL Testing professional with 6-12 years of experience to join our team in Bengaluru. The ideal candidate will have expertise in database testing using SQL and experience working with Databricks. Roles and Responsibility Develop and execute comprehensive test plans, test cases, and test scripts for ETL processes. Verify the accuracy and completeness of data, identifying key ETL mapping scenarios and creating SQL queries to simulate them. Perform data source location and format checks, conduct data counts, and ensure columns and data types meet requirements. Collaborate with cross-functional teams to identify and resolve data-related issues. Write complex SQL quer...

We are looking for a skilled professional with expertise in Databricks and Python to join our team. The ideal candidate will have 6+ years of experience in the field. Roles and Responsibility Design, develop, and implement data pipelines using Azure Data Factory and Azure Data Bricks Spark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems using SQL and Snowflake. Troubleshoot and resolve complex technical issues related to data processing and analysis. Ensure high-quality data delivery by implementing data validation and testing procedures. Participate in code reviews and contribute to improving overall cod...

We are looking for a highly skilled MLOps professional with 0 to 30 days of experience to join our team in Bangalore, Hyderabad, Pune, Chennai, and Kolkata. The ideal candidate will have expertise in MLOps, including Azure/AWS/GCP, Pyspark, and Bigdata. Roles and Responsibility Design and implement scalable data pipelines using MLOps tools. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale machine learning models and algorithms. Ensure seamless integration of MLOps solutions with existing systems and infrastructure. Troubleshoot and resolve complex technical issues related to MLOps. Stay up-to-date with industry trends a...

We are looking for a skilled PySpark Lead with 6-8 years of experience to join our team in Bangalore. The ideal candidate will have expertise in SQL, PySpark (or Spark with Scala), and Design Aspects - ETL/ Data Engineering Pipeline. Roles and Responsibility Design and develop scalable data pipelines using PySpark and other relevant technologies. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems, ensuring high performance and reliability. Troubleshoot and resolve complex technical issues related to data processing and analysis. Implement automated testing and validation frameworks to ensure data quality and ...

We are looking for a skilled professional with 5 to 10 years of experience in the field, based in Hyderabad. The ideal candidate should have a strong background in Computer Science or Information Technology and experience working with Palantir. Roles and Responsibility Develop and implement API's using Agile/DevOps methodologies. Design and develop web applications using HTML, CSS, JavaScript, and Bootstrap. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain relational databases using SQL. Implement automated testing and deployment scripts using Python and PySpark. Participate in code reviews and contribute to improving overall code ...

We are looking for a skilled professional with expertise in Azure data factory, Python, and Pyspark to join our team. The ideal candidate will have 5-10 years of experience. Roles and Responsibility Design, develop, and implement Azure data factory solutions using Python and Pyspark. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data pipelines and architectures. Troubleshoot and resolve complex technical issues related to Azure data factory. Ensure high-quality deliverables and meet project deadlines. Participate in code reviews and contribute to the improvement of the overall codebase. Job Requirements Strong profic...

We are looking for a skilled Data Engineer with 6 to 11 years of experience to join our team in Bangalore. The ideal candidate will have expertise in SQL, ETL, data modeling, and programming languages such as Python, C++, C#, or Scala. Roles and Responsibility Design and develop scalable data pipelines using Big Data/MPP analytics platforms. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale databases and data systems. Ensure data quality and integrity through data validation and testing procedures. Optimize data processing workflows for improved performance and efficiency. Troubleshoot and resolve technical issues relate...

We are looking for a skilled Data Engineer with AWS to join our team in Bangalore. The ideal candidate will have 4 to 6 years of experience and a strong background in data architecture, including data modeling, data mining, and data ingestion. Roles and Responsibility Design and implement data architectures, including data modeling, data mining, and data ingestion. Develop ETL scripts using languages such as Python, Node.js, SQL, and other databases. Integrate batch, near real-time, and real-time data from various sources. Collaborate with cross-functional teams to identify and prioritize project requirements. Ensure data quality and integrity by implementing data validation and testing proc...