23086 Pyspark Jobs - Page 31

Bachelor s or Master s degree in Computer Science, Software Engineering, or a related field 5+ yrs experience in data engineering roles 3+ years of experience in DWH/ETL Domain using AWS tech stack such as Redshift, S3, Glue, Lambda and IAM Proficiency in PySpark/Python/SQL and designing data models and data standards Deep understanding of ETL/ELT processes, data warehousing concepts, and big data technologies Experience building & managing data pipelines using tools like Glue, Airflow, AWS Step Functions, or similar Experience in RDMS & No-SQL databases: SQL Server databases, PostgreSQL, DynamoDB, or Aurora Experience in dimensional modeling & schema design; Star and Snowflake modelProven a...

Role: Azure Bricks Developer Job Description : The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ...

Role: Azure Bricks Developer Job Description : The Azure Data Bricks Engineer plays a critical role in establishing and maintaining an efficient data ecosystem within an organization. This position is integral to the development of data solutions leveraging the capabilities of Microsoft Azure Data Bricks. The engineer will work closely with data scientists and analytics teams to facilitate the transformation of raw data into actionable insights. With increasing reliance on big data technologies and cloud-based solutions, having an expert on board is vital for driving data-driven decision-making processes. The Azure Data Bricks Engineer will also be responsible for optimizing data workflows, ...

Key Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and Apache Spark. Build data ingestion, transformation, and processing frameworks for structured and unstructured data. Work with Azure services including ADLS Gen2, Azure Data Factory (ADF), Azure Synapse, Azure SQL, Delta Lake, and Event Hubs. Optimize and tune Spark jobs for performance, cost efficiency, and scalability. Implement advanced analytics and machine learning workflows using Databricks notebooks. Ensure data quality, data governance, and compliance throughout the data lifecycle. Collaborate with data architects, analysts, and business stakeholders to understand requirements and ...

Build relationships with senior stakeholders across the bank to seek out opportunities to add value through Data Science and GenAI. Develop machine learning models and apply AI to deliver commercial value for the Corporate Bank. Leverage AWS services for data transformation, feature engineering , AI product development and deployment. Communicate complex technical ideas to stakeholders via data visualisation Work closely with our equivalent ML Ops team in technology to streamline the path to production for successful projects. Essential Skills and Qualifications: Bachelor or masters degree in data science (or similar) or equivalent commercial experience Experience with working with senior st...

Lead the design, architecture, and implementation of GCP-based infrastructure projects Collaborate with IT, development, and operations teams to integrate cloud strategies and architecture Responsible for all technical aspects of software development for assigned applications - performs hands-on coding, technical design and development of systems. Develops code specifications. Involved in unit, integration and user acceptance testing. Assists in resolution of defects/ production support issues as required. Analyze requirements/ user stories to appropriately support basic design activities Perform core technical aspects of software development for assigned applications including, developing p...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Microsoft Azure Databricks Good to have skills : Python (Programming Language), PySpark, Microsoft Azure Data Services Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will i...

We are seeking an exceptional AI/ML Lead Architect with strong software engineering fundamentals, deep Python expertise, and hands-on experience building GenAI, LLM, and Agentic AI systems . This role requires an architect who can design, code, and deploy scalable AI systems from scratch not just prototype models. You will drive the end-to-end architecture of production-grade AI/ML and GenAI solutions , lead multi-agent workflows, and mentor engineering teams. Key Responsibilities 1. Architecture & System Design Architect end-to-end AI/ML, LLM, and Agentic AI systems including data, model, API, and deployment layers. Design scalable microservices-based AI architectures using Python, FastAPI,...

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : PySpark Good to have skills : Python (Programming Language) Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education , Summary: As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various stakeholders to ga...

Job Description As a Sr Data Science Supervisor here at Honeywell, you will lead a team of data scientists to drive innovative data solutions that enhance decision-making and operational efficiency across the organization. Your leadership will be crucial in fostering a culture of excellence and collaboration while overseeing the development and implementation of advanced analytics projects. You will report directly to our Sr. Data Engineering Manager, and you’ll work out of our Bengaluru, India location on a Hybrid work schedule. In this role, you will impact the organization by guiding your team in leveraging data science techniques to solve complex business challenges, optimize processes, ...

Our story At Alight, we believe a company’s success starts with its people. At our core, we Champion People, help our colleagues Grow with Purpose and true to our name we encourage colleagues to “Be Alight.” Our Values: Champion People – be empathetic and help create a place where everyone belongs. Grow with purpose – Be inspired by our higher calling of improving lives. Be Alight – act with integrity, be real and empower others. It’s why we’re so driven to connect passion with purpose. Alight helps clients gain a benefits advantage while building a healthy and financially secure workforce by unifying the benefits ecosystem across health, wealth, wellbeing, absence management and navigation....

Our story At Alight, we believe a company’s success starts with its people. At our core, we Champion People, help our colleagues Grow with Purpose and true to our name we encourage colleagues to “Be Alight.” Our Values: Champion People – be empathetic and help create a place where everyone belongs. Grow with purpose – Be inspired by our higher calling of improving lives. Be Alight – act with integrity, be real and empower others. It’s why we’re so driven to connect passion with purpose. Alight helps clients gain a benefits advantage while building a healthy and financially secure workforce by unifying the benefits ecosystem across health, wealth, wellbeing, absence management and navigation....

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Microsoft Azure Databricks Good to have skills : Python on Azure, Python (Programming Language), PySpark Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As a Data Engineer, your typical day involves designing, developing, and maintaining data solutions that facilitate data generation, collection, and processing. You will cr...

Project Role : Custom Software Engineer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : PySpark Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications function seamlessly to support business objectives. You will also engage in problem-solv...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Python (Programming Language) Good to have skills : Apache Spark, Google BigQuery, PySpark Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :Seeking a forward-thinking professional with an AI-first mindset to design, develop, and deploy enterprise-grade solutions using Generative and Agentic AI frameworks ...

Project Role : Custom Software Engineer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : PySpark Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications function seamlessly to support business objectives. You will also engage in problem-solv...

About The Role Project Role : BI Engineer Project Role Description : Develop, migrate, deploy, and maintain data-driven insights and knowledge engine interfaces that drive adoption and decision making. Integrate security and data privacy protection. Must have skills : PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a BI Engineer, you will develop, migrate, deploy, and maintain data-driven insights and knowledge engine interfaces that drive adoption and decision making. Your typical day will involve collaborating with various teams to ensure the integration of security and data privacy protection...

About The Role Project Role : BI Engineer Project Role Description : Develop, migrate, deploy, and maintain data-driven insights and knowledge engine interfaces that drive adoption and decision making. Integrate security and data privacy protection. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a BI Engineer, you will engage in the development, migration, deployment, and maintenance of data-driven insights and knowledge engine interfaces. Your typical day will involve collaborating with various teams to ensure that the solutions you create drive adoption and informed decisio...

About The Role Project Role : BI Engineer Project Role Description : Develop, migrate, deploy, and maintain data-driven insights and knowledge engine interfaces that drive adoption and decision making. Integrate security and data privacy protection. Must have skills : PySpark Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a BI Engineer, you will develop, migrate, deploy, and maintain data-driven insights and knowledge engine interfaces that drive adoption and decision making. Your typical day will involve collaborating with various teams to ensure the integration of security and data privacy protection...

About The Role Project Role : BI Engineer Project Role Description : Develop, migrate, deploy, and maintain data-driven insights and knowledge engine interfaces that drive adoption and decision making. Integrate security and data privacy protection. Must have skills : PySpark Good to have skills : NA Minimum 7.5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a BI Engineer, you will engage in the development, migration, deployment, and maintenance of data-driven insights and knowledge engine interfaces. Your typical day will involve collaborating with various teams to ensure that the solutions you create drive adoption and informed decis...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Microsoft Azure Databricks, Microsoft Azure Analytics Services, PySpark Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and pr...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Microsoft Azure Databricks, Microsoft Azure Analytics Services, PySpark Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and pr...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : Microsoft Azure Databricks, Microsoft Azure Analytics Services, PySpark Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and pr...

Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : PySpark Good to have skills : NA Minimum 5 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education , Summary: As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing applica...

About The Role Project Role : BI Engineer Project Role Description : Develop, migrate, deploy, and maintain data-driven insights and knowledge engine interfaces that drive adoption and decision making. Integrate security and data privacy protection. Must have skills : PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a BI Engineer, you will engage in the development, migration, deployment, and maintenance of data-driven insights and knowledge engine interfaces. Your typical day will involve collaborating with various teams to ensure that the solutions you create drive adoption and informed decisio...