15947 Pyspark Jobs - Page 28

Job Description Senior Manager, Lead Data Engineer The Opportunity Based in Hyderabad, join a global healthcare biopharma company and be part of a 130-year legacy of success backed by ethical integrity, forward momentum, and an inspiring mission to achieve new milestones in global healthcare. Be part of an organisation driven by digital technology and data-backed approaches that support a diversified portfolio of prescription medicines, vaccines, and animal health products. Drive innovation and execution excellence. Join a team that is passionate about using data, analytics, and insights to drive decision-making and create custom software, allowing us to tackle some of the world's greatest h...

About the Role: Grade Level (for internal use): 11 Position Summary The candidate will work along with other data scientists and data engineers to support activities related to development and maintenance of products that generate key insights on industry trend by applying various machine learning and statistical techniques on proprietary as well as open data sources. The person will work alongside Product managers, Industry Analysts and Technology experts to develop and deliver the product in accordance with customer requirements and agreed timeline. The candidate should have 10+ years work experience in developing products/services using Data Science and Analytics. Duties and responsibilit...

Greetings from TCS!! TCS is hiring for AWS Data engineer Role: AWS Data Engineer JOB LOCATION : Chennai, Pune EXPERIENCE REQUIREMENT : 4-8 years Required Technical Skill: Strong Knowledge of Aws Glue/AWS REDSHIFT/SQL/ETL. Good knowledge and experience in Pyspark for forming complex Transformation logic, SQL,ETL, DWH. Secondary : AWS Glue , Airflow Must-Have Good Knowledge of SQL , ETL A minimum of 3 + years' experience and understanding of Python core concepts, and implementing data pipeline frameworks using PySpark and AWS. Work well independently as well as within a team Good knowledge in working with various AWS services including S3, Glue, DMS, Redshift. Proactive, organized, excellent a...

Overview: Apply Data Warehouse and Cloud techniques to build data pipelines to provide data from different sources for our business, customer and Data Science team. This role offers many exciting challenges and opportunities: Emphasis is on managing end-to-end delivery of Data Contribute to how analytical approach is structured for specification of analysis Contribute insights from conclusions of analysis that integrate with initial hypothesis and business objective. Independently address complex problems Participate in design of analysis and modelling approach with managers and modelers This is a role that requires exceptional programming skills, proven technical leadership, ability to coll...

Project Role : Large Language Model Architect Project Role Description : Architect large language models (LLM) that can process and generate natural language. Design neural network parameters, trained on large quantities of unlabeled text data. Must have skills : Large Language Models Good to have skills : NA Minimum 12 year(s) of experience is required Educational Qualification : 15 years full time education SUMMARY: We are seeking a Senior Technical – GenAI Lead with deep expertise in Python, AI/ML concepts, and data engineering to spearhead the design and development of Data Agents powered by Agentic AI. As a senior technical leader, you will drive solution architecture, guide teams, and ...

About the Role: Grade Level (for internal use): 11 Position Summary The candidate will work along with other data scientists and data engineers to support activities related to development and maintenance of products that generate key insights on industry trend by applying various machine learning and statistical techniques on proprietary as well as open data sources. The person will work alongside Product managers, Industry Analysts and Technology experts to develop and deliver the product in accordance with customer requirements and agreed timeline. The candidate should have 10+ years work experience in developing products/services using Data Science and Analytics. Duties and responsibilit...

At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career. Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express. Join Our Team and let's lead the way together. Function Description: Launched in 2012, Amex Offers is a digital advertising platform that connects Merchants and br...

Role: AWS Data Engineer Location: Gurugram Mode: Hybrid Type: Permanent Job Description: We are seeking a talented and motivated Data Engineer with requisite years of hands-on experience to join our growing data team. The ideal candidate will have experience working with large datasets, building data pipelines, and utilizing AWS public cloud services to support the design, development, and maintenance of scalable data architectures. This is an excellent opportunity for individuals who are passionate about data engineering and cloud technologies and want to make an impact in a dynamic and innovative environment. Key Responsibilities: Data Pipeline Development: Design, develop, and optimize en...

Role Description Role Proficiency: Leverage expertise in a technology area (e.g. Infromatica Transformation Terradata data warehouse Hadoop Analytics) Responsible for Architecture for a small/mid-size projects. Outcomes Implement either data extract and transformation a data warehouse (ETL Data Extracts Data Load Logic Mapping Work Flows stored procedures data warehouse) data analysis solution data reporting solutions or cloud data tools in any one of the cloud providers(AWS/AZURE/GCP) Understand business workflows and related data flows. Develop design for data acquisitions and data transformation or data modelling; applying business intelligence on data or design data fetching and dashboar...

Role Description Role Proficiency: Lead solution/application architecture for medium to large engagement. Outcomes Deliver technically sound projects across one / multiple customers within the guidelines of the customer and UST standards and norms Deliver technically complex applications Identify and institutionalize best practices across multiple accounts and manage multiple customers and architects Define architecture for large engagements and act as its design authority Design solutions involving multiple tech components after scoping out requirements for a client with large engagements Measures Of Outcomes Business Development (number of proposals contributed to; number new leads generat...

Join us as a Software Engineer, PySpark And AWS This is an opportunity for a driven Software Engineer to take on an exciting new career challenge Day-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutions Its a chance to hone your existing technical skills and advance your career while building a wide network of stakeholders We're offering this role at associate level What you'll do In your new role, youll be working within a feature team to engineer software, scripts and tools, as well as liaising with other engineers, architects and business analysts across the platform. Youll also be: Producing complex and critical softwar...

TITLE: Lead Engineer Key Skills: SQL, Python, Pyspark, Azure, Data Bricks, Team Leading Work Schedule: Full-time, on-site in Gurgaon, 5 days a week. Job Summary: We are seeking a skilled and experienced Azure Data Engineer to join our team. The ideal candidate will have hands-on experience in Azure Data Engineering, strong expertise in Databricks, PySpark, and SQL, along with proven leadership capabilities managing data engineering teams. This role requires excellent communication skills and the ability to work collaboratively in an on-site environment. Key Responsibilities: Develop and maintain Azure Data Engineering solutions to support business requirements. Lead and mentor a team of 5+ d...

We are looking for a motivated and enthusiastic Trainee Data Engineer to join our Engineering team. This is an excellent opportunity for recent graduates to start their career in data engineering, work with modern technologies, and learn from experienced professionals. The candidate should be eager to learn, curious about data, and willing to contribute to building scalable and reliable data systems. Responsibilities: Understand and align with the values and vision of the organization. Adhere to all company policies and procedures. Support in developing and maintaining data pipelines under supervision. Assist in handling data ingestion, processing, and storage tasks. Learn and contribute to ...

Responsibilities Conducting research aligned to the roadmaps to de-risk/ mature and insert advanced AI concepts to Honeywell products (latest advancements in generative AI / Causal / Knowledge Graphs, machine learning, and deep learning techniques ) Researching, designing, implementing, scalable AI based solutions to solve high value & challenging customer problems and drive opportunities for new growth Leads prototyping (what–ifs) supporting pilot programs for R&D purposes. Collaborating with cross-functional teams to develop data-driven solutions. Responsible for problem analysis, stakeholder interaction, solution design, front end and back-end integration, maintenance, and support of data...

Responsibilities Conducting research aligned to the roadmaps to de-risk/ mature and insert advanced AI concepts to Honeywell products (latest advancements in generative AI / Causal / Knowledge Graphs, machine learning, and deep learning techniques ) Researching, designing, implementing, scalable AI based solutions to solve high value & challenging customer problems and drive opportunities for new growth Leads prototyping (what–ifs) supporting pilot programs for R&D purposes. Collaborating with cross-functional teams to develop data-driven solutions. Responsible for problem analysis, stakeholder interaction, solution design, front end and back-end integration, maintenance, and support of data...

Marsh McLennan is seeking candidates for the following position based in the Pune/Mumbai office. SeniorPrincipal Engineer/Manager – Data Engineer What can you expect? As a Data Engineer, you will be responsible for designing and implementing scalable data pipelines using Databricks and the Medallion Architecture. You will handle end-to-end ETL/ELT processes, manage large datasets, and work with tools like Python, PySpark, and AWS S3 to ensure data is transformed and optimized for analytical use. Additionally, knowledge on Informatica, Unix, and Oracle database is also a good to have. What is in it for you? Holidays (As Per the location) Medical & Insurance benefits (As Per the location) Shar...

Marsh McLennan is seeking candidates for the following position based in the Pune/Mumbai office. SeniorPrincipal Engineer/Manager – Data Engineer What can you expect? As a Data Engineer, you will be responsible for designing and implementing scalable data pipelines using Databricks and the Medallion Architecture. You will handle end-to-end ETL/ELT processes, manage large datasets, and work with tools like Python, PySpark, and AWS S3 to ensure data is transformed and optimized for analytical use. Additionally, knowledge on Informatica, Unix, and Oracle database is also a good to have. What is in it for you? Holidays (As Per the location) Medical & Insurance benefits (As Per the location) Shar...

What you'll do In your new role, youll engineer and maintain innovative, customer centric, high performance, secure and robust solutions. Youll be working within a feature team and using your extensive experience to engineer software, scripts and tools that are often complex, as well as liaising with other engineers, architects and business analysts across the platform. Youll also be: Producing complex and critical software rapidly and of high quality which adds value to the business Working in permanent teams who are responsible for the full life cycle, from initial development, through enhancement and maintenance to replacement or decommissioning Collaborating to optimise our software engi...

Join us as a Quality Automation Specialist In this key role, youll be undertaking and enabling automated testing activities in all delivery models Well look to you to support teams to develop quality solutions and enable continuous integration and assurance of defect free deployment of customer value Working in a fast paced environment, you'll benefit from great variety in your work as well as excellent exposure as you work closely with a variety of people across the bank We're offering this role at vice president level What you'll do As a Quality Automation Specialist, youll be contributing to the transformation of testing using quality processes, tools and methodologies, significantly impr...

Role: Senior Data Engineer Location: Mumbai & Pune Experience: 3yrs to 8yrs Technologies / Skills: Advanced SQL, Python and associated libraries like Pandas, NumPy etc., Pyspark , Shell scripting, Data-Modelling, Big data, Hadoop, Hive, ETL pipelines. Responsibilities: • Proven success in communicating with users, other technical teams, and senior management to collect requirements, describe data modeling decisions and develop data engineering strategy. • Ability to work with business owners to define key business requirements and convert to user stories with required technical specifications. • Communicate results and business impacts of insight initiatives to key stakeholders to collaborat...

Hi, 𝐆𝐫𝐞𝐚𝐭 𝐎𝐩𝐩𝐨𝐫𝐭𝐮𝐧𝐢𝐭𝐲 𝐟𝐨𝐫 𝐃𝐚𝐭𝐚 𝐒𝐜𝐢𝐞𝐧𝐜𝐞 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐬❗ We’re hiring Data Science Engineers (5+ yrs) at Abbacus Technologies. Key Responsibilities: • Decode legacy data flows and design modern ELT patterns. • Develop scalable data models using Domain-Driven Design. • Build efficient ELT pipelines with Snowflake & Matillion. • Ensure high data quality and governance standards. • Lead technical best practices in SQL, Python/PySpark, and CI/CD. Requirements : • Strong experience in data modeling (Star/Snowflake, DDD, Data Vault). • Expertise in SQL and cloud warehouses (Snowflake, Redshift, BigQuery). • Proficiency in Python (PySpark is a plus). • Background in complex industries like telecom or heal...

Job Title: Lead Data Engineer Location: Mumbai / Pune Experience: 8+ yrs Job Summary: We are seeking a technically strong and delivery-focused Lead Engineer to support and enhance enterprise-grade data and application products under the Durables model. The ideal candidate will act as the primary technical interface for the client, ensuring high system availability, performance, and continuous improvement. This role requires a hands-on technologist with strong team management experience, cloud (AWS) expertise, and excellent communication skills to handle client interactions and drive technical decisions. Key Responsibilities: Support & Enhancement Leadership Act as the primary technical lead ...

Key Responsibilities Design, develop, and maintain scalable and efficient data pipelines for batch and real-time processing. Work extensively with Databricks and PySpark to process and transform large datasets. Write complex and optimized SQL queries for data extraction, transformation, and analysis. Develop robust ETL/ELT processes to integrate data from various structured and unstructured sources. Collaborate with data scientists, analysts, and business stakeholders to ensure reliable and accessible data. Implement best practices for data quality, governance, and security on the AWS cloud platform. Monitor and troubleshoot data workflows to ensure high availability and performance. Contrib...

Role : AWS Data Engineer Type : 6-12 month contract Location : Chennai, Tamil Nadu (WFO) Experience : 6-8 Years Requirements: Proficiency in Python and PySpark: Strong experience writing robust Python code and utilizing PySpark for distributed data processing, transformation, and working with large datasets. AWS Data Engineering Expertise: Hands-on experience with AWS Glue, Lambda, S3, Athena, DynamoDB, and Step Functions to build and manage data workflows. SQL Mastery: Deep knowledge of SQL for complex query writing, database design, and performance optimization. Terraform Experience: Familiarity with Terraform for managing AWS infrastructure as code is essential. Cloud Data Architecture: E...

Responsibilities: * Design, develop, and maintain data pipelines using Python, Pandas, Numpy, Django, Power BI/Tableau. * Optimize database performance with PostgreSQL, PySpark, Snowflake. Work from home