23086 Pyspark Jobs - Page 19

Description : We are seeking a talented Data Engineer to join our team and play a pivotal role in transforming raw data into valuable insights. As a Data Engineer, you will design, develop, and maintain robust data pipelines and infrastructure to support our organization's analytics and decision-making processes. Responsibilities : - Data Pipeline Development : Build and maintain scalable data pipelines to extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files) into data warehouses or data lakes. - Data Infrastructure : Design, implement, and manage data infrastructure components, including data warehouses, data lakes, and data marts. - Data Quality : Ensu...

Seeking a Senior Data Engineer (6+ yrs) skilled in Azure Data Factory , ETL/ELT, Python/PySpark, SQL, Azure DevOps, and Terraform . Must have strong data migration and pipeline development experience, plus solid data warehousing and modelling Provident fund

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Development Lead (Full Stack Experience) Work across investment technology stacks, engage directly with global stakeholders across investment, operations, products and wider technology teams and help drive technology solutions that support portfolio management and trading Drive full stack application development using modern technologies across cloud-native and hybrid architectures Own the production environment operations and applications Required 7- 10 years of ...

Strong in Pyspark, Python, PLSQL, SQL Desirable to have ETL with batch and streaming (Kinesis). Build the solution for optimal extraction, transformation, and loading of data from a wide variety of data sources using data ingestion and transformation components. The following technology skills are required Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases. Experience with Dataflow Hands-on experience in using Databricks Hands-on experience with at least one of the leading public cloud data platforms (Amazon Web Services and Google Cloud) Experience with column-oriented database tec...

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. EY-Strategy and Transactions - SaT– DnA Senior Analyst EY’s Data n’ Analytics team is a multi-disciplinary technology team delivering client projects and solutions across Data Management, Visualization, Business Analytics and Automation. The assignments cover a wide range of countries and industry sectors. The opportunity We’re looking for Se...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Position EY’s GDS Tax Technology team’s mission is to develop, implement and integrate technology solutions that better serve our clients and engagement teams. As a member of EY’s core Tax practice, you’ll develop a deep tax technical knowledge and outstanding database, data analytics and programming skills. Ever-increasing regulations require tax departments to gather, organize and analyse more data than ever before. Often the data necessary to satisfy these ever...

Role Overview We are looking for a highly skilled Data Engineer to join our data engineering team. The ideal candidate will have hands-on experience in building and maintaining data pipelines using Azure Databricks , Azure Data Factory , PySpark , and Microsoft Fabric (OneLake, Lakehouse, Dataflows, etc.). You will be responsible for designing robust data architectures, optimizing data workflows, and ensuring data quality and governance. Key Responsibilities Design, build, and maintain scalable data pipelines using Azure Data Factory (ADF) and Microsoft Fabric Pipelines / Dataflows . Develop data transformations using PySpark / Spark SQL in Azure Databricks or Microsoft Fabric Notebooks. Bui...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Position EY’s GDS Tax Technology team’s mission is to develop, implement and integrate technology solutions that better serve our clients and engagement teams. As a member of EY’s core Tax practice, you’ll develop a deep tax technical knowledge and outstanding database, data analytics and programming skills. Ever-increasing regulations require tax departments to gather, organize and analyse more data than ever before. Often the data necessary to satisfy these ever...

Position: Azure Data Engineer Location: Bangalore About Tenjumps At Tenjumps, we specialise in consulting, building, deploying, and managing complex enterprise applications and infrastructure. Our tailored solutions help technology-driven businesses build robust, high-performing, and scalable systems that evolve with their growth. Weve partnered with a leading logistics domain client to modernise their application platforms and unlock new opportunities through AI/ML-powered solutions. We are looking for a Senior Azure Data Engineer with strong expertise in building scalable, cloud-native data pipelines on Azure Databricks, ADF, and PySpark . The ideal candidate will lead end-to-end data engi...

Role & responsibilities Preferred candidate profile

Desired Competency Professional hand-on experience in Python, Pyspark, Java. Professional hand-on experience in Sql and Query Optimization. Experience in programming design patterns. Experience in system design. Should have Experience with CI/CD tool like Jenkins etc. Should have Experience with Orchestration tools like Airflow/ Automic etc. Should have hand-on experience in Data processing and Data manipulation skills like Data warehousing concepts, SCD types etc. Mandatory - one cloud exposure (GCP is preferred to adapt to a fast-paced and continuously evolving environment). Good-to-Have Experience in Apache Spark and Distributed computing frameworks. Professional hand-on experience in Pyt...

As an Application Developer, you will lead IBM into the future by translating system requirements into the design and development of customized systems in an agile environment. The success of IBM is in your hands as you transform vital business needs into code and drive innovation. Your work will power IBM and its clients globally, collaborating and integrating code into enterprise systems. You will have access to the latest education, tools and technology, and a limitless career path with the world’s technology leader. Come to IBM and make a global impact Responsibilities: Responsible to manage end to end feature development and resolve challenges faced in implementing the same Learn new te...

Python with AWS*****Virtual Drive ***19-Nov-25 TCS Hiring !!Virtual Drive *** 19-Nov-25 TCS - Hyderabad ( 2 PM to 5 PM Role** Python with AWS ***** Please read Job description before Applying NOTE: If the skills/profile matches and interested, please reply to this email by attaching your latest updated CV and with below few details: Name: Contact Number: Email ID: Highest Qualification in: (Eg. B.Tech/B.E./M.Tech/MCA/M.Sc./MS/BCA/B.Sc./Etc.) Current Organization Name: Total IT Experience : 3-8 years only Location : Hyderabad Current CTC Expected CTC Notice period: Immediate Whether worked with TCS - Y/N Development: Python(preferred) or Pyspark or Scala & with Spark Framework Cloud: AWS Serv...

Loc: Manyata Tech Park Exp: 2-5 yrs, AI/ML Engineer Skills Required: ? Exp. with Data engineering & Data/ML pipelines for Analytics & AI/ML Models ? Working with large sets of structured & unstructured data from disparate sources ? Foundational acquaintance of Data Science & AI/ML models ? SQL, Python, Spark, PySpark ? Big Data systems - Hadoop Ecosystem (HDFS, Hive, MapReduce) ? Analytics database like Druid, Data visualization/exploration tools like Superset ? Understanding of cloud platforms (AWS, Azure, or GCP) ? GIT Secondary Skills (desired/preferred): ? Apache Airflow, Apache Oozie, Nifi ? GCP cloud exp, Big Query, GCP Big Data Ecosystem ? Trino/Presto ? Familiar with forecasting algo...

Data Engineering Lead- Azure (Data Brick) Company Name: Blackstraw.ai Job Type: Full-time Experience: 10 - 15 Years Candidates who can join immediately will be preferred. Job Description: As a lead data engineer you will oversee data architecture, ETL processes, and analytics pipelines, ensuring efficiency, scalability, and quality. Key Responsibilities: Job Requirements: Participate in the customers system design meetings and collect the functional/technical requirements. Build up data pipelines for consumption by the data science team. Skillful in ETL process and tools. Clear understanding and experience with Azure Data Factory, Python and PySpark or Spark, with HIVE, Airflow, Impala, and ...

Role: Application Architect - Data Governance Duration:3-6 Months Contract to Hire Location: Pune, Maharashtra 411014, India Baseline Qualifications and Experience: This is a technical role with a strong business acumen requirement; hence it is expected the Integration Architect continuously invests in learning and stays on par with the latest in Data Governance, Analytics, and Data Science approaches, solutions, frameworks, technology offerings, and best practices. Education: Undergraduate Degree in one or a combination of the following: Engineering, Information Technology, Information Systems, or Business Analytics. Master's Degree in any of the earlier disciplines preferred. Certification...

Duties & Responsibilities Translate business requirements into scalable and well-documented AI/ML pipelines using Databricks, Azure AI, and Snowflake. Design and deploy GenAI-powered applications using pretrained models (e.g., GPT-4, LLaMA) for tasks such as text generation, summarization, code synthesis, and personalization. Build and orchestrate agentic AI workflows using frameworks like LangChain, OpenAI Agents SDK, and Databricks Genie, enabling autonomous task execution and multi-agent collaboration. Integrate GenAI models into enterprise applications with Human-in-the-Loop (HITL) validation and feedback loops for production readiness. Conduct model selection, fine-tuning, and evaluatio...

Desired Competency Min 8+ years experience in BigData, Spark and Scala Experience in Hadoop, Pyspark, Hive, Sqoop, Oozie Must have expertise in the use of Good Cloud and SQL/HQL Strong skills in In-memory applications, Database Design Experience writing shell scripts using (ksh, bash, perl or python). Experience in Google Cloud Bigquery, Dataproc and Dataflow Good-to-Have Unix, Shell Scripting, knowledge of commercial databases, Kafka Ability to perform data related benchmarking, performance analysis and tuning Solid understanding of Data Warehousing Concepts. Role & responsibilities Good communications skills to clarify requirements or explain technical designs to non- technical business st...

Job Title: Data Engineer Work Location: - BANGALORE Duration of Assignment: 18+ Months ( Contract to Hire) Work mode: Onsite, 5days working from Client Location Experience- 7 to 10years Immediate Joiners or serving Notice Period till November PF is mandatory Skill Required : Digital: Microsoft Azure, Digital: DevOps, Azure Data Factory Role Description - Azure devops Engineer, Azure data Engineer -Azure devops Engineer, Azure data Engineer agile way of working -Agile way of working

Greetings from TCS! We are looking for - Data Engineer(Python, Databricks, SQL) Experience: 6-8 years Location: Hyderabad Desired Competencies- Create and maintain optimal data pipeline architecture, Assemble large, complex data sets that meet functional / non-functional business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal ingestion, transformation, and Publishing of data from a wide variety of data sources using Python/Spark and AWS ‘big data’ technologies. Build analytics tools that utilize the data p...

Description : We are a technology consulting firm operating in Cloud Data Engineering and Analytics, helping enterprise customers build reliable, scalable data platforms and analytics products. Our teams deliver end-to-end data lakes, real-time streaming pipelines, and production-grade ML feature stores using Databricks and modern cloud data tooling. Role & Responsibilities : - Design, build, and maintain scalable batch and streaming ETL pipelines on Databricks using Delta Lake and Delta Live Tables (DLT). - Develop and optimize Spark/PySpark jobs for performance, cost-efficiency, and reliability; tune cluster sizing and autoscaling policies. - Implement data quality, observability, lineage ...

About The Role Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Microsoft Azure Analytics Services, Microsoft SQL Server, Microsoft Azure Databricks, PySpark, Azure Data Factory, Databricks REST API Good to have skills : NA Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :About The Role :Azure Data Engineer – ADF, Databricks, ADLSRole OverviewWe are seeking a skilled ...

About The Role Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Microsoft Azure Data Services Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data qua...

About The Role Project Role : Test Automation Lead Project Role Description : Lead the transformation of testing into a continuous and efficient end-to-end quality engineering function through the use of quality processes, tools, and methodologies significantly improving control, accuracy and integrity. Evolve more predictive and intelligent testing approaches based on automation and innovative testing products and solutions. Must have skills : AWS BigData Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Test Automation Lead, you will lead the transformation of testing into a continuous and efficient e...

About The Role Project Role : Test Automation Lead Project Role Description : Lead the transformation of testing into a continuous and efficient end-to-end quality engineering function through the use of quality processes, tools, and methodologies significantly improving control, accuracy and integrity. Evolve more predictive and intelligent testing approaches based on automation and innovative testing products and solutions. Must have skills : AWS BigData Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Test Automation Lead, you will lead the transformation of testing into a continuous and efficient e...