23086 Pyspark Jobs - Page 44

Job Title: Data Engineer Experience: 5+ Years Location: Chennai/Hyderabad/Mumbai/Pune/Bangalore (Hybrid) Key Responsibilities: Python Proficiency: Demonstrate a strong command of Python programming language, actively contributing to the development and maintenance of data engineering solutions. Data Engineering Expertise: Set up and maintain efficient data pipelines, ensuring smooth data flow and integration across systems. Experience with SQL and Data Warehouse/ Data Lakes is required. Contribute to the establishment and maintenance of Data Lakes, implementing industry best practices. Execute data scrubbing techniques and implement data validation processes to ensure data integrity and qual...

Primary Skills: Databricks, Python with Pyspark and AWS Experience: 5 - 7 years Location: Hyderabad (Hybrid mode) Notice Period: Immediate to 1 month preferred Interview Mode: Face to Face

Join us as a Software Engineer, PySpark This is an opportunity for a technically minded individual to join us as a Software Engineer Youll be designing, producing, testing and implementing working software, working across the lifecycle of the system Hone your existing software engineering skills and advance your career in this critical role We're offering this role at vice president level What you'll do Working in a permanent feature team, youll be developing knowledge of aspects of the associated platform across the disciplines of business, applications, data and infrastructure. Youll also be liaising with principal engineers, architects in the domain and other key stakeholders to understan...

Note: This is 5 Days work from office role JD for Data Scientist: Hands-on experience with data analysis tools: Proficient in using tools such as Python and R for data manipulation, querying, and analysis. Skilled in utilizing libraries like Pandas, NumPy, and Scikit-Learn to perform in-depth data analysis and modeling. Skilled in machine learning and predictive analytics: Expertise in building, training, and deploying machine learning models using frameworks such as TensorFlow and PyTorch. Capable of performing tasks like regression, classification, clustering, and recommendation, leading to data-driven predictions and insights. Expertise in big data technologies Proficient in handling larg...

Key Responsibilities Design, develop, and maintain data solutions leveraging Snowflake as the primary platform. Implement and optimize data warehousing solutions using Snowflake and Azure . Build and manage ETL pipelines with Azure Data Factory (ADF) . Work with additional ETL tools such as AWS Glue , Qlik Replicate , and DBT (added advantage). Collaborate in Agile project environments , ensuring timely delivery of data solutions. Engage with multiple stakeholders (internal and external) to gather requirements and deliver solutions effectively. Required Skills & Experience Strong hands-on experience with Snowflake , including data lake integration. Solid understanding of data warehousing con...

Senior Data Engineer Location: Chennai (Onsite) Role Description The Data Engineer will be responsible for designing and developing data engineering solutions, creating and maintaining data models, building Extract Transform Load (ETL) pipelines, implementing data warehousing, and performing data analytics on large datasets. Qualifications Data Engineering and Data Modeling skills Experience with Extract Transform Load (ETL) pipelines Expertise in implementing data warehousing solutions Analytical skills for performing data analytics on large datasets Strong programming skills in languages like Python, Java, or Scala Experience with Big Data and cloud-based technologies like Hadoop, Spark, a...

Senior Data Engineer Location: Chennai (Onsite) Role Description The Data Engineer will be responsible for designing and developing data engineering solutions, creating and maintaining data models, building Extract Transform Load (ETL) pipelines, implementing data warehousing, and performing data analytics on large datasets. Qualifications Data Engineering and Data Modeling skills Experience with Extract Transform Load (ETL) pipelines Expertise in implementing data warehousing solutions Analytical skills for performing data analytics on large datasets Strong programming skills in languages like Python, Java, or Scala Experience with Big Data and cloud-based technologies like Hadoop, Spark, a...

The Applications Development Senior Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities. Responsibilities: Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas Monitor and control all phases of development pr...

Project Role : Custom Software Engineer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Microsoft Azure Analytics Services Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Roles & Responsibilities: -Develop and Optimize Data Pipelines using PySpark and SQL in Databricks to handle large-scale data transformations and batch/streaming workloads. -Design and Implement ETL Processes by leveraging Databricks notebooks, Delta Lake, and Python-based custom scripts for data ingestion and curation. -Collaborate with Data Analysts and...

Role: Associate Architect - Data Experience Level: 6 to 10 Years Work location: Mumbai, Bangalore or Trivandrum (Hybrid) Notice period: Preferred up to 45 days Role & Responsibilities: 5+ years of experience in building and managing Data Lakes, Data Warehouse, Data Integration, Data Migration and Business Intelligence/Artificial Intelligence solutions on GCP Cloud. Ability to understand business requirements, translate them into functional and non-functional areas, and define non-functional boundaries in terms of Availability, Scalability, Performance, Security, Resilience etc. Experience in architecting, designing, and implementing end-to-end data pipelines and data integration solutions fo...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities An AI Data Scientist at IBM is not just a job title - it’s a mindset. You’ll leverage the watsonx,AWS Sagemaker,Azure Open AI platform to co-create AI value with clients, focusing on technology patterns to enhance repeatability and delight clients. We are seeking an experienced and innovative...

Introduction In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology. Your Role And Responsibilities An AI Data Scientist at IBM is not just a job title - it’s a mindset. You’ll leverage the watsonx,AWS Sagemaker,Azure Open AI platform to co-create AI value with clients, focusing on technology patterns to enhance repeatability and delight clients. We are seeking an experienced and innovative...

Job Title Senior Data Engineer EMEA (Kochi, India) Job Description Summary Utilizing our state-of-the-art intelligent data platform, you will craft a specialized cloud solution designed for the real estate domain. This platform empowers every individual within Cushman & Wakefield to adeptly tackle location-based challenges, thereby guaranteeing improved client service and enabling well-informed, future-proof decision-making. Embrace the opportunity to leverage the expansive potential of our intelligent data platform and commence making a substantial impact in the real estate industry today! Job Description About the Role: Analyze, organize, and model complex datasets in preparation for data ...

REConnect Energy's GRIDConnect platform helps integrate and manage energy generation and consumption for 1000s of renewable energy assets and grid operators. We are currently serving customers across India, Bhutan and the Middle East with expansion planned in US and European markets. We are headquartered in Central Bangalore with a team of 150+ and growing. You will jointhe Bangalore based Engineering team as a senior member and work at the intersection of Energy, Weather & Climate Sciences and AI. Responsibilities: Engineering - Take complete ownership of engineering stacks including Data Engineering and MLOps. Define and maintain software systems architecture for high availability 24x7 sys...

Role & responsibilities Assist in building and maintaining ETL/data pipelines Support in data cleaning, transformation, and integration Write SQL queries for data extraction and analysis Help manage and optimize databases and data warehouses Work with cloud platforms (AWS/Azure/GCP) for data storage and processing Collaborate with data analysts and BI teams on data requirements Contribute to automation of data workflows Maintain documentation of data processes and pipelines Participate in data modeling and architecture discussions Learn and apply new data engineering tools and technologies Preferred candidate profile : Mandatory Skills: SQL, Python Good to have skills: Cloud Platform(Azure, ...

Responsibilities: * Design, develop & maintain Databricks solutions using PySpark within Unity Catalog framework. * Collaborate with cross-functional teams on data engineering projects.

Job Description YOU’LL BUILD TECH THAT EMPOWERS GLOBAL BUSINESSES Our Connect Technology teams are working on our new Connect platform, a unified, global, open data ecosystem powered by Microsoft Azure. Our clients around the world rely on Connect data and insights to innovate and grow. As a Senior Data Engineer, you’ll be part of a team of smart, highly skilled technologists who are passionate about learning and supporting cutting-edge technologies such as Spark, Scala, Pyspark, Databricks, Airflow, SQL, Docker, Kubernetes, and other Data engineering tools. These technologies are deployed using DevOps pipelines leveraging Azure, Kubernetes, Jenkins and Bitbucket/GIT Hub. WHAT YOU’LL DO: Dev...

Junior Data Engineer (Only Chennai Candidates) Location: Chennai (Onsite) Invent Health is looking for a Data Engineer with a passion for Coding Job Description: 2-3 years of relevant work experience To work in capacity of AWS Databricks, Python, Pyspark Hands ON developer Work with stakeholders for regular updates, requirement understanding and design discussions. Hands-on experience on designing and developing scripts for custom ETL processes and automation in AWS, data bricks, Python, Pyspark etc. Should be skilled in creating architecture and design documentation and knowledge of data engineering tools & technologies Expertise in Cloud related Big Data integration and infrastructure Tec ...

At EY, we’re all in to shape your future with confidence. We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. EY GDS – Data and Analytics - D and A – Senior – Full stack As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in sectors like Banking, Insurance, Manufacturing, Government Healthcare, Retail, Manufacturing and Auto, Supply Chain, and Finance. The opportunity We’re looking for candidates with ...

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Objectives and Purpose The Data Engineering Manager leads large scale solution architecture design and optimisation to provide streamlined insights to partners throughout the business. This individual leads the team of Mid and Senior data engineers to partner with visualization on data quality and troubleshooting needs. The Data Engineering m...

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. EY GDS – Data and Analytics (D&A) – Snowflake Data Engineer- Senior We’re looking for candidates with strong technology and data understanding in data engineering space, having proven delivery capability. This is a fantastic opportunity to be part of a leading firm as well as a part of a growing Data and Analytics team Your key responsibiliti...

Join Amgen’s Mission of Serving Patients At Amgen, if you feel like you’re part of something bigger, it’s because you are. Our shared mission—to serve patients living with serious illnesses—drives all that we do. Since 1980, we’ve helped pioneer the world of biotech in our fight against the world’s toughest diseases. With our focus on four therapeutic areas –Oncology, Inflammation, General Medicine, and Rare Disease– we reach millions of patients each year. As a member of the Amgen team, you’ll help make a lasting impact on the lives of patients as we research, manufacture, and deliver innovative medicines to help people live longer, fuller happier lives. Our award-winning culture is collabo...

Role & responsibilities Preferred candidate profile Work Location-Bangalore 4-6 years of Experience Must have skills: Python, DB, Informatica Good to have: Spark DB optimization Data Warehousing experience High Volume data processing.

Key Responsibilities: Cloud & Data Engineering: Design, develop, and maintain data pipelines and ETL workflows using AWS services (S3, Glue, EMR, Lambda, Redshift). Implement distributed data processing using PySpark on AWS EMR or Databricks. Optimize data storage and retrieval for performance and cost efficiency. Programming & Automation: Develop reusable scripts and automation tools in Python for data ingestion, transformation, and validation. Implement best practices for code quality, testing, and version control (Git). Data Architecture & Integration: Work with structured and unstructured data from multiple sources. Integrate data pipelines with analytics and BI tools. Ensure data securi...

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. EY GDS – Data and Analytics (D&A) – Snowflake Data Engineer- Senior We’re looking for candidates with strong technology and data understanding in data engineering space, having proven delivery capability. This is a fantastic opportunity to be part of a leading firm as well as a part of a growing Data and Analytics team Your key responsibiliti...