10148 Data Engineering Jobs - Page 28

What will your role look like Design, develop, and maintain scalable data pipelines and applications using Python and PySpark. Build end-to-end data solutions from scratch, ensuring clean, modular, and reusable code. Write and optimize complex SQL queries for large-scale data processing, transformation, and validation. Develop dynamic, reusable frameworks and implement design patterns for scalability and performance. Refactor legacy codebases to enhance performance, maintainability, and reliability. Collaborate with cross-functional teams including data engineers, analysts, and stakeholders to translate business requirements into technical solutions. Utilize Databricks and Azure Data Factory...

Role : AWS Data Engineer Exp : 5 to 10 Years Location : Hyderabad Mode : Hybrid Job Overview: We are seeking a highly skilled Data Engineer with strong expertise in AWS cloud services and Python programming . The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines, ensuring data availability, quality, and performance across enterprise systems. You will collaborate closely with data analysts, data scientists, and business stakeholders to deliver reliable, high-quality data solutions. Key Responsibilities Design, develop, and maintain ETL/ELT data pipelines using Python and AWS native services (Glue, Lambda, EMR, Step Functions, etc.) Build and ...

Permanent Role with Genpact Location: Genpact, Hyderabad Uppal Office Hybrid Shift Timing- 12:00 PM to 10:00 PM IST Advanced Python Developer / Python Programming Expert (ETL | Cloud | Gen AI) Job Summary: We are seeking an Advanced Python Developer to design and develop scalable, data-driven solutions and contribute to innovative projects. This role is ideal for individuals with a deep understanding of Python programming, ETL processes, and modern data architectures across cloud platforms. Key Responsibilities: Develop and optimize scalable code using Advanced Python Programming . Build and maintain ETL pipelines for extracting, transforming, and loading data. Collaborate on architecture an...

AIA-Pune Job Summary Data Engineer 4 to 9 Years Responsibilities Data Engineering design experience (to support with HLD review LLD build out etc.) Azure experience confident in working with Azure Cloud and Data Engineering capabilities ( e.g. ADLS Gen2 Synapse ADF etc.) Advanced SQL experience query writing & optimisation data aggregation stored procedures etc. Experience writing data Pipelines using Python and PySpark. Relational data model / data schema design experience Understanding of DevOps and CI/CD in the context of Data Engineering Familiarity with working in a scrum environment and supporting scrum ceremonies Certifications Required Azure Pyspark Python

We are seeking a highly skilled and visionary Software Engineering Manager to lead a team of engineers in building Caterpillars next-generation Digital Manufacturing Data Platform . This platform is central to our Future of Manufacturing initiativedesigned to unify and operationalize data across design, engineering, production, and supply chain operations. The ideal candidate will possess deep expertise in Big Data, Data Warehousing, real-time data movement , and Snowflake-based architectures . You will architect and deliver scalable, secure, and intelligent data platforms that enable advanced analytics, AI, and digital twin capabilities across global manufacturing ecosystems. Key Responsibi...

Role & responsibilities We are seeking a skilled and motivated Azure Cloud Data Engineer to join our cloud engineering team. In this role, you will design, develop, and maintain scalable data systems and cloud-based solutions using Microsoft Azure. This role is ideal for someone who thrives at the intersection of data and cloud, with responsibilities split evenly between data a nd cloud engineering. You will play a key role in shaping our cloud data strategy, mentoring junior engineers, and driving best practices across the organization. Key Responsibilities: Data Engineering: Design data architectures that support scalability, security, and performance spanning multiple cloud environments. ...

Role 1: Snowflake_ETL_DataEngineer Role 2: SQL, ETL, Snowflake Role 3: snowflake with dbt

Job Overview We are looking for a Data Engineer who will be part of our Analytics Practice and will be expected to actively work in a multi-disciplinary fast paced environment. This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project; its primary responsibility is the acquisition, transformation, loading and processing of data from a multitude of disparate data sources, including structured and unstructured data for advanced analytics and machine learning in a big data environment. Responsibilities: \u2022Engineer a modern data pipeline to collect, organize, and process data from disparate sources. \u2022Performs d...

Requirements: Python Dev with Bachelors degree in Computer Science, Information Technology, or a related field. Proven experience as a Python Developer with expertise in PySpark. Strong understanding of big data technologies and frameworks. Experience with distributed computing and parallel processing. Proficiency in SQL and experience with database systems. Solid understanding of data engineering concepts and best practices. Ability to work in a fast-paced environment and handle multiple projects simultaneously. Excellent problem-solving and debugging skills. Strong communication and collaboration abilities.

What will your role look like As a Senior Data Engineer, you will architect, build, and maintain scalable data pipelines for both batch and real-time processing. Youll be hands-on with Python, SQL, and modern data tools to ensure data is reliable, consistent, and accessible across the business. You will partner closely with analysts, scientists, and engineers to transform raw data into actionable insights, while optimizing performance and ensuring smooth production operations. Why you will love this role Youll have the opportunity to own end-to-end data engineering solutions that directly impact decision-making and innovation across the company Youll work with a modern data stack (Airflow, F...

Overview As a Senior Software Engineer, youll contribute to Python-based development across data pipelines, AI/ML models, and system integrations to drive automation and intelligence in network operations. Responsibilities Key Responsibilities: Develop Python modules for data ingestion, normalization, and integration from internal and vendor systems. Contribute to building and maintaining the Canonical Circuit Intelligence Graph (CCIG) using graph databases. Implement AI/ML models for anomaly detection, clustering, and forecasting. Support development of GenAI models (e.g., CircuitGPT, ClauseMatcherBERT) for semantic understanding and recommendations. Build Python agents for topology discove...

Overview Prodapt is lookig for Senior Technical Lead in Data Engineering stream having 11+ years of experience having hands on expertise on GCP, BQ, Data Pipeline creation & ETL. Responsibilities Understanding the requirements Performing in-depth analysis on data Setting up secure data pipelines from ingestion to delivery Individual contributor Requirements 11+ years of experience in Data Engineering with strong ability to work independently, communicate effectively, and deliver high-quality solutions. Should have strong SQL programming skills. Experience in cloud-based systems (GCP, BQ) is must. Performs data analysis required to troubleshoot data related issues and assist in the resolution...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Databricks Unified Data Analytics Platform Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :Data Engineering Sr. Advisor demonstrates expertise in data engineering technologies with the focus on engineering, innovation, strategic influence and product mindset. This individual will act as key contributor of the team to design, build, test and deliver large-scale software applications, systems, platforms, services or t...

About The Role Job Title Data Science/Data Engineering ML9 - Sales Excellence COE - Data Engineering Specialist Management Level :ML9 Location:Open Must have skills: strong GCP cloud technology experience, Big query, Data Science basics. Good to have skills: Build and maintain data models from different sources. Experience: Minimum 5 year(s) of experience is required Educational Qualification: Graduate/Post graduate Job Summary :The Center of Excellence (COE) makes sure that the sales and pricing methods and offerings of Sales Excellence are effective. The COE supports salespeople through its business partners and Analytics and Sales Operations teams. The Data Engineer helps manage data sour...

Guarantee that Databricks best practices are applied throughout all projects Your role will involve designing and building robust data pipeline development and deployment of innovative big data and AI applications advanced data engineering techniques Required Candidate profile Expert-level proficiency in Spark Scala, Python, and PySpark. Knowledge of data architecture, including Spark Streaming, Spark SQL Data Bricks expertise Spark Scala, Python and PySpark etc. Perks and benefits Please reply ,share resume at info@whitepepper.in

Coeo are trusted data management and analytics experts, delivering technology strategy and support for business. The team have deep technical and commercial experience working with Microsoft Data Services to help our clients optimise their costs and maximise the benefits from their investments in these technologies. Coeo has been established for over 15 years and has offices in Reading, UK and Hyderabad, India. We are exclusively focused on Microsoft technologies. Our mission is to "help our clients predict their future" through the better use of data, technology, people, and processes. To do this our business has always focused on:. Data Managed Services. Data Platform Consultancy. Analytic...

Role : Data Engineer Experience : 10 to 20 Yrs Key Responsibilities: Serve as the point of contact and subject matter expert for all Databricks-related activities, including architecture, development, and operational best practices. Should work closely with Sales team, propose data roadmap to prospects intending to migrate to cloud, create proof of concepts to showcase our expertise. Design, develop, and manage ETL/ELT pipelines in Databricks using Python (PySpark), integrating various data sources to support business operations. Leverage Unity Catalog to ensure data lineage, security, and governance are properly managed across the Databricks environment. Implement and maintain CI/CD pipelin...

We are seeking an experienced Cloudera Data Engineer to design, build, and optimize real-time data pipelines using Cloudera Platform and Cloudera Streams Messaging (CSM) on AWS. The ideal candidate should have client-facing experience, strong communication skills, and the ability to work in a mixed client/Deloitte team. Key Responsibilities: Design & develop real-time data ingestion and transformation pipelines Optimize streaming data jobs for performance, reliability, and scalability Work with Cloudera Platform & Cloudera Streams Messaging (CSM) Deploy & manage data pipelines on AWS Follow best practices in data engineering & software principles Collaborate with Financial Services clients (...

We are seeking an experienced MLOps Engineer to join our team and help design, implement, and maintain scalable machine learning infrastructure. In this role, you will bridge the gap between data scientists and production environments, ensuring seamless integration, robust pipelines, and reliable operations of machine learning models in production. Typical work week look like:-. Model Deployment:Build and maintain scalable pipelines for deploying machine learning models to production environments. Automation:Automate machine learning workflows, including model training, testing, monitoring, and retraining. Infrastructure Management:Design, build, and maintain cloud and/or on-premises infrast...

Skills: Azure, Data Engineer, python, SQL, ADF, Databrick, Azure Databricks,. Position:Azure Data Engineer. Work Experience:5-8years. Work location:Bangalore/Hyderabad/Pune. Required Skills. Azure, Strong SQL, ADF, Python, PySpark, Databricks. Key Responsibilities. Design and develop data pipelines using Azure services. Collaborate with cross-functional teams to understand data requirements. Optimize and troubleshoot data pipelines for performance and reliability. Implement data security and governance best practices. Stay updated with the latest industry trends and emerging Azure. Show more Show less

Skills: ML, GCP, ML Ops, Ci/CD, Airflow, Keras,. Role:ML Ops Engineer. Experience:6-9 Yrs. Location:Bangalore. Key Responsibilities. Design and implement cloud solutions, build MLOps on cloud (GCP). Build CI/CD pipelines orchestration by GitLab CI, GitHub Actions, Circle CI, Airflow or similar tools. Data science model review, run the code refactoring and optimization, containerization, deployment, versioning, and monitoring of its quality. Data science models testing, validation and tests automation. Communicate with a team of data scientists, data engineers and architects, document the processes. Required Qualifications. Ability to design and implement cloud solutions and ability to build ...

SUMMARY Job Title: Technology Lead, Digital Solutions Location: Bangalore, India Department: Technology Reports To: MD & CBO About EmbarkGCC : EmbarkGCC is a specialized consulting and execution platform designed to help global enterprises build, scale, and evolve Global Capability Centres (GCCs) in India. We offers a comprehensive, end-to-end solution that addresses every critical stage of a GCC’s lifecycle from strategy and setup to long-term transformation. Founded by experienced practitioners with deep domain expertise and ably sponsored by Embassy Group, Embark was created to fill a critical gap in the GCC landscape. Our comprehensive solution spans the entire lifecycle of a GCC from st...

Skills: data engineer, python, pyspark, spark, sql, Data Analysis,. Role:Data Engineer. Location:Bangalore. IN-PERSON INTERVIEW. CANDIDATE MUST BE LOCAL. Data Engineer. Experience in:Python, Spark, Pyspark, SQL, Unix (Shell scripting), Databricks, Azure, R(optional). Packages:Scikit-Learn, NumPy, Pandas, TensorFlow, Keras, pytorch. Database:Mysql, Teradata, No-sql (Mongo preferably), In memory db (Redis preferably). Business Intelligence:Power BI (optional). Show more Show less

Skills: Apache Spark, Data Engineering, Python, SQL, Machine Learning, Data Analysis, ETL,. Company Overview. CN Solutions partners with clients to provide top-notch solutions for their manpower needs. Specializing in Web Technologies, Databases, ERP, Data warehousing, and more. Offering staffing solutions, leadership hiring, RPO services, and more. Job Overview. Senior Databricks role with 7 to 10 years of experience in Hyderabad. Full-Time employment with. Qualifications And Skills. 7-10 years of experience in Data Engineering and Databricks. Proficiency in Python, SQL, and data analysis tools. Strong knowledge of Apache Spark and ETL processes. Experience in Machine Learning and working w...

We are looking for a skilled Business Technology Solutions Consultant - Data Engineer with 5-10 years of experience to join our team in India. The ideal candidate will have a strong background in data engineering and excellent problem-solving skills. Roles and Responsibility Design, develop, and implement data engineering solutions to meet business requirements. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems and architectures. Ensure data quality, integrity, and security. Troubleshoot and resolve complex technical issues. Provide technical guidance and support to junior team members. Job Requirements Stro...