14025 Data Engineering Jobs

Technical Skills * Programming: Python, SQL * Frameworks: PySpark, Spark SQL * AWS Data Services: * EMR * Glue * Lambda * S3 * RDS / MySQL * Redshift * CloudWatch * Secrets Manager * Experience with distributed computing, data modeling, schema handling, and performance tuning. * Strong debugging and problem-solving skills.

Design and implement scalable data pipelines using technologies such as Apache Spark, Kafka, and AWS Glue. Develop and maintain ETL processes to ingest, transform, and store structured and unstructured data. Optimize data storage solutions, including data lakes, data warehouses, and relational databases. Collaborate with cross functional teams to ensure data availability and quality. Work with big data technologies such as Hadoop, Snowflake, or Big Query for large-scale data processing. Monitor and troubleshoot data pipeline failures, optimizing performance and reliability. Implement security best practices for data storage, access control, and encryption. Stay up to date with the latest dat...

As a Data & AI Manager at Perennial Systems, you will play a crucial role in driving AI and data strategies while demonstrating leadership qualities to guide key projects and mentor junior team members. Your responsibilities include: - Setting a clear vision and strategic direction for the practice, anticipating future trends in Data Engineering, Data Science, and Data Visualization. - Utilizing proficiency in AI tools such as TensorFlow, PyTorch, spaCy, and Transformers to innovate and develop cutting-edge solutions for clients' challenges and business growth. - Leading multidisciplinary teams of data scientists, AI engineers, and domain experts to foster a collaborative work environment. -...

As a Senior Platform Engineer Vice President at Barclays, you will play a pivotal role in shaping the evolution of the Reference Data Services function. Your technical expertise and leadership skills will be instrumental in driving the development of the Dart platform team, providing cloud and on-prem infrastructure to front-office developers. Your responsibilities will include platform development, stakeholder engagement, and engineering leadership across both cloud and on-prem environments. Collaboration with internal teams and front-office developers is essential, requiring an understanding of financial services data needs and organizational functions. Key Responsibilities: - Lead the eng...

Role Overview: You are a highly skilled and experienced Senior Data Scientist with a minimum of 3 - 7 years of experience in Data Science and Machine Learning, preferably with expertise in NLP, Generative AI, LLMs, MLOps, Optimization techniques, and AI solution Architecture. Your key role will be in the development and implementation of AI solutions, leveraging your technical expertise. You should have a deep understanding of AI technologies and experience in designing and implementing cutting-edge AI models and systems. Additionally, expertise in data engineering, DevOps, and MLOps practices will be valuable in this role. Responsibilities: - Contribute to the design and implementation of s...

INTERESTED CAN DROP CV FOR 734 885 9771 Project Role : Custom Software Engineer Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs. Must have skills : Data Engineering Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary: As a Custom Software Engineer, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with...

Seeking AWS & MuleSoft devs (ETL, APIs, Glue, Lambda, SNS/SQS, RDS) for cloud/event-based design. Must work NA hours & manage vendor interaction. Strong integration skills required.

We are looking for a highly skilled and experienced Senior Lead Software Engineer to join our team As an Lead Software Engineer you will be responsible for designing and overseeing the construction of buildings and structures Your main goal will be to ensure that all projects are completed on time within budget and meet the client s specifications This is a full-time position and you will be required to work from the clients office The ideal candidate will have at least 6 years of work experience in the field of architecture and a strong understanding of building codes and regulations You should also have excellent communication and project management skills as you will be working closely wi...

Design, develop, and maintain data pipelines and ETL processes using AWS and Snowflake Implement data transformation workflows using DBT (Data Build Tool) Write efficient, reusable, and reliable code in Python Optimize and tune data solutions for performance and scalability Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions Ensure data quality and integrity through rigorous testing and validation Stay updated with the latest industry trends and technologies in data engineering Bachelor''s or Master''s degree in Computer Science, Engineering, or a related field Proven experience as a Data Engineer or similar role Strong pro...

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skills.

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

- Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub. - Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. - Designing and implementing data engineering, ingestion, and transformation functions - Azure Synapse or Azure SQL data warehouse - Spark on Azure is available in HD insights and data bricks - Good customer communication. - Good Analytical skill

Overview : We are seeking a highly skilled and experienced Senior Data Engineer (L4) to join the program in supporting critical customer engagement. We are looking for a candidate to design and build scalable data pipelines on GCP, responsible for replacing end-of-life legacy ETL tools with modern cloud frameworks and building core data models. This role is ideal for a self-starter who possesses strong analytical problem-solving skills, specifically the ability to reverse-engineer undocumented legacy code and map it to modern logic without losing business context. The ideal candidate adapts quickly between different technologies (Java, SQL, Python) and maintains rigorous attention to detail ...

Role : Minimum 10 years of experience and 8 years in Markit EDM mandatory. We are seeking a highly skilled and motivated Senior MarkIT EDM Data Engineer to join our team. The ideal candidate will be a self-starter who can work independently and collaborate effectively with both internal and external stakeholders. This role requires a deep understanding of data engineering principles, strong technical skills, and the ability to drive projects to successful completion. Key Responsibilities. - Minimum 10 years of experience and 8 years in Markit EDM mandatory. - Design, develop, and maintain data solutions using MarkIT EDM. - Collaborate with internal teams and external stakeholders to gather r...

Job Position : GCP Data Engineer Experience : 6+ Yrs into Gcp Data Engineer Notice Period : Immediate - 15 days only JD- Key Responsibilities : We are seeking a skilled Data Engineer to design, build, and maintain data pipelines and data models that support analytical and business intelligence needs. The ideal candidate will have hands-on experience with Python or SQL, Google Cloud Platform (GCP), and a strong understanding of data management, quality, and security best practices. Key Responsibilities : - Build and maintain moderately complex data pipelines, ensuring data flow, transformation, and usability for analytical projects. - Design and implement data models, optimizing for performan...

Job Title : Databricks Tech Lead (Contract) Contract Duration : 4 Months (Extendable based on Performance) Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 7+ Years Job Description : We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment. Key Responsibilities : - Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services. - Productio...

Experience: 6+ Years Domain: Data Engineering / Big Data About the Role: We are seeking a Senior Data Engineer to drive the development of our next-generation Data Lakehouse architecture. You will be responsible for designing, building, and optimizing massive-scale, low-latency data pipelines that support real-time analytics and Machine Learning applications. Key Responsibilities: - Design and build highly optimized, production-grade ETL/ELT pipelines using Apache Spark (PySpark/Scala) to process petabytes of data. - Architect and manage the Data Lakehouse using open-source technologies like Delta Lake or Apache Hudi for ACID transactions and data quality. - Integrate and process real-time d...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...

Role : Senior MarkIT EDM Data Engineer We are seeking a highly skilled and motivated Senior MarkIT EDM Data Engineer to join our team. The ideal candidate will be a self-starter who can work independently and collaborate effectively with both internal and external stakeholders. This role requires a deep understanding of data engineering principles, strong technical skills, and the ability to drive projects to successful completion. Key Responsibilities : - Design, develop, and maintain data solutions using MarkIT EDM. - Collaborate with internal teams and external stakeholders to gather requirements and deliver data solutions. - Ensure data quality, integrity, and security across all data pr...

Job Title: SR DATA ENGINEER Company Name: Kohler Co. Job Description: As a Senior Data Engineer at Kohler Co., you will play a critical role in designing, developing, and maintaining robust data pipelines and architectures that enable data-driven decision-making across the organization. You will collaborate with cross-functional teams to establish data standards, improve data quality, and ensure data accessibility. Your expertise in data engineering practices will help us scale our data capabilities and leverage advanced analytics. Key Responsibilities: - Design, develop, and implement high-performance data pipelines to process large volumes of data from various sources. - Collaborate with d...

Skill: AWS Glue + PySpark Experience : 6 to 13 Years Location : AIA - Pune Key Responsibilities: Design, develop, and optimize AWS Glue ETL jobs using PySpark. Integrate data from multiple sources (S3, RDS, Redshift, DynamoDB, etc.) into a centralized data lake or warehouse. Implement data transformations, cleansing, and enrichment logic in PySpark. Configure and manage AWS Glue Crawlers for schema inference and Data Catalog updates. Optimize job performance by tuning Spark configurations and partitioning strategies. Implement error handling, logging, and monitoring for Glue jobs. Collaborate with data engineers and analysts to define data requirements and transformation rules. Ensure data q...

Job Title : Databricks Developer (Contract) Contract Duration : 4 Months +Exendible based on Performance Job Location : Remote Job Timings : India Evening Shift (till 11 : 30 PM IST) Experience Required : 4+ Years Job Description : We are seeking a skilled Databricks Developer to join our team on a 4-month contract basis. The ideal candidate will have strong expertise in modern data engineering technologies and the ability to work in a fast-paced, remote environment. Key Responsibilities : - Develop, optimize, and manage large-scale data pipelines using Databricks, PySpark, DBT, and AWS S3/Glue/Redshift. - Work in multi-cloud environments including AWS, Azure, and GCP. - Implement workflow o...