1521 Data Lake Jobs - Page 22

As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorpora...

Experience: 7+ years Responsibilities: Ensure cross-functional alignment, resolve conflicts, and manage dependencies. Collaborate with security teams to enforce secure coding practices and compliance standards. Lead troubleshooting efforts, guiding developers through resolving technical challenges. Work closely with Solution Architects to ensure architectural consistency. Identify and mitigate risks related to scalability, security, and technical debt. Drive process improvements and stay current with industry trends to propose innovative solutions. Technologies: Angular, C# .NET, Azure SQL, MongoDB Data Analytics: Data Lake, Databricks, ADF, Power BI Messaging: Azure Service Bus Storage: Azu...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a detail-focused and highly skilled Data Quality Engineer (Automation) to join our data engineering team. The candidate will bring expertise in ETL automation, data warehouse testing, and cloud data services, as well as hands-on knowledge of test automation frameworks and CI/CD methodologies. Responsibilities Define and implement automated test scripts to validate ETL workflows and data pipelines Ensure end-to-end data validation, reconciliation, and testing across large data sets and DWH systems Collaborate with cross-functional teams, including data engineers, analysts, and stakeholders, to establish and meet data quality standards Create reusable and scalable test automatio...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

Job summary: Analytics Team at Grant Thornton INDUS Marketing are seeking a skilled and motivated Microsoft Fabric Engineer. In the given role, you will be responsible for designing, implementing, and maintaining solutions within the Microsoft Fabric ecosystem. Your expertise will contribute to the optimization of enterprise data processes, analytics workflows, and system integrations, ensuring the delivery of high-performance, scalable, and secure solutions. Key responsibilities: Design, implement, and maintain solutions leveraging Microsoft Fabric services (e.g., Power BI, Synapse Analytics, Azure Data Factory, Data Lake) Develop and manage robust data pipelines for ETL/ELT processes withi...

We are hiring for - Role: Senior Data Engineer Experience Range: 3 to 5 Years Qualification: Degree in Computer Science (or similar), alternatively well-founded professional experience in the desired field Roles & Responsibilities: As a Senior Data Engineer, you manage and develop the solutions in close alignment with various business and Spoke stakeholders. You are responsible for the implementation of the IT governance guidelines. Collaborate with the Spokes Data Scientists, Data Analysts, and Business Analysts, when relevant. Tasks Create and manage data pipeline architecture for data ingestion, pipeline setup and data curation Experience working with and creating cloud data solutions Ass...

Role Overview: As an Applications Development Senior Programmer Analyst at our company, you will be an intermediate level professional responsible for participating in the establishment and implementation of new or revised application systems and programs. Your main objective will be to contribute to applications systems analysis and programming activities in coordination with the Technology team. Key Responsibilities: - Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establishment of new or revised applications systems and programs to meet specific business needs or user areas - Monitor and...

Job Description: As a Pre-sales Engineer, you will collaborate with the Sales team to offer comprehensive technical guidance and solutions to both prospects and customers. Your role will involve advising customer teams on various Cloud technologies such as migration, Data lake, AI/ML, security, and best practices to ensure they fully leverage the potential of different cloud offerings. You must possess strong communication and technical skills, along with a high level of collaboration to drive customer success effectively. Key Responsibilities: - Work closely with the Delivery team from the initial concept to deal closure, providing expertise in deploying cloud solutions using modern digital...

We are seeking a talented and experienced Senior Data Engineer to join our team and contribute to building a robust data platform on Azure Cloud. The ideal candidate will have hands-on experience designing and managing data pipelines, ensuring data quality, and leveraging cloud technologies for scalable and efficient data processing. The Data Engineer will design, develop, and maintain scalable data pipelines and systems to support the ingestion, transformation, and analysis of large datasets. The role requires a deep understanding of data workflows, cloud platforms (Azure), and strong problem-solving skills to ensure efficient and reliable data delivery. Key Responsibilities Data Ingestion ...

Design, develop new features for a large-scale cloud data platform using Databricks, snowflake and Fabric, Deeply understand legacy data platforms, and drive their modernization, Provide production support Design data pipelines and data warehouse. Required Candidate profile 5 years in developing and scaling data platforms using Snowflake, databricks, Fabric, Strong Python and SQL Intermediate C# Understanding of modern data architectures such as Data Mesh, Lakehouse

This opening is for PAN India locations (Hybrid). Experience in Design / Implementation of Data Pipelines for processing large volumes of data Experience in ingesting batch and streaming data from various data sources. Strong Experience in writing complex SQL using any RDBMS (Oracle, PostgreSQL, SQL Server etc.) Data Modeling: Proficiency in creating both normalized and denormalized database schemas. Proficient in developing applications in Python. Exposed to python libraries such as NumPy, pandas, beautiful soup, etc Experience in developing ETL, OLAP based and Analytical Applications. Good working experience in Azure / AWS services Experience working in Databricks, Snowflake or other cloud...

Experience- 8 to 16 years Location- Pan India Key Responsibilities: Design and implement data architecture solutions using Databricks on AWS or Azure cloud platforms. Lead the development of robust ETL/ELT pipelines for batch and real-time data processing. Optimize compute, storage, and network resources to ensure cost-effective and high-performance data solutions. Collaborate with cross-functional teams (Data Engineers, Data Scientists, DevOps, and Product teams) to define data requirements and delivery approaches. Implement data governance, security controls, access management, and compliance protocols. Architect data lakes and Delta Lake implementations for structured and unstructured dat...

Job Title : Azure Data Engineer - ADF, SQL & DataBricks Experience : 4 to 8 years Category : Software Development/ Engineering Location : Bangalore only Notice Period: Immediate / Max 30 Days Employment Type : Full Time Qualifying Criteria - 4+ years of experience in Azure Data Engineer - ADF, ADB, ADLS - Notice period / Joining duration - Immediately / Less than 30 Days - Willing to work out of Bangalore - Hybrid model Key Skills : Azure Data Technologies - Azure ADF, Azure Databricks, SQL, Azure Data Lake Storage. Requirements: - Strong proficiency in Microsoft Azure cloud platform services (Azure Data Factory, Azure Data Bricks, Azure SQL Database, Azure Data Lake Storage, Azure Synapse A...

About this role: Wells Fargo is seeking a highly skilled Lead Data Management Analyst to lead strategic data initiatives across business, technology, and compliance functions within the investment banking domain. This role blends data management, business analysis, and project leadership, with a strong focus on modern data architecture. In this role, you will: Organize and lead complex companywide initiatives to ensure that data quality is maintained so that data can effectively support business processes Oversee analysis and reporting in support of regulatory requirements Identify and recommend analysis of data quality or integrity issues Evaluate data quality metrics and data quality audit...

Essential for this role: Education and Qualifications: • Bachelor’s degree in IT, Computer Science, Software Engineering, Business Analytics or equivalent. • Minimum seven plus years of experience in data analytics field • Experience with Azure/AWS Databricks • Experience in building and optimizing data pipelines, architectures and data sets • Excellent experience in Scala or Python, PySpark and SQL • Ability to troubleshoot and optimize complex queries on the Spark platform • Knowledgeable on structured and unstructured data design / modelling, data access and data storage techniques • Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS • Hands on...

Power Platform Developer Job Summary Microsoft Power Platform Developer will be responsible for designing, developing, and maintaining applications and solutions using the Microsoft Power Platform, including MS Copilot Studio, Power Automate, Power Apps, Power Pages, Dataverse and Power BI Knowledge on M365, Azure Services and Tools (including Data Factory, Azure SQL, etc.) and SharePoint Development, Dynamic 365 Integration will be highly beneficial for this role Key Responsibilities Develop, Deploy and Maintain Solutions in Power Platform to address business needs and improve processes Work closely with business analysts, project managers, and other stakeholders to gather and understand re...

Role Overview: At EY, you'll have the opportunity to build a unique career with global scale, support, inclusive culture, and technology to help you become the best version of yourself. Your unique voice and perspective are valued to contribute towards making EY even better. Join EY to create an exceptional experience for yourself and contribute towards building a better working world for all. Key Responsibilities: - Learn and adapt to the ETL technology landscape built on Microsoft Azure Data Services - Create complex SQL queries, including dynamic queries for data processing - Convert business and technical requirements into technical solutions using Azure Data Factory, Databricks, Azure D...

Are you a visionary who thrives on designing future-ready data ecosystems? Lets build the next big thing together! We're working with top retail and healthcare leaders to transform how they harness data and were looking for a Data Architect to guide that journey. We are looking for an experienced Data Architect with deep knowledge of Databricks and cloud-native data architecture. This role will drive the design and implementation of scalable, high-performance data platforms to support advanced analytics, business intelligence, and data science initiatives within a retail or healthcare environment. Key Responsibilities: Define and implement enterprise-level data architecture strategies using ...