1019 Data Lake Jobs - Page 39

Immediate to 30 days notice period candidates preferred Key Responsibilities: Design and implement general architecture for complex data systems. Translate business requirements into functional and technical specifications. Design and implement lakehouse architecture. Develop and manage cloud-based data architecture and reporting solutions. Apply data modelling principles for relational and dimensional data structures. Design Data Warehouses following established principles (e.g., Kimball, Inmon). Create and manage source-to-target mappings for ETL/ELT processes. Mentor junior engineers and contribute to architectural decisions and code reviews. Minimum Qualifications: Bachelors Degree in Co...

We are seeking an experienced and highly skilled Senior Data Engineer to drive data-driven decision-making and innovation. In this role, you will leverage your expertise in advanced analytics, machine learning, and big data technologies to solve complex business challenges. You will be responsible for designing predictive models, building scalable data pipelines, and uncovering actionable insights from structured and unstructured datasets. Collaborating with cross-functional teams, your work will empower strategic decision-making and foster a data-driven culture across the organization. Role & responsibilities 1 Position Overview: We are seeking an experienced and highly skilled Senior Data ...

Job Summary: We are seeking an experienced Azure Data Engineer with strong expertise in big data ecosystems. The ideal candidate will design, develop, and maintain scalable data pipelines, ensuring seamless integration, transformation, and storage of large datasets using Azure cloud services and big data tools. Key Responsibilities: Design, implement, and optimize data pipelines using Azure Data Factory, Databricks, and related services. Develop and maintain data lake solutions using Azure Data Lake Gen2. Implement ETL/ELT processes for structured and unstructured datasets. Collaborate with data scientists, analysts, and business teams to deliver high-quality data solutions. Ensure data qual...

Hey Everyone, I Hope you all are doing great. I am having opportunity for one of my MNC Client. Below are the details. Please check. Employment Type - C2H Experience - 8+ Years Location - Pan India Notice Period - Immediate to 15 Days Max Job Description - Must Have : Minimum experience required is 8 years. Experience in Spark and Scala with exposure about cloud platforms ( AWS, Azure, GCP) for big data processing and storage. Strong experience in Azure DLS Strong experience in Databricks, data pipelines Experience in Hadoop Good To Have : Agile delivery experience, seeking someone with strong backend development expertise, particularly in Java(Spring framework) keywords for your reference: ...

Cloud Data Engineer Req number: R5934 Employment type: Full time Worksite flexibility: Remote Who we are CAI is a global technology services firm with over 8,500 associates worldwide and a yearly revenue of $1 billion+. We have over 40 years of excellence in uniting talent and technology to power the possible for our clients, colleagues, and communities. As a privately held company, we have the freedom and focus to do what is right—whatever it takes. Our tailor-made solutions create lasting results across the public and commercial sectors, and we are trailblazers in bringing neurodiversity to the enterprise. Job Summary We are seeking a motivated Cloud Data Engineer that has experience in bu...

We are looking for an experienced Senior Data Engineer to lead the development of scalable AWS-native data lake pipelines , with a strong focus on time series forecasting , upsert-ready architectures , and enterprise-grade data governance . This role demands end-to-end ownership of the data lifecycle from ingestion to partitioning, versioning, QA, lineage tracking, and BI delivery. The ideal candidate will be highly proficient in AWS data services , PySpark , and versioned storage formats such as Apache Hudi or Iceberg . A strong understanding of data quality , observability , governance , and metadata management in large-scale analytical systems is critical. Roles & Responsibilities Design ...

Sr Data Engineer SR DE-II Highly skilledDataEngineerwith minimum 6+ years of relevant experience in SQL, PySpark, ETL,DataLakes and Azure Tech Stack. Responsibilities 4+ years of experience in buildingdataPipelines with Python/PySpark. 5+ years of experience in Azure ETL stack ( eg. Blog Storage,DataLake,DataFactory, Synapse). 4+ years of experience with SQL. Proficient understanding of code versioning tools such as Git and PM tool like Jira. Should have Experience in Leading the Team Qualifications Excellent verbal and written communication skills. UG: B.Sc in Any Specialization, BCA in Any Specialization, B.Tech/B.E. in Any Specialization. A good internet connection is a must. Azure certif...

Position: Sr. Solutions Architect 100% Remote Time: 3PM onwards Job Summary: Role - Sr. Solutions Architect This project is working on a new analytic platform, consolidation, and implementation and will identify, lead, and deliver data analysis and architecture optimization. Must haves: Must have experience with Data Lake Infrastructure, Data warehousing and Data Analytics Tools. Expertise in SQL Optimization and performance tuning and development of procedures Experience with Database technologies such as SQL Oracle or Informatica Working knowledge of Agile based development including DevOps, DataOps Good problem-solving skills including debugging skills Experience leading 3-4 projects as t...

Must be an expert in SQL, Data Lake, Azure Data Factory, Azure Synapse, ETL, Databricks Must be an expert in data modeling, writing complex queries in SQL, Stored procedures / PySpark. Notebooks for handling complex data transformation Required Candidate profile - Ability to convert SQL code to PySpark - Strong programming skills in languages such as SQL, Python - Exp with data modelling, data warehousing & dimensional modelling concept

Must be an expert in SQL, Data Lake, Azure Data Factory, Azure Synapse, ETL, Databricks Must be an expert in data modeling, writing complex queries in SQL, Stored procedures / PySpark. Notebooks for handling complex data transformation Required Candidate profile - Ability to convert SQL code to PySpark - Strong programming skills in languages such as SQL, Python - Exp with data modelling, data warehousing & dimensional modelling concept

Minimum of 6 yrs of Data Engineering Exp Must be an expert in SQL, Data Lake, Azure Data Factory, Azure Synapse, ETL, Databricks Must be an expert in data modeling, writing complex queries in SQL Ability to convert SQL code to PySpark Required Candidate profile Exp with SQL, Python, data modelling, data warehousing & dimensional modelling concept Familiarity with data governance, data security & Production deployments using Azure DevOps CICD pipelines.

Minimum of 6 yrs of Data Engineering Exp Must be an expert in SQL, Data Lake, Azure Data Factory, Azure Synapse, ETL, Databricks Must be an expert in data modeling, writing complex queries in SQL Ability to convert SQL code to PySpark Required Candidate profile Exp with SQL, Python, data modelling, data warehousing & dimensional modelling concept Familiarity with data governance, data security & Production deployments using Azure DevOps CICD pipelines.

Creating the overall structure and framework for how data is managed, including databases, data warehouses, and data lakes. Defining the logical structure of data, including entities, attributes, and to ensure data is organized effectively Required Candidate profile Strong understanding of database technologies (SQL, NoSQL), data modeling, data warehousing, and cloud platforms. Ability resolve issues related to data infrastructure and data management.

Creating the overall structure and framework for how data is managed, including databases, data warehouses, and data lakes. Defining the logical structure of data, including entities, attributes, and to ensure data is organized effectively Required Candidate profile Strong understanding of database technologies (SQL, NoSQL), data modeling, data warehousing, and cloud platforms. Ability resolve issues related to data infrastructure and data management.

Responsibilities Be responsible for the development of the conceptual, logical, and physical data models, the implementation of RDBMS, operational data store (ODS), data marts, and data lakes on target platforms. Implement business and IT data requirements through new data strategies and designs across all data platforms (relational & dimensional - MUST and NoSQL-optional) and data tools (reporting, visualization, analytics, and machine learning). Work with business and application/solution teams to implement data strategies, build data flows, and develop conceptual/logical/physical data models Define and govern data modeling and design standards, tools, best practices, and related developme...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a highly skilled and detail-oriented Data Quality Engineer (Automation) to join our data engineering team. The ideal candidate will have a strong background in ETL automation, data warehouse testing, and cloud data services, along with hands-on experience in test automation frameworks and CI/CD practices. Responsibilities Design, develop, and execute automated test scripts for validating ETL workflows and data pipelines Perform end-to-end data validation, reconciliation, and DWH/DB testing across large datasets Collaborate with data engineers, analysts, and stakeholders to define test strategies and ensure data quality standards are met Develop reusable and scalable test autom...

We are seeking a detail-focused and highly skilled Data Quality Engineer (Automation) to join our data engineering team. The candidate will bring expertise in ETL automation, data warehouse testing, and cloud data services, as well as hands-on knowledge of test automation frameworks and CI/CD methodologies. Responsibilities Define and implement automated test scripts to validate ETL workflows and data pipelines Ensure end-to-end data validation, reconciliation, and testing across large data sets and DWH systems Collaborate with cross-functional teams, including data engineers, analysts, and stakeholders, to establish and meet data quality standards Create reusable and scalable test automatio...

Position : Data Engineer Senior Consultant Location : Hyderabad (Work From Office) Shift : Night Shifts(08:30 PM to 05:30 AM) Two way cab provided Overall 12+ years of experience with at least 5+ years of experience in working with SQL on MySQL and PostgreSQL databases. Must be skilled in working with advanced SQL functions such as CTEs, window functions, procedural SQL, stored procedures, triggers etc. Significant experience in setting up, securing, and administering databases on premise or cloud (MS Azure). Demonstrated experience in managing data and related infrastructure on MS Azure. Demonstrated experience in building and orchestrating performant, secure, and cost-effective data pipeli...

About The Role : Job Title - IT Architect Specialist, AVP Location - Pune, India Role Description This role is for a Senior business functional analyst for Group Architecture. This role will be instrumental in establishing and maintaining bank wide data policies, principles, standards and tool governance. The Senior Business Functional Analyst acts as a link between the business divisions and the data solution providers to align the target data architecture against the enterprise data architecture principles, apply agreed best practices and patterns. Group Architecture partners with each division of the bank to ensure that Architecture is defined, delivered, and managed in alignment with the...