999 Data Lake Jobs - Page 19

Job Title:Data Engineer Lead / Architect (ADF)Experience9-12YearsLocation:Remote / Hybrid : Role and ResponsibilitiesTalk to client stakeholders, and understand the requirements for building their data warehouse / data lake / data Lakehouse. Design, develop and maintain data pipelines in Azure Data Factory (ADF) for ETL from on-premise and cloud based sources Design, develop and maintain data warehouses and data lakes in Azure Run large data platform and other related programs to provide business intelligence support Design and Develop data models to support business intelligence solutions Implement best practices in data modelling and data warehousing Troubleshoot and resolve issues related...

Job Description_ Data Engineer _ TransOrg Analytics Why would you like to join us? TransOrg Analytics specializes in Data Science, Data Engineering and Generative AI, providing advanced analytics solutions to industry leaders and Fortune 500 companies across India, US, APAC and the Middle East. We leverage data science to streamline, optimize, and accelerate our clients' businesses. Visit at www.transorg.com to know more about us. Responsibilities: Design, develop, and maintain robust data pipelines using Azure Data Factory and Databricks workflows. Develop an integrated data solution in Snowflake to unify data. Implement and manage big data solutions using Azure Databricks. Design and maint...

ETL Tester with 3.6+ Years of Cloud Data testing experience • Experience in data migration and ETL transformation testing • Exposure to Azure Data lakes and data bricks is must • Develop technical specifications for data loading rules • Mentor and provide technical guidance for Software Engineers] • Adhere to and produce artifacts (technical specifications, architectural specifications, data models, installation and qualification guides, and release notes) for the software development process. • Provide application support for existing products • Help with release management, using versioning tools such as SVN, and executing live deployments • Demonstrated ability to liaise with multiple sta...

ETL Tester with 3.6+ Years of Cloud Data testing experience • Experience in data migration and ETL transformation testing • Exposure to Azure Data lakes and data bricks is must • Develop technical specifications for data loading rules • Mentor and provide technical guidance for Software Engineers] • Adhere to and produce artifacts (technical specifications, architectural specifications, data models, installation and qualification guides, and release notes) for the software development process. • Provide application support for existing products • Help with release management, using versioning tools such as SVN, and executing live deployments • Demonstrated ability to liaise with multiple sta...

Azure, Linux OS. Work on advanced technical principles, theories, and concepts. 2. Work closely with geo aligned Tower leads, SMEs and Build managers depending on the stage of opportunity. 3. Work on Azure Expertise: IaaS, PaaS, VM Migrations, V Net, Traffic Manager, Azure Cloud Services, SQL Azure, Active Directory, ADFS, Data Factory, Data Lake, HD Insights, ExpressRoute, Power Shell, OMS, Security Center, Service bus, blob storages. 4. Work on MS Servers: Windows 2000/ 2003/ 2008/ 2012, MOSS 2007, SharePoint 2010/2013, Office 365, Windows Cluster Server, SQL Server, IIS Server, File Server, Proxy Server, Exchange Server, SMS server, Terminal Server. 5. Work on Networking: LB, TCP/ IP Conf...

The project duration for this role is 6 months with a monthly rate of 1.60 Lac. The ideal candidate should possess 4-7 years of experience and the work location is in Bangalore with a Hybrid setup. Key Responsibilities: - Demonstrated strong proficiency in Python, LLMs, Lang Chain, Prompt Engineering, and related Gen AI technologies. - Proficiency in working with Azure Databricks. - Ability to showcase strong analytical skills, problem-solving capabilities, and effective stakeholder communication. - A solid understanding of data governance frameworks, compliance requirements, and internal controls. - Hands-on experience in data quality rule development, profiling, and implementation. - Famil...

Responsibilities for GCP Data Engineer : Job Accountabilities - Hands on Experience in GCP Data Components (BigQuery / Data fusion / Cloudsql etc) Data Lake and Data Warehouse Understanding Manage DevOps lifecycle of project (Code Repository, Build, Release) Good to have : end to end BI Landscape knowledge Participate in unit and integration testing Interaction with business users for Requirement understanding and UAT Understanding of Data Security and Data Compliance Agile Understanding Project Documentation Understanding Good SQL Knowledge Certification (Good to have) Domain Knowledge of Different Industry Sector Mandatory skill sets : GCP

As a Software Engineer- Operational Support Systems at Barclays, you will embark on a transformative journey, spearheading the evolution of the digital landscape and driving innovation and excellence. Your primary responsibility will be the design, build, and maintenance of the underlying OSS infrastructure and toolchains necessary to run the Barclays Global Network, deployed across cloud and On-Prem environments. To excel in this role, you should possess expertise in both front-end and back-end development. Proficiency in Java (Java 17+) and the Spring Ecosystem (Spring MVC, Data JPA, Security, etc.), along with strong SQL and NoSQL integration skills, are essential. Additionally, you shoul...

Architect & manage data solutions using Snowflake & advanced SQL Design & implement data pipelines, data warehouses, & data lakes, ensuring efficient data transformation Develop best practices for data security, access control, & compliance Required Candidate profile Exp 8-14 yrs Strong data architect SQL& Snowflake exp must Collaborate with cross-functional teams, integrate & translate them into robust data architectures Manufacturing industry exp is a must

Develop and maintain data pipelines, ETL/ELT processes, and workflows to ensure the seamless integration and transformation of data. Architect, implement, and optimize scalable data solutions. Required Candidate profile Work closely with data scientists, analysts, and business stakeholders to understand requirements and deliver actionable insights. Partner with cloud architects and DevOps teams

Overall Responsibilities: Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data integrity and accuracy. Data Ingestion: Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP. Data Transformation and Processing: Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements. Performance Optimization: Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and r...

About the Role We are seeking a highly skilled and experienced Data Architect to join our team. The ideal candidate will have at least 18 years of experience in Data engineering and Analytics and a proven track record of designing and implementing complex data solutions. As a senior principal data architect, you will be expected to design, create, deploy, and manage Blackbauds data architecture. This role has considerable technical influence within the Data Platform, Data Engineering teams, and the Data Intelligence Center of Excellence at Blackbaud. This individual acts as an evangelist for proper data strategy with other teams at Blackbaud and assists with the technical direction, specific...

About the Role: Grade Level (for internal use): 10 The Team You will be an expert contributor and part of the Rating Organizations Data Services Product Engineering Team. This team, who has a broad and expert knowledge on Ratings organizations critical data domains, technology stacks and architectural patterns, fosters knowledge sharing and collaboration that results in a unified strategy. All Data Services team members provide leadership, innovation, timely delivery, and the ability to articulate business value. Be a part of a unique opportunity to build and evolve S&P Ratings next gen analytics platform. Responsibilities Design and implement innovative software solutions to enhance S&P Rat...

Overview Enterprise Data Operations Analyst Job OverviewAs an Analyst, Data Modeling , your focus would be to partner with D&A Data Foundation team members to create data models for Global projects. This would include independently analyzing project data needs, identifying data storage and integration needs/issues, and driving opportunities for data model reuse, satisfying project requirements. Role will advocate Enterprise Architecture, Data Design, and D&A standards, and best practices. You will be performing all aspects of Data Modeling working closely with Data Governance, Data Engineering and Data Architects teams. As a member of the data modeling team, you will create data models for v...

Overview As Senior Analyst, Data Modeling, your focus would be to partner with D&A Data Foundation team members to create data models for Global projects. This would include independently analyzing project data needs, identifying data storage and integration needs/issues, and driving opportunities for data model reuse, satisfying project requirements. Role will advocate Enterprise Architecture, Data Design, and D&A standards, and best practices. You will be performing all aspects of Data Modeling working closely with Data Governance, Data Engineering and Data Architects teams. As a member of the data modeling team, you will create data models for very large and complex data applications in p...

Overview This person will look after the environment management of the salesforce orgs and will also be able to handle the deployments within multiple salesforce orgs. Responsibilities Oversee Salesforce Data Cloud environments across development, staging, and production. Define best practices for environment setup, security, and governance. Manage data pipelines, ingestion processes, and harmonization rules for efficient data flow. Establish role-based access control (RBAC) to ensure data security and compliance. Monitor data processing jobs, ingestion performance, and data harmonization. Ensure compliance with GDPR, CCPA, and other data privacy regulations Establish CI/CD pipelines using t...

Overview We are seeking a highly skilled and experienced Azure Data Engineer to join our dynamic team. In this critical role, you will be responsible for designing, developing, and maintaining robust and scalable data solutions on the Microsoft Azure platform. You will work closely with data scientists, analysts, and business stakeholders to translate business requirements into effective data pipelines and data models. Responsibilities Design, develop, and implement data pipelines and ETL/ELT processes using Azure Data Factory, Azure Databricks, and other relevant Azure services. Develop and maintain data lakes and data warehouses on Azure, including Azure Data Lake Storage Gen2 and Azure Sy...

Overview As an Analyst, Data Modeling, your focus would be to partner with D&A Data Foundation team members to create data models for Global projects. This would include independently analyzing project data needs, identifying data storage and integration needs/issues, and driving opportunities for data model reuse, satisfying project requirements. Role will advocate Enterprise Architecture, Data Design, and D&A standards, and best practices. You will be performing all aspects of Data Modeling working closely with Data Governance, Data Engineering and Data Architects teams. As a member of the data modeling team, you will create data models for very large and complex data applications in publi...

Overview Enterprise Data Operations Sr Analyst L08 Job OverviewAs Senior Analyst, Data Modeling , your focus would be to partner with D&A Data Foundation team members to create data models for Global projects. This would include independently analyzing project data needs, identifying data storage and integration needs/issues, and driving opportunities for data model reuse, satisfying project requirements. Role will advocate Enterprise Architecture, Data Design, and D&A standards, and best practices. You will be performing all aspects of Data Modeling working closely with Data Governance, Data Engineering and Data Architects teams. As a member of the data modeling team, you will create data m...

Overview Primary focus would be to lead development work within Azure Data Lake environment and other related ETL technologies, with the responsibility of ensuring on time and on budget delivery; Satisfying project requirements, while adhering to enterprise architecture standards. Role will lead key data lake projects and resources, including innovation related initiatives (e.g. adoption of technologies like Databricks, Presto, Denodo, Python,Azure data factory; database encryption; enabling rapid experimentation etc.)). This role will also have L3 and release management responsibilities for ETL processes Responsibilities Lead delivery of key Enterprise Data Warehouse and Azure Data Lake pro...

Overview PepsiCo operates in an environment undergoing immense and rapid change. Big-data and digital technologies are driving business transformation that is unlocking new capabilities and business innovations in areas like eCommerce, mobile experiences and IoT. The key to winning in these areas is being able to leverage enterprise data foundations built on PepsiCos global business scale to enable business insights, advanced analytics, and new product development. PepsiCos Data Management and Operations team is tasked with the responsibility of developing quality data collection processes, maintaining the integrity of our data foundations, and enabling business leaders and data scientists a...

Overview PepsiCo operates in an environment undergoing immense and rapid change.Big-data and digital technologies are driving business transformation that is unlocking new capabilities and business innovations in areas like eCommerce, mobile experiences and IoT. The key to winning in these areas is being able to leverage enterprise data foundations built on PepsiCos global business scale to enable business insights, advanced analytics, and new product development. PepsiCos Data Management and Operations team is tasked with the responsibility of developing quality data collection processes, maintaining the integrity of our data foundations, and enabling business leaders and data scientists ac...

Overview As an Analyst, Data Modeling, your focus would be to partner with D&A Data Foundation team members to create data models for Global projects. This would include independently analyzing project data needs, identifying data storage and integration needs/issues, and driving opportunities for data model reuse, satisfying project requirements. Role will advocate Enterprise Architecture, Data Design, and D&A standards, and best practices. You will be performing all aspects of Data Modeling working closely with Data Governance, Data Engineering and Data Architects teams. As a member of the data modeling team, you will create data models for very large and complex data applications in publi...

Overview As a member of the data engineering team, you will be the key technical expert developing and overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be an empowered member of a team of data engineers who build data pipelines into various source systems, rest data on the PepsiCo Data Lake, and enable exploration and access for analytics, visualization, machine learning, and product development efforts across the company. As a member of the data engineering team, you will help lead the development of very large and complex data applications into public cloud environments d...

Overview FOBO businesses in Europe, AMESA and APAC have migrated its planning capability from XLS to MOSAIC, an integrated and digital planning tool, in a step forward towards reaching the Financial Planning 2025 Vision. However, the underlaying FOBO operating model limits our ability to capture benefits given the high attrition and lack of process standardization. To become more capable, agile, and efficient a fundamental change in the way we do FOBO Financial Planning is required, which will be addressed by establishing the FOBO Planning Central (FPC). FPC evolves the GBS approach, pivoting from a geography focus to a process focus, and allows BUs to concentrate their attention on the Bott...