999 Data Lake Jobs - Page 17

About The Role Role Overview:Develop efficient SQL queries and maintain views, models, and data structures across federated and transactional DB to support analytics and reporting. SQL (Advanced) Python for data exploration and scripting Shell scripting for lightweight automationKey Responsibilities: Write complex SQL queries for data extraction and transformations Build and maintain views, materialized views, and data models Enable efficient federated queries and optimize joins across databases Support performance tuning, indexing, and query optimization effortsPrimary: Expertise in MS SQL Server / Oracle DB / PostgresSQL , Columnar DBs like DuckDB , and federated data access Good understan...

Role & responsibilities Key Responsibilities: Leadership & Strategy Lead and mentor a team of MDM Specialists, ensuring accountability, performance, and continuous development. Own the execution of MDM strategy in India, aligned with global standards and business goals. Serve as the subject matter expert for master data across the organization, guiding stakeholders and ensuring alignment. Master Data Operations & Oversight Supervise day-to-day MDM activities, including data creation, enrichment, and lifecycle management across multiple domains. Monitor and ensure adherence to SLAs, data quality KPIs, and change request turnaround times. Oversee the execution and enforcement of data governanc...

Who We Are At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward – always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities. The Role Join the innovative team at Kyndryl as a Client Technical Solutioner and unlock your potential to shape the future of technology solutions. As a key player in our organization, you will embark on an exciting journey where you get to work closely with customers, understand their unique challenges, and provide them with cutting-edge technical sol...

Key Skills: . NET Core, C#, Azure Kubernetes Service (AKS), Databricks, Delta Lake, Spark, Data Lake, Palantir Foundry, GenAI, CI/CD, Agile, ITSM, SaaS, backend development, and security remediation. Roles & Responsibilities: Design, build, and enhance P&C solutions technology architecture and engineering. Recommend and implement alternative solutions to business challenges to streamline processes and create competitive advantage. Prioritize efforts based on business benefits and drive execution to ensure tangible outcomes. Collaborate with Engineering Director, P&C Solutions Engineering Leads, Product Owners, Technology Platform Leads, and Operations teams. Lead engineering for specific pro...

As a Cloud Architect with expertise in Azure and Snowflake, you will be responsible for designing and implementing secure, scalable, and highly available cloud-based solutions on AWS and Azure Cloud. Your role will involve utilizing your experience in Azure Databricks, ADF, Azure Synapse, PySpark, and Snowflake Services. Additionally, you will participate in pre-sales activities, including RFP and proposal writing. Your experience with integrating various data sources with Data Warehouse and Data Lake will be crucial for this role. You will also be expected to create Data warehouses and data lakes for Reporting, AI, and Machine Learning purposes, while having a solid understanding of data mo...

As a Microsoft Fabric Professional at YASH Technologies, you will be responsible for leveraging your 12+ years of experience in Microsoft Azure Data Engineering to drive analytical projects. Your expertise will be crucial in designing, developing, and deploying high-volume ETL pipelines using Azure, Microsoft Fabric, and Databricks for complex models. Your hands-on experience with Azure Data Factory, Databricks, Azure Functions, Synapse Analytics, Data Lake, Delta Lake, and Azure SQL Database will be utilized for managing and processing large-scale data integrations. In this role, you will be expected to optimize Databricks clusters and manage workflows to ensure cost-effective and high-perf...

As a Cloud & AI Solution Engineer at Microsoft, you will be part of a dynamic team that is at the forefront of innovation in the realm of databases and analytics. Your role will involve working on cutting-edge projects that leverage the latest technologies to drive meaningful impact for commercial customers. If you are insatiably curious and deeply passionate about tackling complex challenges in the era of AI, this is the perfect opportunity for you. In this role, you will play a pivotal role in helping enterprises unlock the full potential of Microsoft's cloud database and analytics stack. You will collaborate closely with engineering leaders and platform teams to accelerate the Fabric Data...

You are an experienced Data Engineering Manager responsible for leading a team of 10+ engineers in Chennai, Tamil Nadu, India. Your primary role is to build scalable data marts and Power BI dashboards to measure marketing campaign performance. Your deep expertise in Azure, Microsoft Fabric, and Power BI, combined with strong leadership skills, enables you to drive data initiatives that facilitate data-driven decision-making for the marketing team. Your key responsibilities include managing and mentoring the data engineering and BI developer team, overseeing the design and implementation of scalable data marts and pipelines, and leading the development of insightful Power BI dashboards. You c...

As a Junior Architect at Contata Solutions, you will be responsible for various aspects of data architecture, including the design, implementation, and maintenance of data systems. You are expected to continuously develop technical expertise, learn best practices, and evolve as a professional under the guidance of senior team members. Your primary responsibilities will include working with ETL on data warehouse or big data projects, utilizing cloud services like Azure, GCP, or AWS for data-related services, programming using tools on the cloud, and possessing strong SQL skills to efficiently solve complex problems. You will also be involved in identifying and resolving performance and scalab...

Job Summary We are seeking a highly skilled Global Data Engineer at the Senior Associate level to support our domestic Digital Finance Transformation team. The ideal candidate will possess advanced expertise in SQL, Alteryx, Power BI, and Python, along with strong experience in current finance use cases and leading practices related to artificial intelligence (AI) and machine learning (ML). This role will focus on building robust data pipelines, developing analytics and reporting solutions, and integrating AI/ML models to drive business insights and process automation. Job Duties Key Responsibilities Collaborate closely with the stateside Digital/Finance Transformation team to understand bus...

Role: Azure Specialist-DBT Location: Bangalore Mode: Remote Education and Work Experience Requirements: •Overall, 5 to 9 years of experience in IT Industry. Min 6 years of experience working on Data Engineering. Translate complex business requirements into analytical SQL views using DBT Support data ingestion pipelines using Airflow and Data Factory Develop DBT macros to enable scalable and reusable code automation Mandatory Skills: Strong experience with DBT (Data Build Tool)( (or strong SQL / relational DWH knowledge)-Must have Proficiency in SQL and strong understanding of relational data warehouse concept Hands-on experience with Databricks (primarily Databricks SQL) good to have Familia...

Role Description Provides leadership for the overall architecture, design, development, and deployment of a full-stack cloud native data analytics platform. Designing & Augmenting Solution architecture for Data Ingestion, Data Preparation, Data Transformation, Data Load, ML & Simulation Modelling, Java BE & FE, State Machine, API Management & Intelligence consumption using data products, on cloud Understand Business Requirements and help in developing High level and Low-level Data Engineering and Data Processing Documentation for the cloud native architecture Developing conceptual, logical and physical target-state architecture, engineering and operational specs. Work with the customer, user...

Role Description : As a Senior Big Data Platform Engineer at Incedo, you will be responsible for designing and implementing big data platforms to support large-scale data integration projects. You will work with data architects and data engineers to define the platform architecture and build the necessary infrastructure. You will be skilled in big data technologies such as Hadoop, Spark, and Kafka and have experience in cloud computing platforms such as AWS or Azure. You will be responsible for ensuring the performance, scalability, and security of the big data platform and troubleshooting any issues that arise. Roles & Responsibilities: Designing, developing and maintaining large-scale big ...

Role Description : As a Senior Big Data Platform Engineer at Incedo, you will be responsible for designing and implementing big data platforms to support large-scale data integration projects. You will work with data architects and data engineers to define the platform architecture and build the necessary infrastructure. You will be skilled in big data technologies such as Hadoop, Spark, and Kafka and have experience in cloud computing platforms such as AWS or Azure. You will be responsible for ensuring the performance, scalability, and security of the big data platform and troubleshooting any issues that arise. Roles & Responsibilities: Designing, developing and maintaining large-scale big ...

Role Description : As a Senior Big Data Platform Engineer at Incedo, you will be responsible for designing and implementing big data platforms to support large-scale data integration projects. You will work with data architects and data engineers to define the platform architecture and build the necessary infrastructure. You will be skilled in big data technologies such as Hadoop, Spark, and Kafka and have experience in cloud computing platforms such as AWS or Azure. You will be responsible for ensuring the performance, scalability, and security of the big data platform and troubleshooting any issues that arise. Roles & Responsibilities: Designing, developing and maintaining large-scale big ...

Role Description : As a Senior Big Data Platform Engineer at Incedo, you will be responsible for designing and implementing big data platforms to support large-scale data integration projects. You will work with data architects and data engineers to define the platform architecture and build the necessary infrastructure. You will be skilled in big data technologies such as Hadoop, Spark, and Kafka and have experience in cloud computing platforms such as AWS or Azure. You will be responsible for ensuring the performance, scalability, and security of the big data platform and troubleshooting any issues that arise. Roles & Responsibilities: Designing, developing and maintaining large-scale big ...

Piller Soft Technology is looking for Lead Data Engineer to join our dynamic team and embark on a rewarding career journey Designing and developing data pipelines: Lead data engineers are responsible for designing and developing data pipelines that move data from various sources to storage and processing systems. Building and maintaining data infrastructure: Lead data engineers are responsible for building and maintaining data infrastructure, such as data warehouses, data lakes, and data marts. Ensuring data quality and integrity: Lead data engineers are responsible for ensuring data quality and integrity, by setting up data validation processes and implementing data quality checks. Managing...

Should Coordinate with team members, Paris counterparts and work independently. Responsible Accountable to deliver Functional Specifications, Wireframe docs, RCA, Source to Target Mapping, Test Strategy Doc any other BA artifacts as demanded by the project delivery Understanding the business requirements, discuss with Business users. Should be able to write mapping documents from User stories. Follow project documentation standards. should have very good knowledge of Hands - on SQL. Analysis the Production data and derive KPI for business users Well verse with Jira use for project work. Profile required 5+ years of experience in JAVA / Data based projects (Datawarehouse or Datalake) preferab...

5-9 years In Data Engineering, software development such as ELT/ETL, data extraction and manipulation in Data Lake/Data Warehouse environment Expert level Hands to the following: Python, SQL, PySpark DBT and Apache Airflow Postgres/others RDBMS DevOps, Jenkins, CI/CD Data Governance and Data Quality frameworks Data Lakes, Data Warehouse AWS services including S3, SNS, SQS, Lambda, EMR, Glue, Athena, EC2, VPC etc. Source code control - GitHub, VSTS etc. Mandatory Competencies Data on Cloud - Azure Data Lake (ADL) Python - Python Big Data - PySpark DevOps - CI/CD DevOps - Jenkins Beh - Communication Data on Cloud - AWS S3 Database - PostgreSQL DevOps - Github

Design, implement, and maintain data pipelines for processing large datasets, ensuring data availability, quality, and efficiency for machine learning model training and inference. Collaborate with data scientists to streamline the deployment of machine learning models, ensuring scalability, performance, and reliability in production environments. Develop and optimize ETL (Extract, Transform, Load) processes, ensuring data flow from various sources into structured data storage systems. Automate ML workflows using ML Ops tools and frameworks (e.g., Kubeflow, MLflow, TensorFlow Extended (TFX)). Ensure effective model monitoring, versioning, and logging to track performance and metrics in a pro...

Collaborate with business stakeholders and other technical team members to acquire data sources that are most relevant to business needs and goals. Demonstrate deep technical and domain knowledge of relational and non-relation databases, Data Warehouses, Data lakes among other structured and unstructured storage options. Determine solutions that are best suited to develop a pipeline for a particular data source. Develop data flow pipelines to extract, transform, and load data from various data sources in various forms, including custom ETL pipelines that enable model and product development. Write custom scripts to extract data from unstructured/semi-structured sources. Provide clear documen...

As a Senior Big Data Platform Engineer at Incedo, you will be responsible for designing and implementing big data platforms to support large-scale data integration projects. You will work with data architects and data engineers to define the platform architecture and build the necessary infrastructure. You will be skilled in big data technologies such as Hadoop, Spark, and Kafka and have experience in cloud computing platforms such as AWS or Azure. You will be responsible for ensuring the performance, scalability, and security of the big data platform and troubleshooting any issues that arise. Roles & Responsibilities: Designing, developing and maintaining large-scale big data platforms usin...

As a Data Engineer, you will be responsible for designing, developing, and maintaining robust ETL pipelines using Azure Data Factory (ADF) to support complex insurance data workflows. You will integrate and extract data from various Guidewire modules (PolicyCenter, BillingCenter, ClaimCenter) to ensure data quality, integrity, and consistency. Building reusable components for data ingestion, transformation, and orchestration across Guidewire and Azure ecosystems will be a key part of your role. Your responsibilities will also include optimizing ADF pipelines for performance, scalability, and cost-efficiency while following industry-standard DevOps and CI/CD practices. Collaborating with solu...

As a Developer (Japanese-speaking), you will be responsible for supporting projects in the Japan region by utilizing your technical expertise and Japanese language proficiency. This hands-on role will primarily focus on backend development, data engineering, and cloud technologies. Candidates with prior experience working in Japan or those looking to relocate from Japan are highly desired. Your key responsibilities will include designing and developing ETL/ELT pipelines using Azure or equivalent cloud platforms, collaborating with Japanese-speaking stakeholders and internal teams, working with Azure Data Factory, Synapse, Data Lake, and Power BI for data integration and reporting, as well as...

You are a seasoned Confluent & Oracle EBS Cloud Engineer with over 10 years of experience, responsible for leading the design and implementation of scalable, cloud-native data solutions. Your role involves modernizing enterprise data infrastructure, driving real-time data streaming initiatives, and migrating legacy ERP systems to AWS-based platforms. Your key responsibilities include architecting and implementing cloud-based data platforms using AWS services such as Redshift, Glue, DMS, and Data Lake solutions. You will lead the migration of Oracle E-Business Suite or similar ERP systems to AWS while ensuring data integrity and performance. Additionally, you will design and drive the impleme...