1521 Data Lake Jobs - Page 28

You should have a minimum of 8 years of experience in Microsoft SQL Server administration. Additionally, you should possess experience in MySQL, PostgreSQL, and Oracle database administration. Exposure to Data Lake, Hadoop, and Azure technologies as well as DevOps or ITIL would be beneficial. Your main responsibilities will include optimizing database queries for efficient data retrieval, designing and implementing indexing strategies to improve database performance, monitoring and analyzing query execution plans to identify and resolve bottlenecks, scheduling and executing regular maintenance tasks, implementing automated monitoring systems for database health tracking, proactively diagnosi...

Qualcomm India Private Limited is currently looking for a dedicated Staff IT Data Engineer to become a part of the Enterprise Architecture, Data and Services (EADS) Team. In this role, you will be responsible for designing, developing, and supporting advanced data pipelines within a multi-cloud environment, with a primary focus on Databricks. Your main duties will include facilitating data ingestion from various sources, processing data using a standardized data lake approach, and provisioning data for analytical purposes. The position requires a strong implementation of DevSecOps practices, including Continuous Integration/Continuous Deployment (CI/CD), Infrastructure as Code (IaC), and aut...

Role & responsibilities 4+ years in Data warehousing/ Business Intelligence field Architecture design and implementation experience with medium to complex on-prem to cloud migrations with any of the major cloud platforms (preferably Azure). Minimum 3+ years experience in Azure database offerings [Relational, NoSQL, Datawarehouse. 2+ years hands-on experience in various Azure services preferred Azure Data Factory, Kafka, Azure Data Explorer, Storage, Azure Data Lake, Azure Synapse Analytics, Azure Analysis Services & Databricks. Minimum of 3 years of hands-on database design, modelling and integration experience with relational data sources, such as SQL Server databases, Oracle/MySQL, Azure S...

We are Hiring: Data Engineers Enterprise Data Platforms & Analytics (50+ Openings | 3 - 20 Yrs Exp | Multiple Levels) Location: Mumbai / Pune / Chandigarh / Bengaluru / Gurugram / Noida / New Delhi / Kolkata / Hyderabad . Chennai ( Remote and Hybrid options are available for some roles) Industry: Fortune 500 Client Projects (Staffing via Hatchtra Innotech Pvt. Ltd.) Employment Type: Full-Time / Contract (C2H) About us: Hatchtra is a leading staffing and workforce solutions company, trusted by Fortune 500 organizations and global enterprises to build their technology teams. When you join us, you'll work directly with our Fortune 500 client teams that power mission-critical systems worldwide. ...

Collect, clean, and analyze structured & unstructured data from multiple sources. Develop and maintain Power BI dashboards & reports to provide actionable business insights. Collaborate with Data Engineers to manage & optimize Data Lake architecture. Required Candidate profile 3–6 years of proven experience as a Data Analyst / BI Analyst / Data Engineer Strong proficiency in Power BI (DAX, Power Query, Data Modeling) Hands-on experience with Data Lakes (Azure / AWS / GCP)

Experience in Power BI and QlikSense Experience in solution design, Datalakes, Data extraction Experience in OLAP Notice Period - Immediate to 15 days serving

We are seeking a highly motivated and skilled Salesforce Data Cloud Lead to take charge of designing, developing, and optimizing Data Cloud data models and use cases. You will collaborate closely with cross-functional teams, including Marketing, IT, and Data Analytics, to deliver dynamic content and personalized experiences to our customers. Key Responsibilities: Lead the complete implementation of Salesforce Data Cloud (CDP), encompassing data acquisition, integration, quality assurance, and utilization. Configure and execute data-driven segmentation strategies to ensure precise audience targeting and content delivery. Design, document, and implement data models, data pipelines, and transfo...

Your role as a Snowflake Consultant involves driving the implementation of Data warehousing and Data Lake solutions within our center of excellence. You will be responsible for creating data pipelines for extraction, load, and transformations, ensuring data quality, accuracy, and integrity. Your strong technical knowledge of Data Lake, Data warehouse, DevOps, and data CI/CD pipeline in Snowflake will be crucial. Designing and developing data models in Snowflake, along with ensuring successful data loads, Data Tagging, Data profiling, Data cataloging, and Meta data management will also be part of your responsibilities. Experience with various Snowflake connectors and data extraction, as well ...

Data Lake PMO Job Description Experience Required 7-10 years Location - Mumbai 1) Oversee Data lake project planning, execution, and monitoring to meet organizational objectives. 2) Coordinate with cross-functional teams including data engineers, analysts, and stakeholders for project delivery. 3) Ensure Data governance, quality, and compliance standards are maintained across data lake projects. 4) Effectively manage resource allocation, capacity planning, and project schedules. 5) Facilitate clear communication channels between stakeholders and technical teams to align on goals and expectations. 6) Proven experience in effectively utilizing JIRA / other Project Management tools for project ...

Design and maintain data pipelines using Azure (Data Factory, Databricks, Synapse, SQL). Strong PySpark and Python skills required. Ensure performance, security, and compliance. Collaborate with teams to deliver scalable data solutions.

Role Overview : Seasoned Databricks Senior Platform Engineer who can implement scalable, end-to-end data solutions on Databricks over AWS. This role is pivotal in building the foundational infrastructure for data products, ensuring data quality, and enabling machine learning capabilities Key Responsibilities : - Design and implement end-to-end ETL pipelines in Databricks, sourcing data from Amazon S3 and Amazon RDS and other sources. - Design and deploy Databricks workspaces and clusters optimized for performance, scalability, and cost-efficiency. - Set up data quality frameworks and anomaly detection mechanisms to ensure reliability and trust in data products. - Enable machine learning capa...

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Data Engineering Good to have skills : Oracle Procedural Language Extensions to SQL (PLSQL), Google BigQuery, Google Cloud Platform ArchitectureMinimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing...

Role Overview: We are seeking street-smart and technically strong Senior Data Engineers / Leads who can take ownership of designing and developing cutting-edge data and AI platforms using Azure-native technologies and Databricks. You will play a critical role in building scalable data pipelines, modern data architectures, and intelligent analytics solutions. Key Responsibilities: Design and implement scalable, metadata-driven frameworks for data ingestion, quality, and transformation across both batch and streaming datasets. Develop and optimize end-to-end data pipelines to process structured and unstructured data, enabling the creation of analytical data products. Build robust exception han...

My linkedin linkedin.com/in/yashsharma1608. contract period - 6-12month payroll will be - ASV consulting , my company client - will disclose after 1 round Job location - Gurgaon - onsite(WFO) budget - upto 1lpa/month , depending on last (relevant hike) Exprnce - 3+ JD is About the Role We are looking for a skilled AWS Data Engineer with strong hands-on experience in AWS Glue and AWS analytics services. The candidate will be responsible for designing, building, and optimizing scalable data pipelines and ETL processes that support advanced analytics and business intelligence requirements. Key Responsibilities Design and develop ETL pipelines using AWS Glue, PySpark, and AWS services (Lambda, S...

Senior data engineers at Thoughtworks are engineers who build, maintain and test the software architecture and infrastructure for managing data applications. They are involved in developing core capabilities which include technical and functional data platforms. They are the anchor for functional streams of work and are accountable for timely delivery. They work on the latest big data tools, frameworks and offerings (data mesh, etc.), while also being involved in enabling credible and collaborative problem solving to execute on a strategy. Job responsibilities You will develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solution...

Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On projects, they will be leading the design of technical solutions, or perhaps overseeing a program inception to build a new product. Alongside hands-on coding, they are leading the team to implement the solution. Job responsibilities You will lead and manage data engineering projects from inception to completion, including goal-setting, ...

About The Role About The Role Design, develop, and maintain ETL processes using Pentaho Data Integration (Kettle) . Extract data from various sources including databases, flat files, APIs, and cloud platforms. Transform and cleanse data to meet business and technical requirements. Load data into data warehouses, data lakes, or other target systems. Monitor and optimize ETL performance and troubleshoot issues. Collaborate with data architects, analysts, and business stakeholders to understand data requirements. Ensure data quality, integrity, and security throughout the ETL lifecycle.Document ETL processes, data flows, and technical specifications. About The Role - Grade Specific Focus on Ind...

Responsibilities : Participate in the customers system design meetings and collect the functional/technical equirements. Build up data pipelines for consumption by the Data Science team. Experience with ETL process and tools. Clear understanding and experience with Python and PySpark or Spark and SCALA, with HIVE, irflow, Impala, and Hadoop and RDBMS architecture. Experience in writing Python programs and SQL queries. Experience in SQL Query tuning. Experienced in Shell Scripting (Unix/Linux). Build and maintain data pipelines in Spark/PySpark with SQL and Python or SCALA. Knowledge of Cloud (Azure/AWS/GCP, etc..) technologies. Good to have knowledge of Kubernetes, CI/CD concepts, Apache Kaf...

Responsibilities : Designing and implementing data pipelines to collect, clean, and transform data from various sources Building and maintaining data storage and processing systems, such as databases, data warehouses, and data lakes Ensuring data is properly secured and protected Developing and implementing data governance policies and procedures Collaborating with business analysts, data analysts, data scientists, and other stakeholders to understand their data needs and ensure they have access to the data they need Sharing knowledge with the wider business, working with other BAs and technology teams to make sure processes and ways of working is documented. Collaborate with Big Data Soluti...

Software Architect Data & Platform Chennai (Onsite) | Full-time | FYNXT About FYNXT FYNXT is redefining the global broking industry through cutting-edge fintech SaaS solutions. We build multi-asset trading platforms, seamless payment ecosystems, and scalable customer engagement tools for brokers worldwide. Our culture combines startup agility with structured execution – ownership-driven, innovative, and impact-focused. The Role We are looking for a Software Architect (Data & Platform) to define and drive the technical vision of our real-time streaming infrastructure and trading ecosystem . This is a hands-on role where you’ll architect mission-critical financial systems that process billions...

TECHNICAL SKILLS AND EXPERIENCE Most important: 7+ years professional experience as a data engineer, with at least 4 utilizing cloud technologies. Proven experience building ETL or ETL data pipelines with Databricks either in Azure or AWS using PySpark language. Strong experience with the Microsoft Azure Data Stack (Databricks, Data Lake Gen2, ADF etc.) Strong SQL skills and proficiency in Python adhering to standards such as PEP Proven experience with unit testing and applying appropriate testing methodologies using libraries such as Pytest, Great Expectations, or similar. Demonstrable experience with CICD including release and test automation tools and processes such as Azure Devops, Terra...

BDO RISE -Senior Data Engineer I - Bangalore/ Hyderabad The Senior Data Engineer I, Tax Product Development, works with the Data Engineering and Integration team to develop and support BDOs Tax Data Warehouse and Tax Data Lake, manage and maintain data ingestion and Extract/Transform/Load (ETL) processes, as well as provide support for the Power BI business intelligence platform and enterprise applications supporting the tax practice. You will be responsible for implementing and participate in delivery of Data related engagements, including Data modeling, Data cleansing, Data enrichments and Analytical solutions to our clients. Your experience will be instrumental in optimizing data processe...

As a Software Engineer-Full Stack Developer at Barclays, you will embark on a transformative journey to spearhead the evolution of the digital landscape, driving innovation and excellence. Your primary responsibility will be to harness cutting-edge technology to revolutionize digital offerings, ensuring unparalleled customer experiences. You will be part of the Operational Support Systems (OSS) Platform Engineering team within the Network OSS & Tools functional unit at Barclays, responsible for designing, building, and running the underlying OSS infrastructure and toolchain across cloud and on-premises environments. To excel in this role, you should possess demonstrable expertise in front-en...

As a professional in this role, you will be responsible for understanding the automation and data analytics requirements from the business and coordinating with developers to provide technical and functional assistance while estimating the business value for the organization. Your technical expertise should include the ability to create, design, or structure the data architecture in standard environment databases, data lakes, or cloud platforms such as GCP with a focus on data security. You will also need to structure large sets of data into easy and scalable formats, perform data integration and normalization on master data, and develop technical codes to convert business logics into data w...

As a Data Architect, you will leverage your 10+ years of relevant work experience to lead Data related projects focused on Reporting and Analytics. Your responsibilities will include designing, building, and maintaining scalable data lake and data warehouse solutions in the cloud, specifically on Google Cloud Platform (GCP). You will be expected to excel in gathering business requirements, analyzing business needs, and defining the BI/DW architecture to deliver technical solutions for complex business and technical requirements. Your role will involve creating solution prototypes, participating in technology selection, conducting proof of concepts (POC), and delivering technical presentation...