12 Lakehouse Architecture Jobs

You are looking for a skilled and experienced Microsoft Fabric Engineer to join the data engineering team. Your main responsibilities will include designing, developing, and maintaining data solutions using Microsoft Fabric. This will involve working across key workloads such as Data Engineering, Data Factory, Data Science, Real-Time Analytics, and Power BI. In this role, you will need to have a deep understanding of Synapse Data Warehouse, OneLake, Notebooks, Lakehouse architecture, and Power BI integration within the Microsoft ecosystem. Some of your key responsibilities will include designing and implementing scalable and secure data solutions, building and maintaining Data Pipelines usin...

Key Result Areas and Activities: Design and implement Lakehouse architectures using Databricks, Delta Lake, and Apache Spark. Lead the development of data pipelines, ETL/ELT processes, and data integration strategies. Collaborate with business and technical teams to define data architecture standards, governance, and security models. Optimize performance and cost-efficiency of Databricks clusters and jobs. Provide technical leadership and mentorship to data engineers and developers. Integrate Databricks with cloud platforms (Azure, AWS, or GCP) and enterprise systems. Evaluate and recommend tools and technologies to enhance the data ecosystem. Ensure compliance with data privacy and regulato...

Who We Are Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we help clients with total transformation-inspiring complex change, enabling organizations to grow, building competitive advantage, and driving bottom-line impact. To succeed, organizations must blend digital and human capabilities. Our diverse, global teams bring deep industry and functional expertise and a range of perspectives to spark change. BCG delivers solutions through leading-edge management consulting along with technology and design, corporate...

We are looking for a Data Architect with a minimum of 12 years of experience, including at least 4 years in Azure data services. As a Data Architect, you will be responsible for leading the design and implementation of large-scale data solutions on Microsoft Azure. Your role will involve leveraging your expertise in cloud data architecture, data engineering, and governance to create robust, secure, and scalable platforms. Key Skills: - Proficiency in Azure Data Factory, Synapse, Databricks, and Blob Storage - Strong background in Data Modeling and Lakehouse Architecture - Experience with SQL, Python, and Spark - Knowledge of Data Governance, Security, and Metadata Management - Familiarity wi...

Job description Some careers shine brighter than others. If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to...

You are a skilled Data Engineer with expertise in the Azure Data Platform, specifically Azure Databricks, PySpark, Azure Data Factory, and Synapse Analytics. Your primary responsibility will be to design, develop, and optimize large-scale data pipelines and analytics solutions. Your key responsibilities will include developing and optimizing data pipelines using Azure Data Factory for ingestion into Azure Databricks, implementing PySpark transformations in Databricks with a focus on performance tuning, building and maintaining data layers, and ensuring efficient export strategies for reporting and analytics. You will also work with Azure Synapse for advanced querying, transformations, and re...

Job Requirements Job Requirements Role/ Job Title: Data Engineer Function/ Department: Data & Analytics Job Purpose The Data Engineer is responsible for developing day to day data engineering projects and aiding in building the business data collection systems and processing pipelines. Responsible for efficient code for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department. He/She works with Data and Analytics teams in leveraging data with reporting and scientific tools, for example, Tableau, Python, and Spark. The Data Engineer strives to continuously develop most efficient code for data pipe...

LSEG (London Stock Exchange Group) is a prominent provider of financial markets infrastructure worldwide, offering financial data, analytics, news, and index products to a vast customer base spanning over 170 countries. Committed to excellence, we collaborate with our customers throughout the trade lifecycle to assist them in making informed decisions, executing trades, raising capital, and optimizing their operations. With a rich history of over three centuries, cutting-edge technologies, and a global team of 25,000 professionals across 60 countries, we are dedicated to promoting financial stability, driving economic empowerment, and fostering sustainable growth. In the role of Group Direct...

You should have 9+ years of experience and be located in Chennai. You must possess in-depth knowledge of Python and have good experience in creating APIs using FastAPI. It is essential to have exposure to data libraries like Pandas, DataFrame, NumPy etc., as well as knowledge in Apache open-source components. Experience with Apache Spark, Lakehouse architecture, and Open table formats is required. You should also have knowledge in automated unit testing, preferably using PyTest, and exposure in distributed computing. Experience working in a Linux environment is necessary, and working knowledge in Kubernetes would be an added advantage. Basic exposure to ML and MLOps would also be advantageou...

As an ideal candidate for this role, you should possess in-depth knowledge of Python and have good experience in creating APIs using FastAPI. You should also have exposure to data libraries like Pandas, DataFrame, NumPy, as well as knowledge in Apache open-source components and Apache Spark. Familiarity with Lakehouse architecture and Open table formats is also desirable. Additionally, you should be well-versed in automated unit testing, preferably using PyTest, and have exposure to distributed computing. Experience working in a Linux environment is a must, while working knowledge in Kubernetes would be considered an added advantage. Basic exposure to ML and MLOps would also be advantageous ...

Seeking a Principal Data Engineer with the following skills, an excellent opportunity for top-tier coding experts . Experience with building and maintaining a Lakehouse architecture . Advanced expertise in Python and SQL. Expert in leveraging cloud-based data platform ( Snowflake , Databricks ) capabilities to create an enterprise lake house. Advanced expertise with AWS ecosystem Experience using spark for data ingestion/ processing Key Responsibilities Design, develop and maintain scalable data pipelines and architectures to support data ingestion, integration and analytics. Be accountable for technical delivery and take ownership of solutions. Lead a team of senior and junior developers pr...

- exp in data / platform engineering, or cloud architecture - AWS (S3, Redshift, Glue, Lambda, EMR), Kafka - Python, SQL, dbt, Terraform Docker, GitHub Actions, Jenkins - Snowflake, Airflow, Lakehouse Architecture - multi-tenant SaaS platforms