318 Delta Lake Jobs - Page 3

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Associate Job Description & Summary At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals. In business intelligence at PwC, you will focus on leveraging data and analytics to provide strategic insights and drive informed decision-making for clients. You will develop and implement innovative solutions to optimise business performance and enhance competitive advantage. *Why PWC At PwC, you will be pa...

Key Responsibilities: Design and develop the migration strategies and processes Collaborate with stakeholders to understand business requirements and technical challenges. Analyze current data and scope for optimization during the migration process. Define the architecture and roadmap for cloud-based data and analytics transformation on Databricks. Design, implement, and optimize scalable, high-performance data architectures using Databricks. Build and manage data pipelines and workflows within Databricks. Ensure that best practices for security, scalability, and performance are followed. Implement Databricks solutions that enable machine learning, business intelligence, and data science wor...

As an Advisory Consultant at Dell Technologies, you will play a crucial role in delivering consultative business and technical services for complex customer-facing consulting engagements related to digital transformation. Your responsibilities will involve collaborating with Global Pre-Sales, Account Management, and Solutioning Teams to deploy, administer, and configure digital transformation software stacks. Being one of the senior technical members in the Digital Transformation Practice, you will earn customer trust through competence, technical acumen, consulting expertise, and partnership. You will guide and oversee other team members, providing technical grooming activities for their sk...

Key Skills: Spark, Azure Databricks, Apache Spark, Python, Delta Lake, Kafka, Snowflake, Qlik, CI/CD, Azure Roles & Responsibilities: Design, develop, and maintain scalable data pipelines using Azure Databricks and Spark. Implement data processing solutions with Delta Lakes and manage multiple file formats. Collaborate with cross-functional teams to ensure data quality and integrity. Utilize CI/CD practices for efficient deployment and management of data solutions. Work with Snowflake and Qlik for data visualization and reporting. Leverage Kafka or similar streaming technologies for real-time data processing. Ensure compliance with data governance and security standards in Azure. Experience ...

Why Join Us Are you inspired to grow your career at one of India's Top 25 Best Workplaces in IT industry Do you want to do the best work of your life at one of the fastest growing IT services companies Do you aspire to thrive in an award-winning work culture that values your talent and career aspirations It's happening right here at Iris Software. About Iris Software At Iris Software, our vision is to be our client's most trusted technology partner, and the first choice for the industry's top professionals to realize their full potential. With over 4,300 associates across India, U.S.A, and Canada, we help our enterprise clients thrive with technology-enabled transformation across financial s...

Why Join Us Are you inspired to grow your career at one of India's Top 25 Best Workplaces in IT industry Do you want to do the best work of your life at one of the fastest growing IT services companies Do you aspire to thrive in an award-winning work culture that values your talent and career aspirations It's happening right here at Iris Software. About Iris Software At Iris Software, our vision is to be our client's most trusted technology partner, and the first choice for the industry's top professionals to realize their full potential. With over 4,300 associates across India, U.S.A, and Canada, we help our enterprise clients thrive with technology-enabled transformation across financial s...

Work Location: ~CHENNAI, TN. Skill Required: Digital : Cloud Application Migration~Data Migration Experience Range in Required Skills: 5-7 Job Description: • Data Pipeline Design: Architect and implement scalable, reliable data pipelines using Databricks, Spark, and Delta Lake • Medallion Architecture Implementation : Ingest raw data into the Bronze layer, process and clean it in the Silver layer, and aggregate/transform it for analytics in the Gold layer • Data Modeling : Design and optimize data models for efficient storage, retrieval, and analytics • Data Quality & Security: Implement data validation, quality checks, and security controls throughout the pipeline • Automation & Monitoring:...

About the Role We are seeking a highly skilled Senior Data Engineer to design, build, and optimize scalable data platforms that empower data-driven decision-making. This role requires deep technical expertise in modern data engineering frameworks, architectural patterns, and cloud-native solutions on AWS. You will be a key contributor to our data strategy, ensuring data quality, governance, and reliability while mentoring other engineers in the team. Key Responsibilities Design, develop, and own robust, scalable, and maintainable data pipelines (batch & real-time). Architect and implement Data Lake, Data Warehouse, and Lakehouse solutions using modern frameworks and architectural patterns. E...

You specialize in modeling ServiceNow data for analytics, transforming raw extracts into curated lakehouse formats. This includes creating partitioned layers, business domains, and semantic enrichment for consumption by GenAI and BI tools. You ensure schema enforcement, lineage capture, and data classification are integrated into the transformation logic. Additionally, you have experience in building Silver and Gold zones for structured ITSM and CMDB use cases. Your expertise includes ServiceNow schema knowledge, data modeling, Delta Lake, and governance tagging. You are proficient in tools such as Databricks and Delta Lake, as well as holding the Azure Data Engineer Associate certification....

You will be working as a full-time on-site Databricks Developer in Hyderabad. Your responsibilities will include designing, developing, and maintaining highly scalable efficient data pipelines using Databricks, PySpark, and other related technologies to process large-scale datasets. Collaboration with cross-functional teams to design and implement data engineering and analytics solutions will be a key part of your role. To excel in this role, you should have expertise in using Unity Catalog and Metastore, optimizing Databricks notebooks, delta lake, and DLT pipelines. You should also be experienced in using Databricks SQL, implementing highly configured data processing solutions, and solutio...

Key Result Areas and Activities: Design and implement Lakehouse architectures using Databricks, Delta Lake, and Apache Spark. Lead the development of data pipelines, ETL/ELT processes, and data integration strategies. Collaborate with business and technical teams to define data architecture standards, governance, and security models. Optimize performance and cost-efficiency of Databricks clusters and jobs. Provide technical leadership and mentorship to data engineers and developers. Integrate Databricks with cloud platforms (Azure, AWS, or GCP) and enterprise systems. Evaluate and recommend tools and technologies to enhance the data ecosystem. Ensure compliance with data privacy and regulato...

Data Architect - Databricks (Azure/AWS) Role Overview: We are seeking an experienced Data Architect specializing in Databricks to lead the architecture, design, and migration of enterprise data workloads from on-premises systems (e.g., Oracle, Exadata, Hadoop) to Databricks on Azure or AWS . The role involves designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers), supporting large-scale ingestion, transformation, and publishing of data. Required Skills and Experience: 8+ years of experience in data architecture or engineering roles, with at least 3+ years specializing in cloud-based big data solutions. Hands-on experti...

As a Senior Azure Data Engineer, your responsibilities will include: Building scalable data pipelines using Databricks and PySpark Transforming raw data into usable business insights Integrating Azure services like Blob Storage, Data Lake, and Synapse Analytics Deploying and maintaining machine learning models using MLlib or TensorFlow Executing large-scale Spark jobs with performance tuning on Spark Pools Leveraging Databricks Notebooks and managing workflows with MLflow Qualifications: Bachelors/Masters in Computer Science, Data Science, or equivalent 7+ years in Data Engineering, with 3+ years in Azure Databricks Strong hands-on in: PySpark, Spark SQL, RDDs, Pandas, NumPy, Delta Lake Azur...

Mode: Contract As an Azure Data Architect, you will: Lead architectural design and migration strategies, especially from Oracle to Azure Data Lake Architect and build end-to-end data pipelines leveraging Databricks, Spark, and Delta Lake Design secure, scalable data solutions integrating ADF, SQL Data Warehouse, and on-prem/cloud systems Optimize cloud resource usage and pipeline performance Set up CI/CD pipelines with Azure DevOps Mentor team members and align architecture with business needs Qualifications: 10-15 years in Data Engineering/Architecture roles Extensive hands-on with: Databricks, Azure Data Factory, Azure SQL Data Warehouse Data integration, migration, cluster configuration, ...

Role Overview: We are seeking an experienced and visionary AI & Distributed Systems Architect to lead the design and implementation of large-scale, AI-driven distributed systems. The successful candidate will pioneer transformative AI solutions, including leveraging Generative AI, Machine Learning (ML), sophisticated agentic workflows, and context-aware data platforms. You will play a crucial role in defining architecture strategies, driving innovation, and leading technical teams to deliver cutting-edge capabilities at scale. Key Responsibilities: Lead the architecture design, development, and deployment of AI-powered distributed systems. Develop scalable, reliable, and high-performance sol...

We are seeking an experienced PySpark Developer / Data Engineer to design, develop, and optimize big data processing pipelines using Apache Spark and Python (PySpark). The ideal candidate should have expertise in distributed computing, ETL workflows, data lake architectures, and cloud-based big data solutions. Key Responsibilities: Develop and optimize ETL/ELT data pipelines using PySpark on distributed computing platforms (Hadoop, Databricks, EMR, HDInsight). Work with structured and unstructured data to perform data transformation, cleansing, and aggregation. Implement data lake and data warehouse solutions on AWS (S3, Glue, Redshift), Azure (ADLS, Synapse), or GCP (BigQuery, Dataflow). Op...

Work Mode: Hybrid (3 days WFO) Locations: Bangalore, Noida, Pune, Mumbai, Hyderabad (Candidates must be in Accion cities to collect assets and attend in-person meetings as required). Key Requirements: Technical Skills: Databricks Expertise: 5+ years of hands-on experience in data engineering/ETL using Databricks on AWS/Azure cloud infrastructure. Proficiency in Delta Lake, Unity Catalog, Delta Sharing, Delta Live Tables (DLT), MLflow, and Databricks SQL. Experience with Databricks CI/CD tools (e.g., BitBucket, GitHub Actions, Databricks CLI). Data Warehousing & Engineering: Strong understanding of data warehousing concepts (Dimensional, SCD2, Data Vault, OBT, etc.). Proven ability to impleme...

When 5% of Indian households shop with us, it's important to build data-backed, resilient systems to manage millions of orders every day. We've done this - with zero downtime! ???? Sounds impossible Well, that's the kind of Engineering muscle that has helped Meesho become the e-commerce giant that it is today. We value speed over perfection, and see failures as opportunities to become better. We've taken steps to inculcate a strong Founder's Mindset across our engineering teams, making us grow and move fast. We place special emphasis on the continuous growth of each team member - and we do this with regular 1-1s and open communication. Tech Culture We have a unique tech culture where enginee...

Flexera saves customers billions of dollars in wasted technology spend. A pioneer in Hybrid ITAM and FinOps, Flexera provides award-winning, data-oriented SaaS solutions for technology value optimization (TVO), enabling IT, finance, procurement and cloud teams to gain deep insights into cost optimization, compliance and risks for each business service. Flexera One solutions are built on a set of definitive customer, supplier and industry data, powered by our Technology Intelligence Platform, that enables organizations to visualize their Enterprise Technology Blueprint in hybrid environments-from on-premises to SaaS to containers to cloud. We're transforming the software industry. We're Flexe...

As a Lead Databricks Data Engineer based in Ghansoli, Navi Mumbai, you will be responsible for leading the design and implementation of robust data pipelines and solutions. We are looking for a resource who can join our team within 2-3 weeks. With over 8 years of experience in data engineering, you will leverage your deep technical expertise in Databricks, PySpark, and Azure Data Factory (ADF) to ensure the success of our data integration efforts. Your key responsibilities will include designing, developing, and optimizing large-scale data pipelines using Databricks and PySpark, as well as leading data integration efforts using Azure Data Factory. You will collaborate with cross-functional t...

You will be responsible for developing, optimizing, and maintaining business intelligence and data warehouse systems. Your main accountabilities include developing scalable data pipelines, integrating offline and online data for improved customer understanding, managing data quality and governance strategies, transforming data for analysis, and ensuring data accessibility for the enterprise. To qualify for this role, you should have a Masters/Bachelors degree in Engineering/Computer Science/Math/Statistics or equivalent, strong programming skills in Python/Pyspark/SAS, experience with large datasets and technologies like Hadoop, Hive, and Spark optimization. Additionally, experience with clo...

As a Data Architect with over 6 years of experience, you will be responsible for designing and implementing modern data lakehouse architectures on cloud platforms such as AWS, Azure, or GCP. Your primary focus will be on defining data modeling, schema evolution, partitioning, and governance strategies to ensure high-performance and secure data access. In this role, you will own the technical roadmap for scalable data platform solutions, ensuring they are aligned with enterprise needs and future growth. You will also provide architectural guidance and conduct code/design reviews across data engineering teams to maintain high standards of quality. Your responsibilities will include building an...

Job description Some careers shine brighter than others. If you're looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to...

Job Title: Senior Data Engineer Azure Databricks Experience: 7+ Years Location: Navi Mumbai (On-site) Job Overview: We are looking for a highly skilled Senior Data Engineer with expertise in Azure Databricks, big data technologies, and cloud-based data architectures. The ideal candidate will have hands-on experience building scalable data pipelines and platforms for high-volume batch and real-time processing, preferably in the manufacturing or energy sector. Key Responsibilities: • Design and implement robust data pipelines using Azure Databricks, Spark, and other Azure data services. • Collaborate with architects, data scientists, and business teams to understand requirements and translate ...

Bachelors or masters degree in Computer Science, IT, or related field. 5+ years of data engineering experience, including 1+ years on Azure Data Platform. Strong proficiency in Azure Data Factory, Azure Fabric, and Notebooks (PySpark/Python/SQL). Expertise in Delta Lake, Parquet, and modern data lakehouse architectures. Experience with Azure Synapse Analytics, Databricks (if applicable), and data visualization tools such as Power BI. Skilled in SQL, Python, and PySpark for large-scale data transformations. Experience implementing CI/CD for data pipelines and notebooks using Azure DevOps or Git. Knowledge of data governance, security best practices, and compliance frameworks. If youre ready t...