6143 Data Bricks Jobs - Page 45

Hiring a Python Data Engineer to build and maintain on-premise data pipelines using Dagster. Role includes ETL development, SQL/Python work, CI/CD on Kubernetes, edge data processing, and ensuring data quality, governance, and platform reliability

Greetings from HCL! Currently Hiring for DevSecOps / DataOps Engineer Mandatory Skills Tool Knowledge: AWS Cloud like Glue,Redshift,IAM,s3,SQS,SNS etc., Databricks, dbt, Airflow, Matillion (Preferrable) DevOps: GitHub Action, Terraform, AWS CDK (Preferrable) Language: Python, Pyspark, SQL JD 7-15 years of experience in end-to-end application Development, Deployment & Support for AWS based & Databricks lakehouse. Automated setup of AWS infra & management ex. lambda,s3,glue,ecs,redshift,sqs,sns,emr ,transfer family etc Automated setup of Databricks workspace, jobs,UC catalog and mapping to s3 bucket(managed/external), permission model etc. Hands on knowledge in python and Pyspark to help devel...

Qualifications & Experience Bachelors degree in Computer Science, Information Systems, Statistics, or related field. 510 years of experience in data quality, data engineering, analytics operations. Strong command of SQL , Python , and PySpark for data validation and troubleshooting. Hands-on exposure to Databricks for building and running data pipelines. Working knowledge of reconciliation processes, data profiling, and DQ metrics.

Role & responsibilities Job Title: MLOps Engineer Description: We are seeking an experienced MLOps Engineer to join our team and play a key role in deploying, automating, monitoring, and optimizing machine learning solutions. The ideal candidate will have strong experience with Databricks, SQL, Python, PySpark, and CI/CD pipelines, along with hands-on exposure to Jenkins, SonarQube, Jira, and Git. You will collaborate with data scientists, data engineers, and DevOps teams to streamline the ML lifecycle from model development to production deployment and monitoring. Key Responsibilities Design, implement, and maintain MLOps pipelines for machine learning model deployment and monitoring. Colla...

Job Title: MLOps Engineer Description: We are seeking an experienced MLOps Engineer to join our team and play a key role in deploying, automating, monitoring, and optimizing machine learning solutions. The ideal candidate will have strong experience with Databricks, SQL, Python, PySpark, and CI/CD pipelines, along with hands-on exposure to Jenkins, SonarQube, Jira, and Git. You will collaborate with data scientists, data engineers, and DevOps teams to streamline the ML lifecycle from model development to production deployment and monitoring. Key Responsibilities Design, implement, and maintain MLOps pipelines for machine learning model deployment and monitoring. Collaborate with data scienti...

Role & responsibilities Role Overview The Databricks Architect is responsible for defining the enterprise-wide Lakehouse architecture, setting technical direction, and overseeing the design of scalable, secure, and high-performance data solutions on the Databricks platform. This role provides architectural governance, ensures best-practice adoption across teams, and guides Databricks Tech Leads in delivering future-proof data engineering, analytics, and AI solutions. Location: Bengaluru Key Responsibilities Architecture & Strategy Define Lakehouse architecture vision, standards, and reference patterns using Delta Lake, Unity Catalog, Databricks SQL, and ML/AI runtimes. Lead architecture blue...

Job Title: MLOps Engineer Description: We are seeking an experienced MLOps Engineer to join our team and play a key role in deploying, automating, monitoring, and optimizing machine learning solutions. The ideal candidate will have strong experience with Databricks, SQL, Python, PySpark, and CI/CD pipelines, along with hands-on exposure to Jenkins, SonarQube, Jira, and Git. You will collaborate with data scientists, data engineers, and DevOps teams to streamline the ML lifecycle from model development to production deployment and monitoring. Key Responsibilities • Design, implement, and maintain MLOps pipelines for machine learning model deployment and monitoring. • Collaborate with data sci...

Job Title: Data Engineer Experience: 5+ Years Location: Chennai/Hyderabad/Mumbai/Pune/Bangalore (Hybrid) Key Responsibilities: Python Proficiency: Demonstrate a strong command of Python programming language, actively contributing to the development and maintenance of data engineering solutions. Data Engineering Expertise: Set up and maintain efficient data pipelines, ensuring smooth data flow and integration across systems. Experience with SQL and Data Warehouse/ Data Lakes is required. Contribute to the establishment and maintenance of Data Lakes, implementing industry best practices. Execute data scrubbing techniques and implement data validation processes to ensure data integrity and qual...

Experience: 5-8 years Responsibilities: Develop and optimize ETL pipelines in Databricks using PySpark/Spark SQL. Migrate existing Informatica mappings, Unix scripts, and SQL stored procedures to Databricks Analyze legacy ETL workflows to understand data lineage, dependencies, and business logic. Build and maintain STG, EDW, and BI layers using Delta Lake tables with proper partitioning and schema management. Implement incremental (SCD1) and full load strategies along with error handling and logging. Support historical data migration, reconciliation, and performance validation. Collaborate with the lead to ensure data quality and efficient transformation logic. Participate in peer code revie...

Hiring Senior Data Engineer (10+ yrs) for Bengaluru. Must have strong Azure expertise, Python, ADF, ETL/ELT, SQL, and cloud data architecture skills. Experience in building scalable pipelines required. Immediate joiners preferred. Apply ASAP. Today. Provident fund

Role & responsibilities Develop and manage data pipelines on Microsoft Azure/Fabric for internal and external data sources. Ensure data pipelines are optimized for reliability and scalability. Ensure the scalability, security, and performance of our cloud architecture while aligning with business goals. Ensure adherence to security best practices in cloud operations Preferred candidate profile Hands-on experience with Azure storage platforms preferably, Microsoft Fabric/Databricks. Hands-on experience in developing PySpark codes for data processing, transformation, and loading. Hands-on experience in developing backend API frameworks using .Net 8. Deployment of coding artefacts using Azure D...

Develops validation python scripts for automated execution,while creating actionable data quality reports that measure trends over time Analysing test outputs and reporting defects Develop,documents,and implements data quality standards Agile testing Required Candidate profile Experience with ETL testing. Ability to create data bricks notebooks to automate the manual tests. Write manual test cases from business requirements & technical specifications. Experience in Pyspark.

Python Developer Remote/Hyd 4+ Years SQL Graph QL MDM analytics platform Databricks

We are seeking a Databricks Engineer to build data pipelines using Medallion Architecture and support analytics, AI, and reporting. You will integrate enterprise systems, enforce data quality, and ensure strong governance. Responsibilities Build ELT/ETL pipelines using Databricks, Spark, and Delta Lake Implement bronze/silver/gold data layers Ingest data from PeopleSoft, D2L, Salesforce, and other systems Build Databricks Workflows for orchestration and automation Implement data quality checks, anomaly detection, and monitoring Support governance and lineage using Unity Catalog Deliver BI and ML-ready datasets Create documentation and runbooks Required Qualifications Hands-on experience with...

Subject: Exciting Opportunity at Celebal Technologies(SME Application Support Engineer) Apply Now! Hi Candidate, Greetings from Celebal Technologies ! We are currently hiring for the below position and came across your profile, which seems to match our requirements. Please go through the job details and share your updated resume if interested. Position: SME Application Support Engineer Experience: 6 to 12 years Location: Only Mumbai Notice Period: Immediate Job Description 6–10 years of experience in Big Data / Cloud Data Platform Support. SME of Databricks platform (clusters, jobs, repos, MLflow, warehouse) Expertise in experience in UNIX, SQL, Shell Scripting. Expertise in experience in Sp...

Job description Hiring for Azure Databrick Engineer Mandatory Skills: Databrick, Data lake, Azure Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and d...

Job Role & responsibilities: - Understanding operational needs by collaborating with specialized teams Supporting key business operations. This involves architecture designing, building and deploying data systems, pipelines etc Designing and implementing agile, scalable, and cost efficiency solution on cloud data services. Lead a team of developers, implement Sprint planning and executions to ensure timely deliveries Technical Skill, Qualification & experience required:- 5-10 years of experience in Azure Cloud Data Engineering, Azure Databricks, datafactory , Pyspark, SQL,Python Hands on experience in Data Engineer, Azure Databricks, Data factory, Pyspark, SQL Proficient in Cloud Services Az...

Overview Product Data & Analytics team builds internal analytic partnerships, strengthening focus on the health of the business, portfolio and revenue optimization opportunities, initiative tracking, new product development and Go-To Market strategies. Are you excited about Data Assets and the value they brings to an organization? Are you an evangelist for data driven decision making? Are you motivated to be part of a Global Analytics team that builds large scale Analytical Capabilities supporting end users across 6 continents? Do you want to be the go-to resource for data analytics in the company? The ideal candidate has a knack for seeing solutions in sprawling data sets and the business m...

Overview As an Analyst, Data Modeler, your focus would be to partner with D&A Data Foundation team members to create data models for Global projects. This would include independently analysing project data needs, identifying data storage and integration needs/issues, and driving opportunities for data model reuse, satisfying project requirements. Role will advocate Enterprise Architecture, Data Design, and D&A standards, and best practices. You will be performing all aspects of Data Modeling working closely with Data Governance, Data Engineering and Data Architects teams. As a member of the data Modeling team, you will create data models for very large and complex data applications in public...

What you can look forward to as Product Data Analyst- Data Engineering: End-to-End Project Ownership : Drive and manage analytics initiatives from concept to deployment, ensuring high-quality, timely deliverables with minimal supervision. Engineering Best Practices : Apply structured software development principles, including version control (e.g., Git), CI/CD, automated testing, and peer code reviews to ensure robust, scalable, and maintainable solutions. Data Architecture & Pipelines : Design, build, and optimize scalable and reliable data models and pipelines in Databricks, Azure Cloud, and other big data environments to support advanced analytics and AI use cases. Data Modeling & Perform...

Educational Requirements Bachelor of Engineering,BSc,MSc,MCA Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to get to the heart of customer issues, diagnose problem areas, design innovative solutions and facilitate deployment resulting in client delight. You will develop a proposal by owning parts of the proposal document and by giving inputs in solution design based on areas of expertise. You will plan the activities of configuration, configure the product as per the design, conduct conference room pilots and will assist in resolving any queries related to requirements and solution desi...

Educational Requirements Bachelor of Engineering,BBA,BSc,MCA,MSc Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to lead the engagement effort of providing high-quality and value-adding consulting solutions to customers at different stages- from problem definition to diagnosis to solution design, development and deployment. You will review the proposals prepared by consultants, provide guidance, and analyze the solutions defined for the client business problems to identify any potential risks and issues. You will identify change Management requirements and propose a structured approach to...

General Summary: Data Platform, Engineering and Operations lead will play a pivotal role in shaping the future of our data capabilities as part of strategic and critical Enterprise Data Transformation (EDT) program. We're seeking a visionary, hands-on leader to drive the build out of a modern enterprise-wide lake house, ensuring reliability, scalability, security, and compliance. and enabling cutting edge analytics solutions. This role will be leading a talented team of platform and data engineers, fostering a culture of excellence while balancing speed of delivery with innovation and operational excellence. This role will also collaborate with cross functional teams (Solution engineers, SME...

About The Role Project Role : Application Support Engineer Project Role Description : Act as software detectives, provide a dynamic service identifying and solving issues within multiple components of critical business systems. Must have skills : EPIC Systems Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Support Engineer, you will act as software detectives, providing a dynamic service that identifies and resolves issues within various components of critical business systems. Your typical day will involve collaborating with team members to troubleshoot software problems, analyzing syste...

About The Role Project Role : Application Support Engineer Project Role Description : Act as software detectives, provide a dynamic service identifying and solving issues within multiple components of critical business systems. Must have skills : EPIC Systems Good to have skills : NA Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Support Engineer, you will act as software detectives, providing a dynamic service that identifies and resolves issues within various components of critical business systems. Your typical day will involve collaborating with team members to troubleshoot software problems, analyzing syste...