453 Data Engineer Jobs - Page 4

Job Title: Data Engineer (Snowflake Expert) Location: Pune, Maharashtra Job Type: [Full-Time] Experience Level: [Mid-Level ] Job Summary: We are seeking an experienced Data Engineer with strong expertise in Snowflake to design, build, and optimize scalable data solutions. The ideal candidate will have hands-on experience with Snowflake cloud data platform, advanced SQL skills, and modern data engineering practices for data integration, transformation, and delivery. DB : SQL Server (Must), Snowflake (Must) ETL Tools : SSIS (Must), Matillion (Nice to have) Integration Tools : Snowpipe (Nice to have) Orchestration Tools : Dagster (Nice to have) Key Responsibilities: Design, develop, and maintain data pipelines and ETL/ELT processes using Snowflake as the primary data platform. Write complex and optimized SQL scripts for data transformation, aggregation, and analysis. Implement and manage Snowflake features such as virtual warehouses, streams, tasks, data sharing, and secure data access. Build and maintain data models and schemas (Star, Snowflake, Data Vault) to support analytics and reporting needs. Work with business analysts and stakeholders to gather data requirements and deliver solutions that meet business objectives. Integrate Snowflake with upstream and downstream systems using tools like dbt, Airflow, Fivetran, Informatica, or Azure Data Factory . Ensure data quality, governance, and security standards are enforced across pipelines and environments. Monitor, troubleshoot, and optimize performance of Snowflake workloads and data pipelines. Required Qualifications: Bachelors or Master’s degree in Computer Science, Data Engineering, or a related field. 4+ years of experience as a Data Engineer or in a data-intensive role. Strong Snowflake expertise including advanced features (Time Travel, Cloning, Streams & Tasks, External Tables). Proficiency in SQL scripting and performance tuning. Experience with cloud platforms (AWS, Azure, or GCP) and data integration tools. Strong understanding of data warehousing concepts, data modeling, and modern data architectures . Knowledge of ETL/ELT processes and best practices. Preferred Skills: Experience with dbt (Data Build Tool) for transformation pipelines. Familiarity with orchestration tools like Airflow, Dagster, or Prefect . Exposure to BI tools such as Power BI, Tableau, or Looker. Experience with Python, Spark, or Scala for data processing. Understanding of data security, compliance, and governance frameworks (e.g., GDPR, HIPAA).

We're looking for Data Engineers. Must have Qualification/Skills: 2+ years of non-internship professional Data Engineer experience Has data pipeline implementation experience with any of these cloud providers - AWS, Azure, GCP. Experience with cloud storage, cloud database, cloud data warehousing and Data lake solutions Experience in using cloud data integration services for structured, semi-structured and unstructured data Eagerness to learn and grow in a collaborative environment. (*Note: This is a requirement for one of Scout's clients) Roles and openings are subject to change. The client reserves the right to modify, reassign, or withdraw any role, requirements, number of positions, or timelines at any time during the hiring process. Submission of an application does not guarantee assignment to a particular role or an offer of employment. All hires are contingent on standard pre-employment screening and eligibility verification.

Role & responsibilities Design, develop, and maintain scalable and robust data pipelines on Databricks. Collaborate with data scientists and analysts to understand data requirements and deliver solutions. Optimize and troubleshoot existing data pipelines for performance and reliability. Ensure data quality and integrity across various data sources. Implement data security and compliance best practices. Monitor data pipeline performance and conduct necessary maintenance and updates. Document data pipeline processes and technical specifications. Qualifications 5+ years of experience in data engineering. Proficiency with Databricks and pySpark. Strong SQL skills and experience with relational databases. Experience with big data technologies (e.g., Hadoop, Kafka). Knowledge of data warehousing concepts and ETL processes. Excellent problem-solving and analytical skills.

DATA ENGINEER yrs of Exp - 3-10YRS Education - BE/B.TECH - ONLY FROM IITs Skill - Data Engineer + Azure + SQL Should be strong in Azure and SQL Queries Mode - Work from Office Work Location - Noida Notice Period - 0-45 Days Joiners

As a Data Engineer at StratLytics, you will be part of a dynamic team dedicated to crafting cutting-edge data pipelines. Your primary responsibility will involve playing a pivotal role in the architecture, development, and refinement of data infrastructure that is specifically tailored for data science endeavors. Life @ StratLytics is all about empowering individuals to take charge of their own professional journeys. We value opinions and contributions from every team member, recognizing that passion is the driving force behind excellence. Continuous learning, introspection, and constructive debates are integral parts of our work culture, fostering an environment where transparency in office work and business decisions enables everyone to make meaningful contributions. Working at StratLytics means embracing the thrill of steering your own ship while shouldering the responsibility of delivering successful outcomes. If you are ready to navigate this exciting journey and believe in the power of passion over mere talent, then StratLytics is the ideal place for you. If you are interested in joining us and do not find a suitable opening listed, we encourage you to reach out by sending your resume to hr@StratLytics.com along with a brief message explaining why you would be a valuable addition to our team.,

6+ years of experience in information technology, Minimum of 3-5 years of experience in managing and administering Hadoop/Cloudera environments. Cloudera CDP (Cloudera Data Platform), Cloudera Manager, and related tools. Hadoop ecosystem components (HDFS, YARN, Hive, HBase, Spark, Impala, etc.). Linux system administration with experience with scripting languages (Python, Bash, etc.) and configuration management tools (Ansible, Puppet, etc.) Tools like Kerberos, Ranger, Sentry), Docker, Kubernetes, Jenkins Cloudera Certified Administrator for Apache Hadoop (CCAH) or similar certification. Cluster Management, Optimization, Best practice implementation, collaboration and support.

Data Engineer: Experienced Kstream + Ksql dev with in-depth knowledge of specific client systems TAHI Contract and Application, ISP Contract and Application modules. Performs data analysis and writes code to implement functional requirements per LLD and client processes. Minimum skills levels in this specific area Current roles are 5 + years plus Insurnace domain experience These are technical roles, and the prime requirement is for Kstream/ Java/ KSLQDB/ Kafka

We are seeking experienced Data Engineers with over 3 years of experience to join our team at Intuit, through Cognizant. The selected candidates will be responsible for developing and maintaining scalable data pipelines, managing data warehousing solutions, and working with advanced cloud environments. The role requires strong technical proficiency and the ability to work onsite in Bangalore. Key Responsibilities: Design, build, and maintain data pipelines to ingest, process, and analyze large datasets using PySpark. Work on Data Warehouse and Data Lake solutions to manage structured and unstructured data. Develop and optimize complex SQL queries for data extraction and reporting. Leverage AWS cloud services such as S3, EC2, EMR, Athena, and Redshift for data storage, processing, and analytics. Collaborate with cross-functional teams to ensure the successful delivery of data solutions that meet business needs. Monitor data pipelines and troubleshoot any issues related to data integrity or system performance. Required Skills: 3 years of experience in data engineering or related fields. In-depth knowledge of Data Warehouses and Data Lakes. Proven experience in building data pipelines using PySpark. Strong expertise in SQL for data manipulation and extraction. Familiarity with AWS cloud services, including S3, EC2, EMR, Athena, Redshift, and other cloud computing platforms. Preferred Skills: Python programming experience is a plus. Experience working in Agile environments with tools like JIRA and GitHub.

1. The resource should have knowledge on Data Warehouse and Data Lake 2. Should aware of building data pipelines using Pyspark 3. Should be strong in SQL skills 4. Should have exposure to AWS environment and services like S3, EC2, EMR, Athena, Redshift etc 5. Good to have programming skills in Python

Job Type: Full-time, On-site (5 days a week) Notice Period: Immediate to 30 days Mandatory Skills: SAP Python Data Management Preferred Skills: Balance Sheet Management (BSM) Capital Liquidity Reporting Job Description: One of the Big 4 Consulting firms is seeking a skilled Data Engineer with expertise in SAP and data management to join their advisory team in the financial services domain. The role involves working on projects related to Balance Sheet Management (BSM), capital products, and liquidity reporting while contributing to data warehousing and simulation initiatives. Key Responsibilities: Develop and implement data engineering solutions with SAP expertise (technical focus). Manage large datasets, including data management and data warehousing processes. Support Balance Sheet Management (BSM) and simulation requirements. Utilize Python for data processing and automation tasks. Collaborate on projects involving capital products and liquidity reporting.

We are looking for immediate joiner - Mumbai and Hyderabad location (Work from office) - Data Processing: BigQuery, Apache Spark, Hadoop, Dataflow - BI Tools: Tableau, Power BI, Looker - Languages: Python, SQL, Java, Scala - ETL Tools: Apache Nifi, Talend, Informatica, Dataform - Cloud: GCP (BigQuery, Dataflow, Pub/Sub, Cloud Storage) - Data Modeling: Kimball, Star Schema, Snowflake Schema - Version Control: Git, GitLab

We are looking for immediate joiner - Mumbai and Hyderabad location (Work from office) - Data Processing: BigQuery, Apache Spark, Hadoop, Dataflow - BI Tools: Tableau, Power BI, Looker - Languages: Python, SQL, Java, Scala - ETL Tools: Apache Nifi, Talend, Informatica, Dataform - Cloud: GCP (BigQuery, Dataflow, Pub/Sub, Cloud Storage) - Data Modeling: Kimball, Star Schema, Snowflake Schema - Version Control: Git, GitLab

As a Data Engineer at our company, you will play a crucial role in designing, building, and maintaining scalable data pipelines and systems to facilitate analytics and data-driven decision-making. Your responsibilities will include utilizing your expertise in data processing, data modeling, and big data technologies to ensure the smooth functioning of our data infrastructure. You will collaborate closely with cross-functional teams to understand their data requirements and provide efficient solutions to meet their needs. Additionally, you will be expected to stay updated on emerging trends in data engineering and continuously improve our data processes for enhanced performance and efficiency. The ideal candidate for this role is someone who is passionate about leveraging data to drive business outcomes and possesses a strong background in data engineering practices. If you are a skilled Data Engineer looking to make a significant impact in a dynamic and innovative environment, we invite you to join our team.,

The Senior GCP Data Engineer role at Deutsche Bank in Bangalore, India, offers you the opportunity to be part of an ambitious initiative focused on Sustainable Finance, ESG Risk Mitigation, and Corporate Sustainability. As Climate Change presents new challenges and opportunities, the bank is investing in developing a Sustainability Technology Platform, Sustainability data products, and sustainability applications to support its goals. To contribute to this initiative, we are seeking a highly motivated and experienced Senior GCP Data Engineer who is passionate about Climate Change and eager to leverage their Technology Skillset in Cloud/Hybrid Architecture. In this role, you will be responsible for designing, developing, and maintaining robust data pipelines that transform raw data into valuable insights for the organization. You will work on GCP services like Dataflow, Dataproc, and Pub/Sub to design and implement data ingestion and transformation processes using tools such as Apache Beam and Apache Spark. Additionally, you will manage and optimize data storage solutions on GCP, including Big Query, Cloud Storage, and Cloud SQL, while also implementing data security and access controls using GCP's Identity and Access Management (IAM) and Cloud Security Command Center. Key responsibilities: - Design, develop, and maintain data pipelines using GCP services. - Implement data ingestion and transformation processes with tools like Apache Beam and Apache Spark. - Manage and optimize data storage solutions on GCP. - Implement data security and access controls using GCP's tools. - Monitor and troubleshoot data pipelines and storage solutions. - Collaborate with data experts, analysts, and product teams. - Automate data processing tasks using Python. - Participate in code reviews and contribute to best practices. - Stay updated on advancements in GCP services and technologies. Your skills and experience should include: - 5+ years of experience as a Data Engineer or similar role. - Expertise in designing, developing, and deploying data pipelines. - Proficiency in GCP and its core data services. - Strong skills in Python & SQL for data manipulation. - Experience with distributed data processing frameworks like Apache Beam or Apache Spark. - Familiarity with data security and access control principles. - Excellent communication, collaboration, and problem-solving abilities. - Ability to work independently, manage multiple projects, and meet deadlines. - Knowledge of Sustainable Finance/ESG Risk/CSRD/Regulatory Reporting is a plus. - Knowledge of cloud infrastructure and data governance best practices is a plus. - Knowledge of Terraform is a plus. At Deutsche Bank, you will receive training, coaching, and support to excel in your career. The culture fosters continuous learning, and you can benefit from a range of flexible benefits tailored to suit your needs. The company promotes a positive, fair, and inclusive work environment where all individuals are welcome to apply and contribute to the success of the organization.,

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture, and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. We are seeking a passionate data analyst to transform data into actionable insights and support decision-making in a global organization focused on pricing and commercial strategy. This role spans business analysis, requirements gathering, data modeling, solution design, and visualization using modern tools. The analyst will also maintain and improve existing analytics solutions, interpret complex datasets, and communicate findings clearly to both technical and non-technical audiences. **Essential Functions of the Job:** - Analyze and interpret structured and unstructured data using statistical and quantitative methods to generate actionable insights and ongoing reports. - Design and implement data pipelines and processes for data cleaning, transformation, modeling, and visualization using tools such as Power BI, SQL, and Python. - Collaborate with stakeholders to define requirements, prioritize business needs, and translate problems into analytical solutions. - Develop, maintain, and enhance scalable analytics solutions and dashboards that support pricing strategy and commercial decision-making. - Identify opportunities for process improvement and operational efficiency through data-driven recommendations. - Communicate complex findings in a clear, compelling, and actionable manner to both technical and non-technical audiences. **Analytical/Decision Making Responsibilities:** - Apply a hypothesis-driven approach to analyzing ambiguous or complex data and synthesizing insights to guide strategic decisions. - Promote adoption of best practices in data analysis, modeling, and visualization, while tailoring approaches to meet the unique needs of each project. - Tackle analytical challenges with creativity and rigor, balancing innovative thinking with practical problem-solving across varied business domains. - Prioritize work based on business impact and deliver timely, high-quality results in fast-paced environments with evolving business needs. - Demonstrate sound judgment in selecting methods, tools, and data sources to support business objectives. **Knowledge and Skills Requirements:** - Proven experience as a data analyst, business analyst, data engineer, or similar role. - Strong analytical skills with the ability to collect, organize, analyze, and present large datasets accurately. - Foundational knowledge of statistics, including concepts like distributions, variance, and correlation. - Skilled in documenting processes and presenting findings to both technical and non-technical audiences. - Hands-on experience with Power BI for designing, developing, and maintaining analytics solutions. - Proficient in both Python and SQL, with strong programming and scripting skills. - Skilled in using Pandas, T-SQL, and Power Query M for querying, transforming, and cleaning data. - Hands-on experience in data modeling for both transactional (OLTP) and analytical (OLAP) database systems. - Strong visualization skills using Power BI and Python libraries such as Matplotlib and Seaborn. - Experience with defining and designing KPIs and aligning data insights with business goals. **Additional/Optional Knowledge and Skills:** - Experience with the Microsoft Fabric data analytics environment. - Proficiency in using the Apache Spark distributed analytics engine, particularly via PySpark and Spark SQL. - Exposure to implementing machine learning or AI solutions in a business context. - Familiarity with Python machine learning libraries such as scikit-learn, XGBoost, PyTorch, or transformers. - Experience with Power Platform tools (Power Apps, Power Automate, Dataverse, Copilot Studio, AI Builder). - Knowledge of pricing, commercial strategy, or competitive intelligence. - Experience with cloud-based data services, particularly in the Azure ecosystem (e.g., Azure Synapse Analytics or Azure Machine Learning). **Supervision Responsibilities:** - Operates with a high degree of independence and autonomy. - Collaborates closely with cross-functional teams including sales, pricing, and commercial strategy. - Mentors junior team members, helping develop technical skills and business domain knowledge. **Other Requirements:** - Collaborates with a team operating primarily in the Eastern Time Zone (UTC 4:00 / 5:00). - Limited travel may be required for this role. **Job Requirements:** **Education:** A bachelor's degree in a STEM field relevant to data analysis, data engineering, or data science is required. Examples include (but are not limited to) computer science, statistics, data analytics, artificial intelligence, operations research, or econometrics. **Experience:** 3-6 years of experience in data analysis, data engineering, or a closely related field, ideally within a professional services environment. **Certification Requirements:** No certifications are required for this role. EY | Building a better working world EY exists to build a better working world, helping to create long-term value for clients, people, and society and build trust in the capital markets. Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform, and operate. Working across assurance, consulting, law, strategy, tax, and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.,

As a Data Engineer, you will provide support to the Global BI team for Isolation Valves in their migration to Microsoft Fabric. Your primary focus will be on data gathering, modeling, integration, and database design to facilitate efficient data management. Your responsibilities will include developing and optimizing scalable data models to meet analytics and reporting requirements and utilizing Microsoft Fabric and Azure technologies for high-performance data processing. In this role, you will collaborate with cross-functional teams, including data analysts, data scientists, and business collaborators, to understand their data needs and deliver effective solutions. You will leverage Fabric Lakehouse for data storage, governance, and processing to support Power BI and automation initiatives. Expertise in data modeling, with a specific emphasis on data warehouse and lakehouse design, will be essential. You will be responsible for designing and implementing data models, warehouses, and databases using MS Fabric, Azure Synapse Analytics, Azure Data Lake Storage, and other Azure services. Additionally, you will develop ETL processes using tools such as SQL Server Integration Services (SSIS) and Azure Synapse Pipelines to prepare data for analysis and reporting. Implementing data quality checks and governance practices to ensure data accuracy, consistency, and security will also be part of your role. Your tasks will involve supervising and optimizing data pipelines and workflows for performance, scalability, and cost efficiency, utilizing Microsoft Fabric for real-time analytics and AI-powered workloads. Proficiency in Business Intelligence (BI) tools like Power BI and Tableau, along with experience in data integration and ETL tools such as Azure Data Factory, will be beneficial. You are expected to have expertise in Microsoft Fabric or similar data platforms and a deep understanding of the Azure Cloud Platform, particularly in data warehousing and storage solutions. Strong communication skills are essential, as you will need to convey technical concepts to both technical and non-technical stakeholders. The ability to work independently as well as within a team environment is crucial. Preferred qualifications for this role include 3-5 years of experience in Data Warehousing with on-premises or cloud technologies, strong analytical abilities, and proficiency in database management, SQL query optimization, and data mapping. A willingness to work flexible hours based on project requirements, strong documentation skills, and advanced SQL skills are also required. Hands-on experience with Medallion Architecture for data processing, prior experience in a manufacturing environment, and the ability to quickly learn new technologies are advantageous. Travel up to 20% may be required. A Bachelor's degree or equivalent experience in Science, with a focus on MIS, Computer Science, Engineering, or a related field, is preferred. Good interpersonal skills in English for efficient collaboration with overseas teams and Agile certification are also desirable. At Emerson, we value an inclusive workplace where every employee is empowered to grow and contribute. Our commitment to ongoing career development and fostering an innovative and collaborative environment ensures that you have the support to succeed. We provide competitive benefits plans, medical insurance options, employee assistance programs, recognition, and flexible time off plans to prioritize employee wellbeing. Emerson is a global leader in automation technology and software, serving industries such as life sciences, energy, power, renewables, and advanced factory automation. We are committed to diversity, equity, and inclusion, and offer opportunities for career growth and development. Join our team at Emerson and be part of a community dedicated to making a positive impact through innovation and collaboration.,

You will be responsible for developing SAP BW4HANA information models, implementing end-to-end SAP data warehouse solutions, and working with business stakeholders to understand and translate business requirements into technical specifications. Your role will involve hands-on experience in BW application areas such as SD, MM, PP, VC, PM, and FICO, as well as new technologies like HANA and SQL. As a Manager- SAP BW4HANA + Azure Developer, you will also be involved in Data Lake build and design, Azure platform development alongside SAP data warehouse, and knowledge in analytical platforms. You should have a deep understanding of info-providers like CP, ADSO, and Open ODS, as well as experience with ETL from SAP transactional systems. Additionally, you will be expected to have hands-on experience with BW ABAP/AMDP scripts for routines, transformations, or customer exits, resolving issues in process chains and user reports, developing queries in BW4HANA and analytics using Analysis for Office, and preparing technical documentation. Knowledge of BO report development is considered advantageous. The ideal candidate should have a minimum of 5-7 years of experience, with a specific certification in SAP BW4HANA being a plus. Educational qualifications in BSc.IT, BSc.CS, or BE are required. Strong communication skills and analytical abilities are essential for this role. You will also be responsible for monitoring system performance and making necessary adjustments when needed.,

Hiring, Data Engineer 4+ years of hands-on experience Experience in building and optimizing Big Data data pipelines, architectures, and data sets. Experience with big data tools: Hadoop, Spark Optimization, Spark-Streaming, Hive, Kafka, Hbase, Airflow, etc. Hands-on in SQL

Role & responsibilities Azure Data Engineer involved in all stages of the development life cycle, through requirements analysis, coding, unit testing, system testing, UAT and warranty-support. Analyzing data sets and designing and coding stable and scalable data pipelined also integrating into existing data pipelines. Ensure timely delivery of systems that meet functional requirements by performing comprehensive unit testing. Implement processes and tools to ensure data quality and reliability. Review, refactor, and debug code to identify and resolve defects, improve performance, and address technical debt. Provide technical support and troubleshooting assistance for developed systems, ensuring smooth operation and issue resolution. Collaborate with clients and stakeholders to understand business requirements and align technical solutions to meet their needs. Preferred candidate profile Azure Database -Advance Azure Synapse-Intermediate Azure Data Factory-Intermediate Python, Pyspark-Intermediate

ob Description Fabric Data Engineer Experience: 5–7 years We are seeking a skilled and motivated Fabric Data Engineer to join our data engineering team. The ideal candidate will have strong proficiency in SQL and PySpark for data wrangling, transformation, and modeling, along with solid experience working in Microsoft Azure environments . Familiarity with Power BI for data visualization and analytics is highly desirable. You will play a key role in building robust data pipelines and scalable solutions that support business intelligence and advanced analytics initiatives . Mandatory Skills: Microsoft Fabric SQL PySpark Nice to Have: Power BI

LOOKING FOR IMMEDIATE/ EARLY JOINERS ONLY. Role: Lead Data engineer Location: Bengaluru Mode: 5 days work from office (Alternate Saturday work from home) Budget: 20 LPA- 30 LPA Max. A Data Engineer with PySpark experience develops, maintains, and optimizes scalable data pipelines and infrastructure, focusing on data extraction, transformation, and loading (ETL/ELT) processes using PySpark, Python, and SQL. ETL/ELT Process Optimization Data System Design Data Quality & Integrity Collaboration Performance Optimization Essential Skills & Qualifications: Programming Languages : Proficiency in Python and advanced SQL PySpark : Strong hands-on experience with PySpark, DataFrames, RDDs, and SparkSQL Cloud Platforms : Experience with cloud data platforms such as AWS (S3, Glue, Redshift), Azure (Databricks), or GCP Big Data Technologies : Strong understanding of big data concepts, distributed computing, and parallel processing frameworks DevOps & CI/CD : Familiarity with DevOps principles, source control systems (e.g., Git), and CI/CD tools

Role & responsibilities Bachelors or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related technical field. 7+ years of experience in data engineering, solution architecture, or enterprise data management roles. 5+ years of experience leading delivery of complex data integration or analytics solutions in a cloud environment (GCP, AWS, or Azure). 5+ years of hands-on experience in data modeling, database design, and architecture (dimensional, normalized, and canonical modeling). Expertise in cloud-native data architecture, particularly with Google Cloud Platform (BigQuery, Cloud Storage, Dataflow, Cloud Composer). Deep understanding of ETL/ELT frameworks, batch and streaming architectures, and orchestration tools (Airflow/Cloud Composer). Proven ability to design dimensional and canonical data models for enterprise analytics and reporting use cases. Hands-on experience with SQL, Python, and modern data engineering toolsets. Solid grasp of data governance, security, quality, and metadata management principles. Strong understanding of MDM solutions and data stewardship principles. Experience managing cross-functional delivery teams in Agile or hybrid delivery environments. Excellent communication and stakeholder management skills. Experience in Retail, Media, or Consumer Products industries is a plus Working knowledge of machine learning/AI, LLMs, or agentic AI data system design is a plus

Data_and_MLOps_Engineer - Contract - onsite Experience within the Azure ecosystem, including Azure AI Search, Azure Storage Blob, Azure Postgres, with expertise in leveraging these tools for data processing, storage, and analytics tasks. Proficiency in data preprocessing and cleaning large datasets efficiently using Azure Tools, Python, and other data manipulation tools. Strong background in Data Science/MLOps, with hands-on experience in DevOps, CI/CD, Azure Cloud computing, and model monitoring. Expertise in healthcare data standards, such as HIPAA and FHIR, with a deep understanding of sensitive data handling and data masking techniques to protect PII and PHI. In-depth knowledge of search algorithms, indexing techniques, and retrieval models for effective information retrieval tasks. Experience with chunking techniques and working with vectors and vector databases like Pinecone. Ability to design, develop, and maintain scalable data pipelines for processing and transforming large volumes of structured and unstructured data, ensuring performance and scalability. Implement best practices for data storage, retrieval, and access control to maintain data integrity, security, and compliance with regulatory requirements. Implement efficient data processing workflows to support the training and evaluation of solutions using large language models (LLMs), ensuring that models are reliable, scalable, and performant. Proactively identify and resolve data quality issues, pipeline failures, or resource contention to minimize disruption to systems. Experience with large language model frameworks, such as Langchain, and the ability to integrate them into data pipelines for natural language processing tasks. Familiarity with Snowflake for data management and analytics, with the ability to work within the Snowflake ecosystem to support data processes. Knowledge of cloud computing principles and hands-on experience with deploying, scaling, and monitoring AI solutions on platforms like Azure, AWS, and Snowflake. Ability to communicate complex technical concepts effectively to both technical and non-technical stakeholders, and collaborate with cross-functional teams. Analytical mindset with attention to detail, coupled with the ability to solve complex problems efficiently and effectively. Knowledge of cloud cost management principles and best practices to optimize cloud resource usage and minimize costs. Experience with ML model deployment, including testing, validation, and integration of machine learning models into production systems. Knowledge of model versioning and management tools, such as MLflow, DVC, or Azure Machine Learning, for tracking experiments, versions, and deployments. Model monitoring and performance optimization, including tracking model drift and addressing performance issues to ensure models remain accurate and reliable. Automation of ML workflows through CI/CD pipelines, ensuring efficient delivery, testing, and deployment. Monitoring and logging of AI/ML systems post-deployment to ensure reliability and performance. Collaboration with data scientists and engineering teams to deliver integrated retraining, fine-tuning, and updating. Familiarity with containerization technologies, such as Docker and Kubernetes, for scaling machine learning models in production environments. Ability to implement model governance practices to ensure compliance and provide transparency into models in production environments. Understanding of model explainability and interpretability techniques to provide transparent insights into model behavior. Must Have: Minimum of 10 years experience as a data engineer. Hands-on experience using Azure Cloud ecosystem. Deep knowledge in AI/ML and MLOps. Hands-on experience with DevOps, CI/CD, model monitoring. Hands-on experience in healthcare domain. Experience with containerization technologies. Hands-on experience working with unstructured data. Good to Have: Experience with Snowflake, MLflow, DVC, experiment intelligence, Snowflake, function app, Azure AI Search.

Bachelors or masters degree in Computer Science, IT, or related field. 5+ years of data engineering experience, including 1+ years on Azure Data Platform. Strong proficiency in Azure Data Factory, Azure Fabric, and Notebooks (PySpark/Python/SQL). Expertise in Delta Lake, Parquet, and modern data lakehouse architectures. Experience with Azure Synapse Analytics, Databricks (if applicable), and data visualization tools such as Power BI. Skilled in SQL, Python, and PySpark for large-scale data transformations. Experience implementing CI/CD for data pipelines and notebooks using Azure DevOps or Git. Knowledge of data governance, security best practices, and compliance frameworks. If youre ready to make an impact with your expertise in data engineering and Azure, this is the role for you!

Key Skills: Google Cloud Platform, Data Engineer, Data Engineering Roles and Responsibilities: Design and implement scalable data pipelines and ETL/ELT workflows using GCP-native tools. Develop and optimize relational and non-relational databases for performance and reliability. Build and manage data lakes and warehouses using BigQuery, Cloud Storage, and Dataproc. Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions. Ensure data governance, security, and compliance with industry standards. Lead architectural discussions and mentor junior engineers. Hands-on experience with GCP services: BigQuery, Dataflow, Dataproc, Cloud Storage, Cloud Composer, Pub/Sub. Proficiency in Python, PySpark for data processing. Experience of large size database migration to GCP using DMS, ETL tools, etc. Experience with big data tools (Apache Spark, Kafka, Beam). Familiarity with CI/CD pipelines and infrastructure-as-code (Terraform, Deployment Manager). Excellent problem-solving and communication skills. Skills Required: Expertise in GCP data tools: BigQuery, Dataflow, Dataproc, Cloud Storage Proficiency in Python and PySpark for scalable data processing Solid SQL and database design skills (BigQuery, PostgreSQL, MySQL) Experience with ETL/ELT workflows and large-scale data migrations Strong understanding of CI/CD and Infrastructure-as-Code (Terraform, Deployment Manager) Familiarity with big data technologies like Apache Spark, Kafka, Beam Excellent stakeholder collaboration and problem-solving skills Education: Bachelor's Degree in related field