2103 Data Engineer Jobs - Page 33

1. The resource should have knowledge on Data Warehouse and Data Lake 2. Should aware of building data pipelines using Pyspark 3. Should be strong in SQL skills 4. Should have exposure to AWS environment and services like S3, EC2, EMR, Athena, Redshift etc 5. Good to have programming skills in Python

Skill: Hadoop-hive, Pyspark, SQL and Unix 4.5 to 5.5 years experience as Data Engineering side using Big Data, Postgres, Spark skillsAbility to work as an independent developer Good in understand the tables structures and ability to write Complex SQL queries Good understanding on the DHW concepts and Basic knowledge on Unix/Shellscript Good in Written and communication skills Design end-to-end Data pipeline and optimize Data ingestion Pipelines and manage data flow to the upstream for consumption

We are looking for a highly skilled Data Engineer to join our team at Crest Infosystems. The ideal candidate will have a strong background in data engineering and be able to design, develop, and implement large-scale data systems. Roles and Responsibility Design and develop scalable data pipelines using various technologies. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data warehouses and data lakes. Ensure data quality and integrity by implementing data validation and testing procedures. Optimize data systems for performance and efficiency. Troubleshoot and resolve complex technical issues related to data engineeri...

Qualification : B.Tech MCA BCA B.Sc (Computer Science or related field) Notice Period : Immediate Joiners Preferred We Have an Immediate Requirement For Azure Data Engineers at Senior and Lead levels to work on cutting-edge data projects in Azure ecosystem. Primary Skills: Azure Data Factory (ADF) Azure Databricks SQL Development & Optimization Power BI Reports, Dashboards, DAX Data Integration & ETL Pipelines Data Modeling and Transformation Secondary Skills: Azure Synapse Analytics Python PySpark in Databricks DevOps exposure for data pipelines Version control using Git Exposure to Agile & CI/CD Environments Strong communication and client-handling skills Key Responsibilities: Design and i...

Role : AWS Data Engineer Experience: 7 to 12 years Location : Hyderabad Notice Period : Immediate to 7 days Budget : up to 30 LPA Job Overview: We are looking for an experienced Senior Data Engineer to design, build, and optimize scalable, high-performance data platforms using AWS cloud services and Python. The ideal candidate will play a key role in architecting end-to-end data pipelines, driving automation, ensuring data quality, and enabling analytics and AI workloads across the organization. This role requires deep technical expertise in AWS data services, modern data architecture, and a passion for delivering reliable, high-quality data solutions at scale. Required Skills & Qualificatio...

Opti Value Tek is seeking a skilled Senior Data Engineer with strong expertise in SQL, Looker, and Snowflake to join our team. The ideal candidate will design, develop, and optimize scalable data pipelines, ensuring reusability, performance, and business alignment. Roles & Responsibilities Write reusable, modular SQL code for data engineering solutions. Work with dimensional modelling (facts, dimensions, SCD, star schemas). Build and maintain Looker semantic layers over Snowflake with focus on scalability and accuracy. Define LookML models, explores, views, and dashboards. Collaborate with business teams to define metrics and dimensions aligned to business goals. Ensure design and QA alignme...

Senior Data Engineer with strong expertise in SQL, Looker, and Snowflake f2f interview contract Opti Value Tek is seeking a skilled Senior Data Engineer with strong expertise in SQL, Looker, and Snowflake to join our team The ideal candidate will design, develop, and optimize scalable data pipelines, ensuring reusability, performance, and business alignment Roles & Responsibilities Write reusable, modular SQL code for data engineering solutions Work with dimensional modelling (facts, dimensions, SCD, star schemas) Build and maintain Looker semantic layers over Snowflake with focus on scalability and accuracy Define LookML models, explores, views, and dashboards Collaborate with business team...

Joining : Immediate to 15 days Key Skills: Strong hands-on experience in PySpark Expertise in Big Data technologies Solid understanding of SQL, Data Pipelines, and ETL.

Candidate Specification : Minimum 3+ years of experience into Data Engineering (GCP+Pyspark) Job Description: Expertise in SQL programming and database management systems Experience with building ETL pipelines in object-oriented/object function scripting languages Apache Spark Experience in managing and Maintaining Pipelines Google Cloud Platform (GCP), and experience with relevant services (eg GCP Dataflow, GCP DataProc, Biq Query, Procedures, Cloud Composer etc) Experience with tools like Airflow/cloud composer Experience with BI tools and visualization platforms (eg Tableau) is a plus Contact Person- Sheena Rakesh Email id- sheena@gojobs.biz

Candidate Specification: Minimum 5+ years of experience into Agentic AI AI Integration: Implement LLM-based workflows, RAG pipelines, and agentic orchestration using frameworks like LangChain, LangGraph, CrewAI, or similar Data Pipeline Development: Design, develop, and maintain efficient and scalable data pipelines using Python and Azure tech stack Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs Automation & Deployment: Containerize and deploy solutions using Docker, Kubernetes, and cloud-native architectures (Azure/AWS/GCP) Location - Bangalore, Pune, Hyderabad, Chennai, Noida,...

Job Summary: We are seeking a skilled Data Engineer with expertise in Data Modeling, SQL, Snowflake, Python, AWS, and NoSQL . The ideal candidate will be responsible for designing and implementing scalable data solutions, ensuring efficient data storage, retrieval, and processing. Experience in NoSQL Data Modeling is a plus. Key Responsibilities: Design and implement data models to support analytical and operational workloads. Develop, optimize, and manage SQL queries for data extraction, transformation, and loading (ETL). Work extensively with Snowflake to build scalable data pipelines and warehouses. Develop and maintain Python scripts for data processing and automation. Implement AWS serv...

Key Responsibilities: Design, develop, and optimize complex PL/SQL procedures, packages, triggers, and functions . Write efficient Python scripts for data transformation, ETL automation, system integration, and reporting tasks. Analyze business requirements and translate them into technical specifications. Troubleshoot and resolve performance issues in PL/SQL queries and stored procedures. Ensure data integrity, quality, and availability across systems. Integrate data between systems using Python (e.g., REST APIs, flat files, database connectors). Support deployment, monitoring, and maintenance of applications in production. Collaborate with data engineers, analysts, and application develope...

Responsibilities include : Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data. Develop data pipelines to extract and transform data in near real-time using cloud-native technologies. Implement data validation and quality checks to ensure accuracy and consistency. Monitor system performance, troubleshoot issues, and implement optimizations to enhance reliability and efficiency . Collaborate with business users, analysts, and other stakeholders to understand data requirements and deliver tailored solutions. Document technical designs, workflows, and best practices to facilitate knowledge sharin...

Requirements: Must have experience as a Data Scientist or transitioned from Data Engineer Proficient in AWS Expertise in ML Model Design & Development Strong Feature Engineering experience Good to Have: NLP, GraphQL Graph ML, Generative AI Education: B.Tech M.Tech in Computer Science, IT, or ECE

We are hiring a Senior Data Engineer for a 6-month remote contract position. The ideal candidate is highly skilled in building scalable data pipelines and working within the Azure cloud ecosystem, especially Databricks, ADF, and PySpark. You'll work closely with cross-functional teams to deliver enterprise-level data engineering solutions. #KeyResponsibilities Build scalable ETL pipelines and implement robust data solutions in Azure. Manage and orchestrate workflows using ADF, Databricks, ADLS Gen2, and Key Vaults. Design and maintain secure and efficient data lake architecture. Work with stakeholders to gather data requirements and translate them into technical specs. Implement CI/CD pipeli...

Requirements: Must have experience as a Data Scientist or transitioned from Data Engineer Proficient in AWS Expertise in ML Model Design & Development Strong Feature Engineering experience Good to Have: NLP, GraphQL Graph ML, Generative AI Education: B.Tech M.Tech in Computer Science, IT, or ECE Location: Delhi NCR,Bangalore,Chennai,Pune,Kolkata,Ahmedabad,Mumbai,Hyderabad

Role : Azure Data Engineer Senior Lead Experience : 5 to 10+ Years Qualification : B.Tech MCA BCA B.Sc (Computer Science or related field) Notice Period: Immediate Joiners Preferred We Have an Immediate Requirement For Azure Data Engineers at Senior and Lead levels to work on cutting-edge data projects in Azure ecosystem. Primary Skills: Azure Data Factory (ADF) Azure Databricks SQL Development & Optimization Power BI Reports, Dashboards, DAX Data Integration & ETL Pipelines Data Modeling and Transformation Secondary Skills: Azure Synapse Analytics Python PySpark in Databricks DevOps exposure for data pipelines Version control using Git Exposure to Agile & CI/CD Environments Strong communica...

Experience : 3--5 years (DS) 5--8 years (Sr DS) Positions : 3 (2 DS + 1 Sr DS) Requirements: Must have experience as a Data Scientist or transitioned from Data Engineer Proficient in AWS Expertise in ML Model Design & Development Strong Feature Engineering experience Good to Have: NLP, GraphQL Graph ML, Generative AI, D365 CRM Integrations Specialist Azure Logic Apps, Power Automate, and Azure Service Bus. Education: B.Tech M.Tech in Computer Science, IT, or ECE

Greetings Everyone We are seeking a big data engineer with Java experience to join our modernization initiative team. The candidate will be responsible for implementing and maintaining high volume online and offline batch and transactional client facing applications in cloud and On-prem. Candidate will also responsible for the creation and maintenance of our Code & database infrastructure, including collecting and maintaining data, ensuring the integrity of our data, and training data models. Responsibilities: * Design and implement efficient data handling system based on Java and Spark. * Perform and oversee tasks such as programming, writing scripts, calling APIs, and writing complex SQL q...

What will you do? What are we looking for? Technical Skill set: Having 3-5 years of experience in implementingBig Data solution in Cloud environments Hands-on in implenting Big data solution using Airflow DAG,Spark,Python,Hive,Trino,S3 Bucket,etcHaving experience Cloud Data platform like SnowFlake , Databricks is a plus Knowledge of Financial domain-Credit Card industry is a plus Responsibilites: Creating an ELT Data pipelines using python script and Airflow DAG Optimizing the Data pipelines by configuring the Spark system resourcesDebug an existing Microsoft BI ETLs and understand business logic and convert in to big data pipeline using Airflow DAG Able to interact with business partners to...

Technical Manager, Data Engineering Background Mastercard is on a journey to modernize and advance our Finance Technology landscape, covering Billing, Financial Planning, Accounting, Settlement, Treasury, Reporting and Analytics This position will be responsible for developing and delivering high-impact technology solutions that align with our business and technical objectives By collaborating with cross-functional teams, including business/product owners and other technical experts, you will ensure that our solutions meet evolving customer needs while improving performance, scalability, and reliability Responsibilities Manage a technical team, mentoring them, providing technical guidance, m...

Employment Type: Contract / Permanent (as per project need) Senior Data Engineer with strong expertise in SQL, Looker, and Snowflake f2f interview contract Opti Value Tek is seeking a skilled Senior Data Engineer with strong expertise in SQL, Looker, and Snowflake to join our team The ideal candidate will design, develop, and optimize scalable data pipelines, ensuring reusability, performance, and business alignment Roles & Responsibilities Write reusable, modular SQL code for data engineering solutions Work with dimensional modelling (facts, dimensions, SCD, star schemas) Build and maintain Looker semantic layers over Snowflake with focus on scalability and accuracy Define LookML models, ex...

As an Associate Software Developer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big ...

As Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets Required education Bachelor's Degree Preferred education Master's Degree Required technical and professional expertise Expertise in Data warehousing/ information Management/ Data Integra...

* Hiring manager As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the client’s needs. * Your primary responsibilities include* Design, build, optimize and support new and existing data models and ETL processes based on our client’s business requirements * Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. * Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need Requir...