14139 Data Engineering Jobs - Page 23

We are seeking a skilled Data Engineer with strong expertise in Python, PySpark, and the AWS Data Engineering stack to design, develop, and optimize large-scale ETL pipelines. The ideal candidate will have a solid foundation in data engineering best practices, automation, and modern cloud-based architectures, along with a passion for leveraging Generative AI tools to boost productivity and code quality. Core Must-Have Skills: Strong proficiency in Python and PySpark for developing and optimizing ETL pipelines. In-depth understanding of data engineering best practices , including data validation, transformation logic, and performance optimization. Experience working with large-scale data proc...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and Azure Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark and Hive, Hbase or other NoSQL databases on Azure Cloud Data Platform or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developing strea...

Databricks: AWS: S3, DMS, Redshift, EC2, VPC, Lambda, Delta Lake, CloudWatch etc. Bigdata: Databricks, Spark, Glue and Athena Expertise in Lake Formation, Python programming, Spark, Shell scripting Minimum Bachelors degree with 5+ years of experience in designing, building, and maintaining Databricks and AWS data components 3+ years of experience in data component configuration, related roles and access setup Expertise in Python programming Knowledge in all aspects of DevOps (source control, continuous integration, deployments, etc.) Strong hands-on coding skills in Python, processing large-scale data set and developing machine learning models. Experience programming in Python, R, and SQL.

The Impact you will have in this role: Work with modern PAM technologies in a supportive, learning-focused environment. Knowledge Sharing: Collaborate with IT, risk, and cybersecurity teams, contribute to documentation, and participate in technical workshops and team reviews to expand your expertise Be part of a team committed to innovation, operational excellence, and professional development. Over time, you will have the opportunity to cross-train in other IAM tools and technologies, including Active Directory, HashiCorp Vault, Terraform, Ansible, PKI/HSM, and Identity Governance platforms. Were seeking a hands-on, detail-oriented Senior/Lead PAM Systems Engineer who combines technical exp...

The Impact you will have in this role: IAM Tech Ops team is dedicated to ensure availability, security, and seamless operation of the organizations IAM systems. We play a critical role in safeguarding corporate data and enhancing user experience by managing system upgrades, onboarding and offboarding processes, troubleshooting access issues, and implementing robust observability and monitoring practices. Through proactive support and continuous improvement, the team ensures secure and reliable identity services across the enterprise. Key Responsibilities Assist in designing and implementing systems to discover and track certificates, keys, and cryptographic protocols across endpoints and ser...

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities Strong programming skills in Python, Java, C++, SQL etc. Deep expertise with cloud platforms like AWS, Azure, or GCP. Hands-on experience with Databricks, Snowflake, AWS RDS, Azure SQL, or GCP Cloud SQL, Apache Spark and Apache Airflow. Strong understanding of Lakehouse architecture, Data Mesh principles, data governance frameworks, and modern data pipelines. Proven ability to deliver high-impact, API-first services at scale. Proficiency with tools like Terraform or AWS CloudFormation is critical for managing cloud infrastructure programmatically, ensuring consistency, and enabling automation...

Were seeking a Consulting Lead-Databricks with strong experience in Databricks, Python, and PySpark, along with hands-on knowledge of Unity Catalog (UC) and Lakehouse architecture. In this role, youll design and optimize large-scale data pipelines, implement robust governance and security through UC, and build scalable solutions in a modern cloud data environment. The ideal candidate must have 5+ years in data engineering, expertise with any one of the cloud platforms (Azure/AWS/GCP), and proven ability to deliver high-performing ETL/ELT pipelines that support business-critical analytics. Qualification Hands-on expertise in Databricks, Python, PySpark, and Delta Lake/Lakehouse architecture. ...

Role & responsibilities Data & ML Engineering Leadership Build and scale ML engineering and data engineering functions. Establish MLOps frameworks for standardized, production-grade model development and monitoring. Design and operationalize a centralized decisioning platform that integrates low-code model development, AutoML, rule engines, and workflow automation. Build modern, scalable data platforms (real-time ingestion, lakehouse, event-driven systems). Preferred candidate profile 15 ~ 20 years of experience in data engineering, ML engineering, or platform leadership, with at least 8 ~10 years in senior management roles. Build modern, scalable data platforms (real-time ingestion, lakehou...

Interested candidates can apply or share cv at aishwarya.joshi@espire.com Job Summary: We are looking for a Senior Agile Project Manager with 10+ years of experience in managing complex technology and data projects within Agile environments. The ideal candidate will have strong expertise in cloud cost optimization , Databricks workspace management , and Agile/Scrum project delivery . This role requires both strategic oversight and hands-on management to drive successful execution across cloud and data initiatives. Key Responsibilities: Project Management & Delivery Lead multiple end-to-end Agile projects across cloud, data, and analytics domains. Manage project scope, timelines, budgets, and...

Notice Period-Immediate-10 days The Presales Solutions Engineer partners with Sales to qualify opportunities and craft winning, scalable solutions on Google Cloud. The role blends hands-on GCP expertise across Data Engineering, Analytics, AI/ML/GenAI, and App Modernization with strong consulting, estimation, and proposal-building skills. You will translate complex business problems into feasible architectures, author LOEs and SOWs, and present compelling narratives to both technical and executive stakeholders. Success requires collaborating across Sales, Delivery, and Product teams in global setups, while balancing detail with big-picture outcomes. Required Qualifications 8+ years overall ex...

JOB DISCRIPTION Experience: 6+Years Location: Pune 4+ years of relevant IT experience in the BI/DW domain with minimum 2 years of hands-on experience on Azure modern data platform that includes Data Factory, Databricks, Synapse (Azure SQL DW) and Azure Data Lake . Expert in creating data structures optimized for storage and various query patterns for e.g. Parquet and Delta Lake . Experience in traditional data warehousing / ETL tools (Informatica, IBM Datastage, Microsoft SSIS) Proficient at working with large and complex code bases (Github, Gitflow, Fork/Pull Model) • Working experience in Agile methodologies (SCRUM, XP, Kanban) • Data Modelling - One to Three Years • Developer / Software E...

Role & responsibilities Designing, building, and maintaining ETL/ELT pipelines using dbt Creating and managing data models and transformations using dbt to ensure efficient and accurate data consumption for analytics and reporting. Developing and maintaining a data quality framework, including automated testing and cross-dataset validation. Writing and optimizing SQL queries for efficient data processing within cloud components. Working with data engineers, analysts, scientists, and business stakeholders to deliver data solutions. Supporting day-to-day incident and ticket resolution related to data pipelines. Creating and maintaining comprehensive documentation for data pipelines, configurat...

Core Purpose To design, build, and maintain scalable data pipelines and platforms that enable secure, efficient, and reliable data processing. This role focuses on transforming raw data into structured, high-quality datasets for analytics, AI/ML, and business intelligence. The Data Engineer collaborates with stakeholders to deliver robust data solutions that drive innovation and operational excellence. Key Responsibilities Data Engineering and Pipeline Development Build and maintain ETL/ELT pipelines for batch and streaming data. Develop data ingestion frameworks integrating APIs, transactional systems, and streaming platforms. Optimize data workflows for performance and cost efficiency in c...

As a Data Engineer at Micron Technology, you will play a crucial role in developing, delivering, and supporting data engineering and advanced analytics solutions. Here are the key responsibilities you will be handling: - Utilize your 6+ years of experience to develop and deliver data engineering and advanced analytics solutions. - Demonstrate your 3-5 years of hands-on experience in building Cloud Data centric solutions in Snowflake, GCP, or other cloud platforms. - Showcase your proficiency with ETL and Orchestration tools. - Leverage your programming knowledge, preferably in Python. - Utilize your proficiency in various database management systems including Snowflake, SQL Server, HANA, Ora...

Role Overview: You will be responsible for performing ETL testing, including data validation transformation checks and source-to-target mapping verification. You will also be required to write and execute complex SQL queries for data quality and integrity checks. Additionally, you will develop and maintain automation test scripts using Playwright. Collaboration with development and data engineering teams to identify issues and ensure high-quality releases will be a key part of your role. Documenting test results and supporting defect resolution will also be within your responsibilities. Key Responsibilities: - Perform ETL testing, data validation transformation checks, and source-to-target m...

We have developed API gateway aggregators using frameworks like Hystrix and spring-cloud-gateway for circuit breaking and parallel processing. Our serving microservices handle more than 15K RPS on normal days and during saledays this can go to 30K RPS. Being a consumer app, these systems have SLAs of ~10ms Our distributed scheduler tracks more than 50 million shipments periodically fromdifferent partners and does async processing involving RDBMS. We use an in-house video streaming platform to support a wide variety of devices and networks. What Youll Do Design and implement scalable and fault-tolerant data pipelines (batch and streaming) using frameworks like Apache Spark , Flink , and Kafka...

Role Overview: As a Data Engineering Tech Lead at EY Consulting in Hyderabad, you will be responsible for designing, developing, and optimizing data pipelines and architectures to support scalable and efficient data processing. Your role will involve working with complex datasets, integrating multiple data sources, and ensuring high data quality and reliability. Key Responsibilities: - Develop and maintain efficient ETL (Extract, Transform, Load) pipelines that ingest, process, and transform large datasets from diverse sources. - Build and manage data storage solutions using relational and non-relational databases, ensuring high availability, performance, and scalability. - Work with APIs, t...

As the Delivery Manager AI & Digital Transformation, your role involves leading the end-to-end delivery of AI, Data, Cloud, and Digital Transformation programs for global clients. Your expertise in managing large delivery teams, ensuring operational excellence, customer satisfaction, and scaling digital practices will be essential. You should be a strategic thinker with hands-on experience in implementing AI/ML solutions and enterprise digital initiatives. Key Responsibilities: - Lead end-to-end delivery of AI, ML, Data Engineering, Cloud, and Digital Transformation engagements. - Ensure on-time, high-quality execution aligned with client expectations and SLAs. - Establish delivery governanc...

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities In-depth knowledge of the Palantir Foundry platform, including data integration (Data Connections), Data Transformation (Code Repository & Pipeline Builder), analysis (Contour and Quiver), visualization (Workshop) and Ontology Manager. Must also have a good knowledge of Spark (PySpark) & TypeScript. Knowledge on AIP is an added advantage. Additional Responsibilities: Ability to develop value-creating strategies and models that enable clients to innovate, drive growth and increase their business profitability Good knowledge on software configuration management systems Awareness of latest techn...

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities In-depth knowledge of the Palantir Foundry platform, including data integration (Data Connections), Data Transformation (Code Repository & Pipeline Builder), analysis (Contour and Quiver), visualization (Workshop) and Ontology Manager. Must also have a good knowledge of Spark (PySpark) & TypeScript. Knowledge on AIP is an added advantage. Additional Responsibilities: Ability to develop value-creating strategies and models that enable clients to innovate, drive growth and increase their business profitability Good knowledge on software configuration management systems Awareness of latest techn...

As a Technical Lead specializing in Appian components such as Process Designing, SAIL, Rules, Security, and Integration, you will spearhead the design, development, and ongoing enhancement of cutting-edge software solutions. Your primary responsibilities will include leading the technical design and solutioning processes, crafting and deploying high-performance, scalable microservices using Spring Boot and other relevant technologies. The ideal candidate should have a strong background in Java and JavaScript development, Microservices architecture, and a passion for delivering high-quality software. Join a team of passionate engineers at ValueMomentums Engineering Center, who thrive on tackl...

General Summary: Data Platform, Engineering and Operations lead will play a pivotal role in shaping the future of our data capabilities as part of strategic and critical Enterprise Data Transformation (EDT) program. We're seeking a visionary, hands-on leader to drive the build out of a modern enterprise-wide lake house, ensuring reliability, scalability, security, and compliance. and enabling cutting edge analytics solutions. This role will be leading a talented team of platform and data engineers, fostering a culture of excellence while balancing speed of delivery with innovation and operational excellence. This role will also collaborate with cross functional teams (Solution engineers, SME...

About The Role Job Title :Industry & Function AI Data Engineer + S&C GN Management Level :09 - Consultant Location :Primary - Bengaluru, Secondary - Gurugram Must-Have Skills :Data Engineering expertise, Cloud platforms:AWS, Azure, GCP, Proficiency in Python, SQL, PySpark and ETL frameworks Good-to-Have Skills :LLM Architecture, Containerization tools:Docker, Kubernetes, Real-time data processing tools:Kafka, Flink, Certifications like AWS Certified Data Analytics Specialty, Google Professional Data Engineer,Snowflake,DBT,etc. Job Summary : As a Data Engineer, you will play a critical role in designing, implementing, and optimizing data infrastructure to power analytics, machine learning, an...

About The Role Project Role : Cloud Platform Engineer Project Role Description : Designs, builds, tests, and deploys cloud application solutions that integrate cloud and non-cloud infrastructure. Can deploy infrastructure and platform environments, creates a proof of architecture to test architecture viability, security and performance. Must have skills : AWS Analytics Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Cloud Platform Engineer, you will engage in the design, construction, testing, and deployment of cloud application solutions that seamlessly integrate both cloud and non-cloud infrastructu...

About The Role Job Title - Decision Science Practitioner Analyst S&C GN Management Level:Analyst Location:Bangalore/ Kolkata/ Gurugram/Hyderabad Must have skills: Generative AI Good to have skills: Data Science Job Summary : We are seeking a highly skilled and motivated Data Science Analyst to lead innovative projects and drive impactful solutions in domains such as Consumer Tech , Enterprise Tech , and Semiconductors . This role combines hands-on technical expertise , and client delivery management to execute cutting-edge projects in Generative AI, decision science, and cloud-based analytics. Key Responsibilities Generative AI Expertise Develop fine-tune models for NLP, Computer Vision, and...