354 Aws Emr Jobs - Page 3

Shuru is a self-managed technology team specializing in accelerating visions through product, technology, and AI leadership. With a focus on bespoke execution, we deliver impactful solutions that are scalable and designed for success. At Shuru, we deliver mobile solutions that meet and exceed customer expectations. Our collaborative and fast-paced environment encourages creativity and innovation. Our Data Platform team is hiring a Senior Data Engineer to build and maintain scalable pipelines, shape our data architecture on Azure Databricks, and mentor other engineers. Responsibilities: Design, build, and maintain efficient and scalable data pipelines that collect, transform, and load data. D...

Your role at WRD involves working on cutting-edge data engineering challenges by merging human expertise with cutting-edge AI technology. As part of the team, you will be responsible for the following key responsibilities: - Develop & Maintain data pipelines to support business and analytical needs. - Work with structured and unstructured data to ensure high data quality and availability. - Build scalable and efficient data workflows using SQL, Python, and Java/Scala. - Manage and process large datasets on AWS EMR and other cloud platforms. - Develop and maintain batch and stream data processing jobs using tools like Spark (preferred). - Deploy, monitor, and troubleshoot workflows on orchest...

As a Java developer at the company, your responsibilities will include: - Working with client developers & off-shore developers to deliver project tasks - Analyzing system domain and client requirements - Collaborating with the team to create and maintain a robust system supporting business with a focus on usability - Ensuring productivity and teamwork within the constraints of a planned project to complete necessary milestones - Solving challenging real-world problems using OOP & functional programming concepts - Writing production-grade backend code, SP, and upholding high code quality standards You must have the following qualifications: - Very strong knowledge in OOP and Core Java - Inhe...

Design, develop new features for a large-scale cloud data platform using Databricks, snowflake and Fabric, Deeply understand legacy data platforms, and drive their modernization, Provide production support Design data pipelines and data warehouse. Required Candidate profile 5 years in developing and scaling data platforms using Snowflake, databricks, Fabric, Strong Python and SQL Intermediate C# Understanding of modern data architectures such as Data Mesh, Lakehouse

The Role: Enterprise Architect - Integration The Team: The OSTTRA Technology team is composed of Capital Markets Technology professionals, who build, support and protect the applications that operate our network. The technology landscape includes high-performance, high-volume applications as well as compute intensive applications, leveraging contemporary microservices, cloud-based architectures. The Impact: Together, we build, support, protect and manage high-performance, resilient platforms that process more than 100 million messages a day. Our services are vital to automated trade processing around the globe, managing peak volumes and working with our customers and regulators to ensure the...

Preferred candidate profile Mainly looking for 1) AWS - GLUE - EMR - SPARK 2) Clear concepts of DE Job Summary: We are seeking a skilled and experienced Data Engineer to join our team in Pune. The ideal candidate will have a strong foundation in data engineering principles and hands-on expertise with AWS services such as Glue , EMR , and Spark . You will be responsible for designing, building, and maintaining scalable data pipelines and solutions that support business analytics and decision-making. Key Responsibilities: Design, develop, and maintain robust ETL/ELT pipelines using AWS Glue and Spark Work with large-scale distributed data processing systems on AWS EMR Optimize data workflows f...

Work Location: Hyderabad /Kolkata Experience :6-10yrs Required Skills: Experience AWS cloud and AWS services such as S3 Buckets, Lambda, API Gateway, SQS queues; Experience with batch job scheduling and identifying data/job dependencies; Experience with data engineering using AWS platform and Python; Familiar with AWS Services like EC2, S3, Redshift/Spectrum, Glue, Athena, RDS, Lambda, and API gateway; Familiar with software DevOps CI/CD tools, such Git, Jenkins, Linux, and Shell Script Thanks & Regards Suganya R suganya@spstaffing.in

Role & responsibilities Expansion of Data Products Support pipeline buildout, implementation, and RFC deployments for new business use cases • Assignment group/project repo segregation / automated CICD deployments for better organization and support • Enhance existing pipelines for performance and reliability Platform Integrations Falcon Shield (security & threat intelligence) • Collibra (data governance & cataloging) • XSOAR (security orchestration) • BIG ID (data privacy & compliance) • SPLUNK (SIEM & monitoring) Evidence Collection & Compliance Enable traceability of pipelines for audit readiness • Collaborate with ISS to collect and publish control implementation evidence • Support onboa...

Role Overview: At Toyota Connected, you will have the opportunity to be a part of a team that is dedicated to transforming the automotive industry and positively impacting others on a global scale. In a collaborative and fast-paced environment, we focus on continual improvement and deliver exceptional value through connected products and services that wow and delight our customers. Key Responsibilities: - Design, develop, and maintain automation test frameworks and scripts specifically for API-level Testing - Validate and verify complex Machine Learning applications, ensuring models and data pipelines perform accurately in the production environment - Conduct extensive testing on large-scale...

Ascendeum is looking for veterans with extensive hands-on experience in the field of data engineering to build cutting-edge solutions for large-scale data extraction, processing, storage, and retrieval. About Us: We provide AdTech strategy consulting to leading Internet websites and apps globally hosting over 200 million monthly worldwide audiences. Since 2015, our team of consultants and engineers have been consistently delivering intelligent solutions that enable enterprise-level websites and apps to maximize their digital advertising returns. Job Responsibilities: Understand long-term and short-term business requirements to precisely match them with the capabilities of different distribut...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

We are looking for an experienced Data Engineering Manager with a strong foundation in Python, SQL, and Spark , and hands-on expertise in AWS, Databricks . In this role, you will build and maintain scalable data pipelines and architecture to support analytics, data science, and business intelligence initiatives. Youll work closely with cross-functional teams to drive data reliability, quality, and performance. Responsibilities: Design, develop, and optimize scalable data pipelines using Databricks in AWS such as Glue, S3, Lambda, EMR, Databricks notebooks, workflows and jobs. Building data lake in WS Databricks. Build and maintain robust ETL/ELT workflows using Python and SQL to handle struc...

Summary Position Summary Artificial Intelligence & Engineering (AI&E) Join our AI & Engineering team in transforming technology platforms, driving innovation, and helping make a significant impact on our clients success. You'll work alongside talented professionals reimagining and re-engineering operations and processes that are critical to businesses. Your contributions can help clients improve financial performance, accelerate new digital ventures, and fuel growth through innovation. AI & Engineering leverages cutting-edge engineering capabilities to build, deploy, and operate integrated/verticalized sector solutions in software, data, AI, network, and hybrid cloud infrastructure. These so...

As an ETL Developer at our company, you will be responsible for designing, developing, and implementing performant ETL pipelines using Python API (pySpark) of Apache Spark on AWS EMR. Your role will involve the following key responsibilities: - Programming with Python with a strong proficiency in Python - Utilizing functional programming concepts and Data Modelling - Hands-on experience in developing ETL data pipelines using pySpark on AWS EMR - Understanding Sparks Data frame and API - Configuring EMR clusters on AWS - Dealing with AWS S3 object storage from Spark - Troubleshooting Spark jobs and monitoring them using Spark UI Qualifications required for this role include: - 5+ years of exp...

Description: Responsibilities : Design and implement scalable data pipelines using AWS Glue for ETL processes. Develop data processing applications with Python ensuring data integrity and efficiency. Utilize AWS Lambda to create serverless solutions that automate data workflows. Monitor data pipelines and implement best practices for data governance and security. Document architecture processes and workflows for future reference and compliance. Must-Have Qualifications: Strong experience as a Data Engineer with a focus on AWS services. Proficiency in AWS Glue for building ETL workflows. Advanced programming skills in Python with experience in data manipulation libraries (e.g. Pandas NumPy). ...

Job Summary We are seeking a highly skilled Technical Architect with deep expertise in Big Data technologies and AWS Cloud services. The ideal candidate will lead the design and implementation of scalable, secure, and high-performance data solutions using Python, PySpark, Spark, Hive, AWS EMR, and AWS Glue. This role requires strong leadership, architectural vision, and hands-on technical proficiency. We are looking for immediate joiner or (Max 15 Day NP) Multiple Location - Indore/Pune/Hyderabad/Noida/Gurgaon/Bangalore Key Responsibilities Architect and design end-to-end Big Data solutions on AWS Cloud Lead development of scalable data pipelines using PySpark, Spark, and Hive Optimize perfo...

We are looking for a Manager Software Engineering to join our IMS Team in Bangalore . This is an amazing opportunity to work on Big Data technologies involved in content ingestion. The team consists of 10-12 engineers and is reporting to the Sr Manager. We have a great skill set in Spark, Java, Scala, Hive, Sql, XSLT, AWS EMR, S3, etc and we would love to speak with you if you have skills in the same. About You experience, education, skills, and accomplishments: Work Experience: Minimum 7 years experience in Big Data projects involved in content ingestion, curation, transformation Technical Skill: Databricks, Spark, Python/Java , Scala, AWS EMR, S3, SQS, Hive, XSLT, Education (bachelors degr...

Role & Responsibilities : - Design, develop, and maintain robust AWS-based data pipelines using Python and PySpark. - Implement efficient ETL processes, ensuring data integrity and optimal performance across AWS services (S3, Glue, EMR, Redshift). - Collaborate with cross-functional teams to integrate data engineering solutions within broader business-critical applications. - Troubleshoot and optimize existing data workflows, ensuring high availability, scalability, and security of cloud solutions. - Exercise best practices in coding, version control, and documentation to maintain a high standard of engineering excellence. Skills & Qualifications : Must-Have : 4+ years of hands-on experience...

EPAM has presence across 40+ countries globally with 55,000 + professionals & numerous delivery centers, Key locations are North America, Eastern Europe, Central Europe, Western Europe, APAC, Mid East & Development Centers in India (Hyderabad, Pune & Bangalore) . Location: Chennai/Coimbatore Work Mode: Hybrid (2-3 days office in a week) Full time Job Description: 5-14 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python/Scala/Java Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert requirements to fe...

As AWS Data Engineer at organization, you will play a crucial role in the design, development, and maintenance of our data infrastructure. Your work will empower data-driven decision-making and contribute to the success of our data-driven initiatives You will design and maintain scalable data pipelines using AWS data analytical resources, enabling efficient data processing and analytics. Key Responsibilities: - Highly experiences in developing ETL pipelines using AWS Glue and EMR with PySpark/Scala. - Utilize AWS services (S3, Glue, Lambda, EMR, Step Functions) for data solutions. - Design scalable data models for analytics and reporting. - Implement data validation, quality, and governance ...

As a hands-on technical lead and senior individual contributor in BI Engineering within Visa's Global Data organization, reporting to the Lead of Data Insights Studio, you will partner with internal analytics teams to deliver high-impact BI solutions across products and markets. Operating from the Global Shared Services hub in Bengaluru, you will leverage Visa's rich datasets to drive actionable insights that create business value. **Key Responsibilities:** - Partner with internal analytics and business teams to translate complex, ambiguous problems into clear, measurable outcomes and actionable data products. - Lead end-to-end BI solution delivery, including requirements refinement, data mo...

Job Description: Certification Preferred: Databricks Certified Data Engineer, Informatica (IDMC) Job Summary We are seeking a skilled and motivated Data Engineer with strong experience in Data Warehousing concepts, ETL/ELT development, and hands-on expertise in Informatica. The ideal candidate will have a solid understanding of data modeling, excellent communication skills, and experience working in Agile environments. Certification in Databricks is a plus. Key Responsibilities Design and develop ETL/ELT processes using Guardian-approved tools such as Informatica, SSIS, and Syncsort. Work with big data technologies including HIVE, HAWQ, Spark SQL, Python, PySpark and AWS EMR. Perform hands-o...

About the role You will be a hands-on technical lead and senior individual contributor in BI Engineering within Visa's Global Data organization, reporting to the Lead, Data Insights Studio. You will operate from our Global Shared Services hub in Bengaluru, partnering with internal analytics teams to deliver high-impact BI solutions across products and markets. You will turn Visa's rich datasets (billions of cards, hundreds of billions of transactions annually) into actionable insights that drive business value. Primary responsibilities Partner with internal analytics and business teams to transform complex, ambiguous problems into clear, measurable outcomes and actionable data products. Lead...