1847 Aws Glue Jobs - Page 47

UnifyCX is looking for a highly experienced Senior Data Scientist to join the team. As a Senior Data Scientist, you will be responsible for developing and implementing complex statistical models and machine learning algorithms to support data-driven decision-making processes. This role involves leading data analysis projects, collaborating with cross-functional teams, and identifying key business insights, trends, and opportunities. We believe that data transformation and integration play a crucial role in generating business insights. Therefore, we are seeking a Senior Data Scientist with experience in automating data pipelines to ensure proper data transformation and integration for analyt...

As a software development partner for a large enterprise financial products and services firm listed on NYSE, you were involved in re-architecting and migrating 30-year legacy systems to cloud-based microservices on AWS. These systems are crucial for their trading, journal, and bookkeeping platforms. Your contributions included developing an event-driven ETL pipeline using AWS services to transfer end-of-day securities data from the mainframe to a relational database. You also built highly resilient and available containerized microservices to ensure smooth operations. In addition, you automated build and deployment processes using Jenkins and Terraform across various environments such as Sa...

As a Lead Data Engineer with over 7 years of experience, you will be responsible for designing, developing, and maintaining data pipelines, ETL processes, and data warehouses. You will be based in Hyderabad and should be able to join immediately. Your primary skills should include proficiency in SQL, Python, PySpark, AWS, Airflow, Snowflake, and DBT. Additionally, you should have a Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or a related field. In this role, you will need a minimum of 4+ years of hands-on experience in data engineering, ETL, and data warehouse development. You should have expertise in ETL tools such as Informatica Power Center or IDMC...

Job Title: AWS Data Engineer Location: Gurgaon, (Onsite) Interview Process: Face to Face About the Role: We are looking for a skilled AWS Data Engineer with hands-on experience in AWS Glue and other AWS analytics services. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and ETL processes that support data ingestion, transformation, and loading for advanced analytics and business intelligence. Key Responsibilities: Design and develop ETL pipelines using AWS Glue , PySpark , and other AWS services (Lambda, Step Functions, S3, etc.) Build and maintain scalable data lakes and data warehouses using AWS S3 , Athena , Redshift , and Glue Da...

Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose the relentless pursuit of a world that works better for people – we serve and transform leading enterprises, including the Fortune Global 500, with our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI. Inviting applications for the role of Lead Consultant-Data Engineer, AWS+Python, Spark, Kafka for ETL! Responsibilities Develop, deploy, and manage ETL...

My linkedin linkedin.com/in/yashsharma1608. contract period - 6-12month payroll will be - ASV consulting , my company client - will disclose after 1 round Job location - Gurgaon - onsite(WFO) budget - upto 1lpa/month , depending on last (relevant hike) Exprnce - 3+ JD is About the Role We are looking for a skilled AWS Data Engineer with strong hands-on experience in AWS Glue and AWS analytics services. The candidate will be responsible for designing, building, and optimizing scalable data pipelines and ETL processes that support advanced analytics and business intelligence requirements. Key Responsibilities Design and develop ETL pipelines using AWS Glue, PySpark, and AWS services (Lambda, S...

Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose the relentless pursuit of a world that works better for people – we serve and transform leading enterprises, including the Fortune Global 500, with our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI. Inviting applications for the role of Lead Consultant-Data Engineer, AWS+Python, Spark, Kafka for ETL! Responsibilities Develop, deploy, and manage ETL...

Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose the relentless pursuit of a world that works better for people – we serve and transform leading enterprises, including the Fortune Global 500, with our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI. Inviting applications for the role of Lead Consultant-Data Engineer, AWS+Python, Spark, Kafka for ETL! Responsibilities Develop, deploy, and manage ETL...

Position Overview We are seeking a highly skilled Senior Data Engineer to join our data platform team, responsible for designing and implementing scalable, cloud-native data solutions. This role focuses on building modern data infrastructure using AWS and Azure services to support our growing analytics and machine learning initiatives. Key Responsibilities Business Translation : Collaborate with business stakeholders to understand requirements and translate them into scalable data models, architectures, and robust end-to-end data pipelines ETL/ELT Implementation : Design, develop, and maintain high-performance batch and real-time data pipelines using modern frameworks like Apache Spark, Delt...

Roles and responsibilities Design AWS architectures based on business requirements. Create architectural diagrams and documentation. Present cloud solutions to stakeholders. Skills and Qualifications: Design, develop, and maintain scalable ETL/ELT pipelines using AWS services like Glue, Lambda, and Step Functions. Work with batch and real-time data processing using AWS Glue, Kinesis, Kafka, or Apache Spark. Optimize data pipelines for performance, scalability, and cost-effectiveness. Identify bottlenecks and optimize query performance on Redshift, Athena, and Glue. Strong knowledge of AWS services: EC2, S3, RDS, Lambda, IAM, VPC, CloudFormation, CloudWatch, etc. Experience with serverless ar...

Required Skills and Experience 8 to 10 years of experience in data engineering or backend development in cloud-based environments. Strong Python development experience, with emphasis on Pyspark for data transformation and parsing. Proficiency with PySpark for distributed data processing. Deep understanding of SQL, with hands-on experience in MySQL, PostgreSQL, or Snowflake. Strong experience with AWS services, especially: AWS Lambda and API Gateway for API development AWS Glue for orchestration and transformation Amazon S3, RDS, and ECS for data storage and compute Experience developing and managing RESTful APIs. Experience implementing and managing CI/CD pipelines. Hands-on experience with ...

As a PySpark Data Engineer at Viraaj HR Solutions, you will be an integral part of our dynamic team in India. Your primary responsibility will be to design, develop, and maintain robust data pipelines using PySpark. You will play a crucial role in implementing ETL processes for data extraction, transformation, and loading, while ensuring data quality from structured and unstructured sources. Your expertise in optimizing data processing workflows will be essential for enhancing performance and efficiency. Collaboration with data scientists and analysts is key to understanding data requirements and creating data warehousing solutions for end-user access. You will integrate data from various so...

As a Data Engineering Lead, you will be responsible for developing and implementing data engineering projects, including enterprise data hubs or Big data platforms. Your role will involve defining reference data architecture, leveraging cloud-native data platforms in AWS or Microsoft stack, and staying updated on the latest data trends like data fabric and data mesh. You will play a key role in leading the Center of Excellence (COE) and influencing client revenues through innovative data and analytics solutions. Your responsibilities will include guiding a team of data engineers, overseeing the design and deployment of data solutions, and strategizing new data services and offerings. Collabo...

We are searching for a highly skilled and seasoned Senior ETL & Data Streaming Engineer with over 10 years of experience to take on a crucial role in the design, development, and maintenance of our robust data pipelines. The ideal candidate will possess in-depth expertise in batch ETL processes as well as real-time data streaming technologies, along with extensive hands-on experience with AWS data services. A proven track record of working with Data Lake architectures and traditional Data Warehousing environments is a must. Your responsibilities will include designing, developing, and implementing highly scalable, fault-tolerant, and performant ETL processes using leading ETL tools to extrac...

You will be responsible for designing, building, and maintaining scalable and efficient data pipelines to facilitate the movement of data between cloud-native databases (e.g., Snowflake) and SaaS providers using AWS Glue and Python. Your role will involve implementing and managing ETL/ELT processes to ensure seamless data integration and transformation while adhering to information security and compliance with data governance standards. Additionally, you will be tasked with maintaining and enhancing data environments, including data lakes, warehouses, and distributed processing systems. It is crucial to utilize version control systems (e.g., GitHub) effectively to manage code and collaborate...

***We are looking for immediate joiners for our Bangalore Location*** Role & responsibilities Hands on experience in Admin activities on Snowflake, DBT, Matillion & AWS-Glue , with exposure in Development & Support environments. Experience in real time CDC tools - IBM Infosphere and Qlik Replicate. Willingness to quickly learn/adapt other ETL/BI cloud tools if & as needed. Possess strong Snowflake SQL & Database skills. Strong written and oral communication skills. Should be able to understand business users, their issues and ability to take them to closure. Should possess the ability to learn & adapt, understand business requirements, data models and mapping documents. Independently design ...

As a Data Architect, you will be responsible for leading Data related projects in the field of Reporting and Analytics. With over 10 years of relevant work experience, you will design, build, and maintain scalable data lake and data warehouse in the cloud, particularly on Google Cloud Platform (GCP). Your expertise will be crucial in gathering business requirements, analyzing business needs, and defining the BI/DW architecture to deliver technical solutions for complex business and technical requirements. You will create solution prototypes, participate in technology selection, and perform POCs and technical presentations. In this role, you will architect, develop, and test scalable data war...

As an Associate/Architect - Data (AbInitio) at Quantiphi, you will play a crucial role in migrating ETL jobs from existing Ab Initio to AWS Cloud. Your responsibilities will include designing and implementing a migration strategy, interpreting complex graphs and jobs in Ab Initio, and assisting the Data Engineering team in understanding business logic and data sources. You will be expected to create high-level block diagrams and flows of jobs, assist in validating output data generated through Spark code, and serve as a technical point of contact between clients and the offshore technical team. Strong communication skills to convey technical roadmaps, challenges, and mitigation strategies ar...

You have over 10 years of hands-on experience within a data engineering organization, with a specific focus on Informatica IICS (Informatica Intelligent Cloud Services) for at least 5 years. Additionally, you have a minimum of 5 years of experience working with Cloud Data Warehouse Platforms such as Snowflake. Your expertise includes utilizing tools like Spark, Athena, AWS Glue, Python/PySpark, etc. Your role involves developing architectures to transition from legacy to modern data platforms. Previous exposure to ETL/ELT tools, particularly Informatica Power Center in Data Warehouse environments, is essential. You are well-versed in agile software development processes and are familiar with...

Design, build, and maintain data pipelines on the AWS platform. Work with AWS services like S3, Glue, EMR, and Redshift. Process and analyze large datasets to support business insights. Ensure data quality, integrity, and security in the data lake. Location - Pan India.

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

Skills AWS Glue or Databricks Python Pyspark

Role & responsibilities: AWS Glue Python Pyspark

About Markovate: At Markovate, you don't just follow trends, we drive them. We transform businesses through innovative AI and digital solutions that turn vision into reality. Our team harnesses breakthrough technologies to craft bespoke strategies that align seamlessly with our clients" ambitions. From AI Consulting And Gen AI Development To Pioneering AI Agents And Agentic AI, We Empower Our Partners To Lead Their Industries With Forward-thinking Precision And Unmatched Overview. We are seeking a highly experienced and innovative Senior Data Engineer with a strong background in hybrid cloud data integration, pipeline orchestration, and AI-driven data modelling. Requirements: - 9+ years of e...

Role & responsibilities Data Ingestion Framework: Design & Development: Architect, develop, and maintain an end-to-end data ingestion framework that efficiently extracts, transforms, and loads data from diverse sources. Framework Optimization: Use AWS services such as AWS Glue, Lambda, EMR, ECS , EC2 and Step Functions to build highly scalable, resilient, and automated data pipelines. Data Quality & Validation: Validation Processes: Develop and implement automated data quality checks, validation routines, and error-handling mechanisms to ensure the accuracy and integrity of incoming data. Monitoring & Reporting: Establish comprehensive monitoring, logging, and alerting systems to proactively...