869 Aws Glue Jobs - Page 13

AWS-Specific Skills AWS Storage & Compute Services S3: Deep understanding of bucket policies, lifecycle rules, and data organization Glue: For ETL jobs, crawlers, and data cataloging Athena: Querying data directly from S3 using SQL EMR: For big data processing using Spark/Hadoop Lakehouse Technologies Apache Hudi: Knowledge of transactional data lakes, schema evolution, etc Data Cataloging & Governance AWS Glue Data Catalog Integration with Lake Formation Security & Compliance IAM roles and policies Encryption Data masking and tokenization strategies Mandatory Competencies Cloud - AWS - AWS S3, S3 glacier, AWS EBS Big Data - Big Data - Hadoop Big Data - Big Data - SPARK Database - Database P...

Roles and Responsibilities Design, develop, and maintain large-scale data pipelines using AWS services such as S3, Lambda, Step Functions, etc. Develop ETL processes using PySpark and Redshift to extract insights from NoSQL databases like DynamoDB. Ensure high availability and scalability of the data warehousing infrastructure on AWS. Troubleshoot complex issues related to data processing, storage, and retrieval. Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.

We are seeking an experienced AWS PySpark Developer with 2-8 years of experience to design, build, and optimize our data pipelines and analytics architecture. The ideal candidate will have a strong background in data wrangling and analysis, with a deep understanding of AWS data services. Key Responsibilities: Design, build, and optimize robust data pipelines and data architecture on the AWS cloud platform. Wrangle, explore, and analyze large datasets to identify trends, answer business questions, and pinpoint areas for improvement. Develop and maintain a next-generation analytics environment, providing a self-service, centralized platform for all data-centric activities. Formulate and implem...

Job Title: AWS Data Engineer Location: Gurugram, India No of openings: Experience Required: Company: PwC India Job Description: PwC India is seeking a talented AWS Data Engineer to join our team in Gurgaon. The ideal candidate will have 2-7 years of experience with a strong focus on AWS services, data engineering, and analytics. This role offers an exciting opportunity to work on cutting-edge projects for global clients while leveraging your expertise in cloud technologies and data management. Key Responsibilities: 1. AWS Service Implementation: Design, develop, and maintain data solutions using AWS services, with a particular focus on S3, Athena, Glue, EMR, and Redshift. Implement and optim...

At PwC, the focus in data and analytics engineering is on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. You play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will concentrate on designing and building data infrastructure and systems to enable efficient data processing and analysis. Your responsibilities include developing and implementing data pipelines, data integration, and data transformation solutions. As an AWS Architect / Manager at PwC - AC, you will interact with Offshore Manager/Onsite Business Analyst to under...

The ideal candidate for the Data QA position should have 6 to 10 years of experience in ETL+ DWH Testing and Cloud Testing with expertise in AWS (S3 & Glue) and Azure, along with proficiency in SQL Server, including writing SQL queries. Additionally, familiarity with Snowflake, ADF, and API Testing is essential. Key technical skills required for this role include a strong understanding of Boomi or similar integration tools like MuleSoft, Informatica, Talend, or Workato, experience with Data Bricks, in-depth knowledge of ETL Testing, proficiency in writing SQL Queries, and good technical documentation skills. The candidate should also possess strong communication skills to effectively interac...

As an AWS Developer at PwC's Advisory Acceleration Center, you will collaborate with the Offshore Manager and Onsite Business Analyst to comprehend requirements and take charge of implementing Cloud data engineering solutions on AWS, such as Enterprise Data Lake and Data hub. With a focus on architecting and delivering scalable cloud-based enterprise data solutions, you will bring your expertise in end-to-end implementation of Cloud data engineering solutions using tools like Snowflake utilities, SnowSQL, SnowPipe, ETL data Pipelines, and Big Data model techniques using Python/Java. Your responsibilities will include loading disparate data sets, translating complex requirements into detailed...

As a Snowflake Data Engineer with 3-5 years of experience, you will be responsible for designing, developing, and optimizing cloud-based data warehousing solutions. This is an exciting opportunity to work on a flagship data initiative for a premier Big 4 consulting client, offering ample scope for technical innovation, learning, and career growth. Your key responsibilities will include: - Designing and developing high-performance data pipelines in Snowflake for data ingestion, transformation, and storage. You will focus on external tables, semi-structured data handling, and transformation logic. - Optimizing Snowflake workloads to ensure optimal query execution and cost-effective utilization...

As an AWS Developer at PwC's Acceleration Center in Bangalore, you will be responsible for the end-to-end implementation of Cloud data engineering solutions like Enterprise Data Lake and Data hub in AWS. You will collaborate with Offshore Manager/Onsite Business Analyst to understand requirements and architect scalable, distributed, cloud-based enterprise data solutions. Your role will involve hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, ETL data Pipelines, and Big Data model techniques using Python/Java. You must have a deep understanding of relational and NoSQL data stores, methods, and approaches such as star and snowflake dimensional modeling. Strong expertise in AWS ...

As an AWS Data Engineer with 1.6 to 6 years of experience, you will be responsible for designing, developing, and maintaining scalable data pipelines and ETL workflows on AWS. Your expertise in Snowflake, Python, and SQL will be crucial as you process, transform, and analyze data. You will work towards optimizing existing data architectures to ensure high performance and cost efficiency while integrating data from multiple sources into a unified data warehouse. Collaboration with data analysts, data scientists, and business stakeholders is essential to meet data requirements effectively. You will implement data governance, quality checks, and best practices to enhance data engineering proces...

As a software developer, you will be working in a constantly evolving environment driven by technological advances and the strategic direction of the organization you are employed by. Your primary responsibilities will include creating, maintaining, auditing, and enhancing systems to meet specific needs, often based on recommendations from systems analysts or architects. You will be tasked with testing both hardware and software systems to identify and resolve system faults. Additionally, you will be involved in writing diagnostic programs and designing and developing code for operating systems and software to ensure optimal efficiency. In situations where necessary, you will also provide re...

You have 5+ years of experience in designing and building data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. You possess hands-on expertise with streaming and messaging systems like Apache Kafka, Confluent Cloud, RabbitMQ, or Azure Event Hub. Your experience includes creating producers, consumers, and topics, and integrating them into downstream processing. You have a deep understanding of relational databases and Change Data Capture (CDC). You are proficient in SQL Server, Oracle, or other RDBMSs, and have experience capturing change events using tools like Debezium or native CDC tools, transforming them for downstream consumption. Your proficiency extends to ...

As a Business Intelligence Analyst in our team, you will collaborate with product managers, engineers, and business stakeholders to establish key performance indicators (KPIs) and success metrics for Creator Success. Your role involves creating detailed dashboards and self-service analytics tools utilizing platforms like QuickSight, Tableau, or similar Business Intelligence (BI) tools. You will conduct in-depth analysis on customer behavior, content performance, and livestream engagement patterns. Developing and maintaining robust ETL/ELT pipelines to handle large volumes of streaming and batch data from the Creator Success platform is a key responsibility. Additionally, you will be involved...

We are seeking a highly skilled Senior Data Scientist to join our India-based team in a remote capacity. Your primary responsibility will involve building and deploying advanced predictive models to influence key business decisions. To excel in this role, you should possess a strong background in machine learning, data engineering, and cloud environments, with a particular emphasis on AWS. Your main tasks will include collaborating with cross-functional teams to design, develop, and deploy cutting-edge ML models using tools like SageMaker, Bedrock, PyTorch, TensorFlow, Jupyter Notebooks, and AWS Glue. This position offers an excellent opportunity to work on impactful AI/ML solutions within a...

Our client is a trusted global innovator of IT and business services, helping clients transform through consulting, industry solutions, business process services, digital & IT modernization, and managed services. They enable clients and society to move confidently into the digital future, committed to long-term success by combining global reach with local client attention in over 50 countries worldwide. The position available is for Container Security located in Bangalore, requiring a minimum of 8 years of experience. It is a Contract to Hire job with an immediate joiner's notice period. The Security Technology Engineering team collaborates with various Business Units and stakeholders to man...

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Our Technology team builds innovative digital solutions rapidly and at scale to deliver the next generation of Financial and Non-Financial services across the globe. The Position is a senior technical, hands-on delivery role, requiring the knowledge of data engineering, cloud infrastructure and platform engineering, platform operations, and p...

As a Senior AWS Data Engineer Cloud Data Platform at Teamware Solutions, a division of Quantum Leap Consulting Pvt. Ltd, located in Bangalore, you will be responsible for end-to-end implementation of Cloud data engineering solutions like Enterprise Data lake and Data hub in AWS. Working onsite in an office environment for 5 days a week, you will collaborate with the Offshore Manager and Onsite Business Analyst to understand the requirements and deliver scalable, distributed, cloud-based enterprise data solutions. You should have a strong background in AWS cloud technology, with 4-8 years of hands-on experience. Proficiency in architecting and delivering highly scalable solutions is a must, a...

At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative, and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow, informed and validated by science and data, superpowered by creativity and design, all underpinned by technology created with purpose. Your role involves having IT experience with a minimum of 5+ years in creating data warehouses, data lakes, ETL/ELT, data pipelines on cloud. You should have experience in data pipeline implementation with cloud providers such as AWS, Azure, ...

As a Lead Data Engineer at Rearc, you will play a crucial role in establishing and maintaining technical excellence within our data engineering team. Your extensive experience in data architecture, ETL processes, and data modeling will be key in optimizing data workflows for efficiency, scalability, and reliability. Collaborating closely with cross-functional teams, you will design and implement robust data solutions that align with business objectives and adhere to best practices in data management. Building strong partnerships with technical teams and stakeholders is essential as you drive data-driven initiatives and ensure their successful implementation. With over 10 years of experience ...

As a software development partner for a very large enterprise financial products and services firm listed in NYSE, you were involved in re-architecting and migrating 30-year legacy systems to cloud-based microservices on AWS. These systems are crucial for their trading, journal, and bookkeeping platforms. Your contributions included developing an event-driven ETL pipeline using AWS services to efficiently transfer end-of-day securities data from the mainframe to a relational database. Additionally, you built highly resilient and available containerized microservices to enhance the overall system performance. Automation played a key role in your work, as you automated build and deployment pro...

You will be working as a software development partner for a very large enterprise financial products and services firm listed in NYSE. Your main task will involve re-architecting and migrating 30-year legacy systems to cloud-based microservices on AWS. These systems are crucial for their trading, journal, and bookkeeping platforms. Your contributions at BeeHyv will include developing an event-driven ETL pipeline using AWS services to transfer end-of-day securities data from the mainframe to a relational database. You will also be responsible for building highly resilient and available containerized microservices. Automation of build and deployment processes using Jenkins and Terraform for va...

As the Engineering Team Lead, you will play a crucial role in guiding a cross-functional team to deliver high-quality technical solutions for our strategic data products. Leading by example, you will be involved in designing solutions, writing code, mentoring engineers, and collaborating with architects and stakeholders to drive the team forward. In this hands-on leadership position, you will be actively engaged in coding, shaping architecture, refining team processes, and fostering the technical development of your colleagues. Working closely with the Data and Analytics leadership team, including the VP Data and Analytics, architect, data operations lead, and product manager, you will trans...

We are seeking a skilled and experienced Data Engineer with a minimum of 5 years of expertise in data engineering and data migration projects. The ideal candidate will have a strong proficiency in SQL, Python, data modeling, data warehousing, and ETL pipeline development. It is essential to have hands-on experience with big data tools such as Hadoop and Spark, as well as familiarity with various AWS services including Redshift, S3, Glue, EMR, and Lambda. This position offers a fantastic opportunity to contribute to large-scale data solutions that drive data-informed decision-making and operational efficiency. As a Data Engineer, your responsibilities will include designing, building, and mai...

As a Senior Data Engineer at our organization, you will play a crucial role in migrating and constructing large-scale data pipelines on Databricks utilizing Azure or AWS platforms. Your primary responsibilities will revolve around the implementation of both batch and streaming pipelines, the application of the bronze-silver-gold data lakehouse model, and the assurance of scalable and dependable data solutions. With over 6 years of hands-on experience in data engineering, including a minimum of 2 years focusing on Databricks within Azure or AWS environments, you will be proficient in building and enhancing Spark pipelines for both batch and streaming data processing. Your strong expertise in ...

As a Data Engineer with 6-9 years of experience, your role will be based in Pune. You should have a minimum of 5 years of experience as a Data Engineer, along with hands-on expertise in Star/Snowflake schema design, data modeling, data pipelining, and MLOps. Your proficiency in Data Warehouse technologies like Snowflake, AWS Redshift, and AWS data pipelines (Lambda, AWS Glue, Step Functions, etc.) will be crucial. Strong skills in SQL and at least one major programming language (Python/Java) are required. Additionally, you should be experienced with Data Analysis Tools such as Looker or Tableau, and have familiarity with Pandas, Numpy, Scikit-learn, and Jupyter notebooks. Knowledge of Git, G...