1333 Aws Glue Jobs - Page 32

You have 5+ years of experience in designing and building data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. You possess hands-on expertise with streaming and messaging systems like Apache Kafka, Confluent Cloud, RabbitMQ, or Azure Event Hub. Your experience includes creating producers, consumers, and topics, and integrating them into downstream processing. You have a deep understanding of relational databases and Change Data Capture (CDC). You are proficient in SQL Server, Oracle, or other RDBMSs, and have experience capturing change events using tools like Debezium or native CDC tools, transforming them for downstream consumption. Your proficiency extends to ...

As a Business Intelligence Analyst in our team, you will collaborate with product managers, engineers, and business stakeholders to establish key performance indicators (KPIs) and success metrics for Creator Success. Your role involves creating detailed dashboards and self-service analytics tools utilizing platforms like QuickSight, Tableau, or similar Business Intelligence (BI) tools. You will conduct in-depth analysis on customer behavior, content performance, and livestream engagement patterns. Developing and maintaining robust ETL/ELT pipelines to handle large volumes of streaming and batch data from the Creator Success platform is a key responsibility. Additionally, you will be involved...

We are seeking a highly skilled Senior Data Scientist to join our India-based team in a remote capacity. Your primary responsibility will involve building and deploying advanced predictive models to influence key business decisions. To excel in this role, you should possess a strong background in machine learning, data engineering, and cloud environments, with a particular emphasis on AWS. Your main tasks will include collaborating with cross-functional teams to design, develop, and deploy cutting-edge ML models using tools like SageMaker, Bedrock, PyTorch, TensorFlow, Jupyter Notebooks, and AWS Glue. This position offers an excellent opportunity to work on impactful AI/ML solutions within a...

Our client is a trusted global innovator of IT and business services, helping clients transform through consulting, industry solutions, business process services, digital & IT modernization, and managed services. They enable clients and society to move confidently into the digital future, committed to long-term success by combining global reach with local client attention in over 50 countries worldwide. The position available is for Container Security located in Bangalore, requiring a minimum of 8 years of experience. It is a Contract to Hire job with an immediate joiner's notice period. The Security Technology Engineering team collaborates with various Business Units and stakeholders to man...

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Our Technology team builds innovative digital solutions rapidly and at scale to deliver the next generation of Financial and Non-Financial services across the globe. The Position is a senior technical, hands-on delivery role, requiring the knowledge of data engineering, cloud infrastructure and platform engineering, platform operations, and p...

As a Senior AWS Data Engineer Cloud Data Platform at Teamware Solutions, a division of Quantum Leap Consulting Pvt. Ltd, located in Bangalore, you will be responsible for end-to-end implementation of Cloud data engineering solutions like Enterprise Data lake and Data hub in AWS. Working onsite in an office environment for 5 days a week, you will collaborate with the Offshore Manager and Onsite Business Analyst to understand the requirements and deliver scalable, distributed, cloud-based enterprise data solutions. You should have a strong background in AWS cloud technology, with 4-8 years of hands-on experience. Proficiency in architecting and delivering highly scalable solutions is a must, a...

At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative, and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow, informed and validated by science and data, superpowered by creativity and design, all underpinned by technology created with purpose. Your role involves having IT experience with a minimum of 5+ years in creating data warehouses, data lakes, ETL/ELT, data pipelines on cloud. You should have experience in data pipeline implementation with cloud providers such as AWS, Azure, ...

As a Lead Data Engineer at Rearc, you will play a crucial role in establishing and maintaining technical excellence within our data engineering team. Your extensive experience in data architecture, ETL processes, and data modeling will be key in optimizing data workflows for efficiency, scalability, and reliability. Collaborating closely with cross-functional teams, you will design and implement robust data solutions that align with business objectives and adhere to best practices in data management. Building strong partnerships with technical teams and stakeholders is essential as you drive data-driven initiatives and ensure their successful implementation. With over 10 years of experience ...

As a software development partner for a very large enterprise financial products and services firm listed in NYSE, you were involved in re-architecting and migrating 30-year legacy systems to cloud-based microservices on AWS. These systems are crucial for their trading, journal, and bookkeeping platforms. Your contributions included developing an event-driven ETL pipeline using AWS services to efficiently transfer end-of-day securities data from the mainframe to a relational database. Additionally, you built highly resilient and available containerized microservices to enhance the overall system performance. Automation played a key role in your work, as you automated build and deployment pro...

You will be working as a software development partner for a very large enterprise financial products and services firm listed in NYSE. Your main task will involve re-architecting and migrating 30-year legacy systems to cloud-based microservices on AWS. These systems are crucial for their trading, journal, and bookkeeping platforms. Your contributions at BeeHyv will include developing an event-driven ETL pipeline using AWS services to transfer end-of-day securities data from the mainframe to a relational database. You will also be responsible for building highly resilient and available containerized microservices. Automation of build and deployment processes using Jenkins and Terraform for va...

As the Engineering Team Lead, you will play a crucial role in guiding a cross-functional team to deliver high-quality technical solutions for our strategic data products. Leading by example, you will be involved in designing solutions, writing code, mentoring engineers, and collaborating with architects and stakeholders to drive the team forward. In this hands-on leadership position, you will be actively engaged in coding, shaping architecture, refining team processes, and fostering the technical development of your colleagues. Working closely with the Data and Analytics leadership team, including the VP Data and Analytics, architect, data operations lead, and product manager, you will trans...

We are seeking a skilled and experienced Data Engineer with a minimum of 5 years of expertise in data engineering and data migration projects. The ideal candidate will have a strong proficiency in SQL, Python, data modeling, data warehousing, and ETL pipeline development. It is essential to have hands-on experience with big data tools such as Hadoop and Spark, as well as familiarity with various AWS services including Redshift, S3, Glue, EMR, and Lambda. This position offers a fantastic opportunity to contribute to large-scale data solutions that drive data-informed decision-making and operational efficiency. As a Data Engineer, your responsibilities will include designing, building, and mai...

As a Senior Data Engineer at our organization, you will play a crucial role in migrating and constructing large-scale data pipelines on Databricks utilizing Azure or AWS platforms. Your primary responsibilities will revolve around the implementation of both batch and streaming pipelines, the application of the bronze-silver-gold data lakehouse model, and the assurance of scalable and dependable data solutions. With over 6 years of hands-on experience in data engineering, including a minimum of 2 years focusing on Databricks within Azure or AWS environments, you will be proficient in building and enhancing Spark pipelines for both batch and streaming data processing. Your strong expertise in ...

As a Data Engineer with 6-9 years of experience, your role will be based in Pune. You should have a minimum of 5 years of experience as a Data Engineer, along with hands-on expertise in Star/Snowflake schema design, data modeling, data pipelining, and MLOps. Your proficiency in Data Warehouse technologies like Snowflake, AWS Redshift, and AWS data pipelines (Lambda, AWS Glue, Step Functions, etc.) will be crucial. Strong skills in SQL and at least one major programming language (Python/Java) are required. Additionally, you should be experienced with Data Analysis Tools such as Looker or Tableau, and have familiarity with Pandas, Numpy, Scikit-learn, and Jupyter notebooks. Knowledge of Git, G...

Job Description: We are seeking a Senior Data Engineer with over 5 years of experience in Python, SQL, PySpark, and various AWS services such as Lambda, Glue Jobs, and Athena. This position is located in Hyderabad and necessitates on-site work. As a Senior Data Engineer, you will be tasked with constructing robust data pipelines, creating scalable ETL solutions, and overseeing cloud-based data systems. Key Responsibilities: - Designing, developing, and enhancing large-scale ETL/ELT pipelines utilizing PySpark and AWS Glue. - Deploying serverless workflows using AWS Lambda, Athena, and S3 for data processing and querying purposes. - Crafting efficient and scalable SQL queries to handle data i...

As a Senior/Lead Data Engineer at Srijan Technologies PVT LTD, you will play a crucial role in designing, developing, and maintaining data pipelines for our clients. Your expertise in AWS Glue, Apache Airflow, Kafka, SQL, Python, and DataOps tools will be key in ensuring the efficient and reliable flow of data across the organization. You will collaborate with cross-functional teams to understand data requirements, implement DataOps practices, and optimize data pipeline performance. Monitoring data pipeline performance, troubleshooting data-related issues, and staying up-to-date with the latest industry trends in data engineering and DataOps will also be part of your responsibilities. Key Re...

As an AWS Consultant specializing in Infrastructure, Data & AI, and Databricks, you will play a crucial role in designing, implementing, and optimizing AWS Infrastructure solutions. Your expertise will be utilized to deliver secure and scalable data solutions using various AWS services and platforms. Your responsibilities will also include architecting and implementing ETL/ELT pipelines, data lakes, and distributed compute frameworks. You will be expected to work on automation and infrastructure as code using tools like CloudFormation or Terraform, and manage deployments through AWS CodePipeline, GitHub Actions, or Jenkins. Collaboration with internal teams and clients to gather requirements...

Capgemini Invent is the digital innovation, consulting, and transformation brand of the Capgemini Group. As a global business line, Capgemini Invent combines expertise in strategy, technology, data science, and creative design to assist CxOs in envisioning and constructing what's next for their businesses. In this role, you will be responsible for developing and maintaining scalable data pipelines using AWS services. Your tasks will include optimizing data storage and retrieval processes, ensuring data security and compliance with industry standards, and handling large volumes of data while maintaining accuracy, security, and accessibility. Additionally, you will be involved in developing da...

We are looking for a seasoned Senior Data Architect with extensive knowledge in Databricks and Microsoft Fabric to join our team. In this role, you will be responsible for leading the design and implementation of scalable data solutions for BFSI and HLS clients. As a Senior Data Architect specializing in Databricks and Microsoft Fabric, you will play a crucial role in architecting and implementing secure, high-performance data solutions on the Databricks and Azure Fabric platforms. Your responsibilities will include leading discovery workshops, designing end-to-end data pipelines, optimizing workloads for performance and cost efficiency, and ensuring compliance with data governance, security...

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture, and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Our Technology team builds innovative digital solutions rapidly and at scale to deliver the next generation of Financial and Non-Financial services across the globe. The Position is a senior technical, hands-on delivery role, requiring knowledge of data engineering, cloud infrastructure, platform engineering, platform operations, and product...

Job Description: We are seeking a skilled Data Engineer with 7+ years of experience in data processing, ETL pipelines, and cloud-based data solutions. The ideal candidate will have strong expertise in AWS Glue, Redshift, S3, EMR, and Lambda , with hands-on experience using Python and PySpark for large-scale data transformations. The candidate will be responsible for designing, building, and maintaining scalable data pipelines and systems to support analytics and data-driven decision-making. Additionally, need to have strong expertise in Terraform and Git-based CI/CD pipelines to support infrastructure automation and configuration management. Key Responsibilities: ETL Development & Automation...

Position - AWS Data Engineer Job Description: We are seeking a skilled Data Engineer with 7+ years of experience in data processing, ETL pipelines, and cloud-based data solutions. The ideal candidate will have strong expertise in AWS Glue, Redshift, S3, EMR, and Lambda , with hands-on experience using Python and PySpark for large-scale data transformations. The candidate will be responsible for designing, building, and maintaining scalable data pipelines and systems to support analytics and data-driven decision-making. Additionally, need to have strong expertise in Terraform and Git-based CI/CD pipelines to support infrastructure automation and configuration management. Key Responsibilities:...

As a Lead Data Engineer with 7-12 years of experience, you will be an integral part of our team, contributing significantly to the design, development, and maintenance of our data infrastructure. Your primary responsibilities will revolve around creating and managing robust data architectures, ETL processes, data warehouses, and utilizing big data and cloud technologies to support our business intelligence and analytics needs. You will lead the design and implementation of data architectures that facilitate data warehousing, integration, and analytics platforms. Developing and optimizing ETL pipelines will be a key aspect of your role, ensuring efficient processing of large datasets and impl...

As a Digital Product Engineering company, Nagarro is seeking a talented individual to join our dynamic and non-hierarchical work culture as a Data Engineer. With over 17500 experts across 39 countries, we are scaling in a big way and are looking for someone with 10+ years of total experience to contribute to our team. **Requirements:** - The ideal candidate should possess strong working experience in Data Engineering and Big Data platforms. - Hands-on experience with Python and PySpark is required. - Expertise with AWS Glue, including Crawlers and Data Catalog, is essential. - Experience with Snowflake and a strong understanding of AWS services such as S3, Lambda, Athena, SNS, and Secrets Ma...

As a Senior Lead Engineer specializing in Python and Spark within the AWS environment, you will have the crucial responsibility of designing, building, and maintaining robust, scalable, and efficient ETL pipelines. Your primary focus will be on ensuring alignment with data lakehouse architecture on AWS and optimizing workflows using AWS services such as Glue, Glue Data Catalog, Lambda, and S3. Your expertise will play a key role in implementing data quality and governance frameworks to maintain reliable and consistent data processing across the platform. Collaboration with cross-functional teams to gather requirements, provide technical insights, and deliver high-quality data solutions will ...