869 Aws Glue Jobs - Page 10

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

We are looking for data engineers who have the right attitude, aptitude, skills, empathy, compassion, and hunger for learning. Build products in the data analytics space. A passion for shipping high-quality data products, interest in the data products space; curiosity about the bigger picture of building a company, product development and its people. Roles and Responsibilities Develop and manage robust ETL pipelines using Apache Spark (Scala) Understand park concepts, performance optimization techniques and governance tools Develop a highly scalable, reliable, and high-performance data processing pipeline to extract, transform and load data from various systems to the Enterprise Data Warehou...

Database Development & Management Strong experience in SQL development and query optimization. Hands-on experience with Snowflake (preferred), Oracle, or any relational database. Understanding of database indexing, partitioning, and performance tuning. . ETL/ELT Development Expertise in ETL/ELT development, preferably using Talend (or any other ETL tool). Strong proficiency in Advanced SQL scripting for data transformation and processing. Ability to design, develop, and optimize data pipelines for structured/unstructured data. Experience in error handling, logging, and recovery mechanisms for ETL processes. Data Warehousing & Modeling Understanding of Data Warehousing concepts, including Sta...

The healthcare industry presents a significant opportunity for software development, and Health Catalyst stands out as a leading company in this sector. At Health Catalyst, we are dedicated to addressing national healthcare challenges and improving the lives of millions of individuals, including your loved ones. As a rapidly expanding organization, we value individuals who are intelligent, hardworking, and humble. Our product teams are focused on developing innovative tools to support our mission of enhancing healthcare performance, reducing costs, and improving quality. We are currently looking for a Principal Snowflake Data Engineer and Data Engineering Lead with a minimum of 8-10 years of...

Job Title :AWS Data Engineer Location :Bangalore Notice Period : Immediate to 60 Days Preferred. Job Description: We are seeking skilled and dynamic Cloud Data Engineers specializing in AWS, Databricks. The ideal candidate will have a strong background in data engineering, with a focus on data ingestion, transformation, and warehousing. They should also possess excellent knowledge of PySpark or Spark, and a proven ability to optimize performance in Spark job executions. Key Responsibilities: - Design, build, and maintain scalable data pipelines for a variety of cloud platforms including AWS. - Implement data ingestion and transformation processes to facilitate efficient data warehousing. - U...

Roles and Responsibilities : Design, develop, test and maintain ETL processes using AWS Glue to extract data from various sources. Collaborate with cross-functional teams to identify business requirements and design scalable data pipelines. Develop automated testing frameworks for ETL processes to ensure high-quality deliverables. Troubleshoot issues related to ETL pipeline failures and optimize performance. Job Requirements : 8-15 years of experience in developing ETL solutions using Python or similar technologies. Strong understanding of SQL concepts and ability to write complex queries. Experience with CI/CD tools such as Jenkins or equivalent for continuous integration and deployment. Pr...

As a Project Management Professional at YASH Technologies, you will be part of a team dedicated to reimagining operating models, enhancing competitiveness, optimizing costs, fostering exceptional stakeholder experiences, and driving business transformation. With a focus on bringing real positive changes in an increasingly virtual world, you will work with cutting-edge technologies and collaborate with cross-functional teams to lead end-to-end DWH projects. Your technical skills will be put to the test as you handle ETL Migration and Visualization projects utilizing technologies like AWS Glue, Redshift, Power BI, Tableau, Azure ADF, and Data bricks. Leading technical design and architecture d...

You are looking for an experienced and skilled Technical Leader to join our AWS Data Engineering practice as an AWS Data Architect. In this role, you will be responsible for driving the Data Engineering strategy, architecting scalable data solutions, and leading the implementation of data projects on AWS. Your deep understanding of AWS services and data engineering best practices will be crucial for success. Your main responsibilities will include establishing and enhancing the company's Data Engineering Services Practice, working closely with senior stakeholders to understand business needs, and delivering technical solutions. This role is ideal for a technically proficient individual who t...

Overview Annalect is currently seeking a data engineering lead to join our technology team. In this role you will be building data pipelines and developing data set processes. We are looking for people who have a shared passion for technology, design & development, data, and fusing these disciplines together to build cool things. In this role, you will lead teams working on one or more software and data products in the Annalect Engineering Team. You will participate in technical architecture, design and development of software products as well as research and evaluation of new technical solutions. You will also help us drive the vision forward for data engineering projects by helping us exte...

Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Responsibilities: Design and implement the data modeling, data ingestion and data processing for various datasets Design, develop and maintain ETL Framework for various new data source Develop data ingestion using AWS Glue/ EMR, data pipeline using PySpark, Python and Databricks. Build orchestration workflow using Airflow & databricks Job workflow Develop and execute adhoc data ingestion to support business analytics. Proactively interact with vendors for any questions and...

Design, implement, and maintain data pipelines for processing large datasets, ensuring data availability, quality, and efficiency for machine learning model training and inference. Collaborate with data scientists to streamline the deployment of machine learning models, ensuring scalability, performance, and reliability in production environments. Develop and optimize ETL (Extract, Transform, Load) processes, ensuring data flow from various sources into structured data storage systems. Automate ML workflows using ML Ops tools and frameworks (e.g., Kubeflow, MLflow, TensorFlow Extended (TFX)). Ensure effective model monitoring, versioning, and logging to track performance and metrics in a pro...

As a Big Data Engineer, you will develop, maintain, evaluate, and test big data solutions. You will be involved in data engineering activities like creating pipelines/workflows for Source to Target and implementing solutions that tackle the clients needs.Your primary responsibilities include: Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements. Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization. Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too Required education Bachelor...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

As Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in the development of data solutions using Spark Framework with Python or Scala on Hadoop and AWS Cloud Data Platform Responsibilities: Experienced in building data pipelines to Ingest, process, and transform data from files, streams and databases. Process the data with Spark, Python, PySpark, Scala, and Hive, Hbase or other NoSQL databases on Cloud Data Platforms (AWS) or HDFS Experienced in develop efficient software code for multiple use cases leveraging Spark Framework / using Python or Scala and Big Data technologies for various use cases built on the platform Experience in developin...

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : AWS Glue Good to have skills : NAMinimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications function seamlessly within the existing infrastructure. You will also engage in troubleshoo...

About the Role Responsibilities Design and implement scalable and secure cloud integration solutions using AWS services such as AWS Lambda, API Gateway, AWS Glue, and Amazon S3, alongside Confluent Kafka for data streaming and Databricks for data processing and analytics. Develop and maintain integration architectures, data flows, and APIs to connect various cloud and on-premises systems, leveraging Confluent Kafka for real-time data processing or other integration tools for batch processing. Design and implement scalable data solutions on AWS, ensuring optimal performance, reliability and cost effectiveness while adhering to best practices in cloud data management. Develop comprehensive clo...

Support enhancements to the MDM and Performance platform Track System Performance Troubleshoot issues Resolve production issues Required Candidate profile 5+ years in Python and advanced SQL including profiling, refactoring Experience with REST API and Hands on AWS Glue EMR etc Experience with Markit EDM or Semarchy or MDM will be plus

Inviting applications for the role of Lead Consultant- Snowflake Data Engineer( Snowflake+Python+Cloud)! In this role, the Snowflake Data Engineer is responsible for providing technical direction and lead a group of one or more developer to address a goal. Job Description: Experience in IT industry Working experience with building productionized data ingestion and processing data pipelines in Snowflake Strong understanding on Snowflake Architecture Fully well-versed with data warehousing concepts. Expertise and excellent understanding of Snowflake features and integration of Snowflake with other data processing. Able to create the data pipeline for ETL/ELT Excellent presentation and communic...

Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose the relentless pursuit of a world that works better for people – we serve and transform leading enterprises, including the Fortune Global 500, with our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI. Inviting applications for the role of Lead Consultant-Data Engineer, AWS+Python, Spark, Kafka for ETL! Responsibilities Develop, deploy, and manage ETL...

5-10 years of experience in database development or a related field. Proven experience with database design, development, and management. Experience working with large-scale databases and complex data environments Experience with data modelling and database design. Knowledge of database performance tuning and optimization. Architect, Develop and maintain tables, views, procedures, functions and packages in Database MUST HAVE Performing complex relational databases queries using SQL (AWS RDS for PostgreSQL) and Oracle PLSQL MUST HAVE Familiarity with ETL processes and tools (AWS Batch, AWS Glue etc) MUST HAVE Familiarity with CI/CD Pipelines, Jenkins Deployment, Git Repository MUST HAVE Perfo...

Role Description: As a Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining secur...

Role Description: As a Senior Cloud Data Platform (AWS) Specialist at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that meet those requirements. Roles & Responsibilities: Designing, developing and deploying cloud-based data platforms using Amazon Web Services (AWS) Integrating and processing large amounts of structured and unstructured data from various sources Implementing and optimizing ETL processes and data pipelines Developing and maintaining secu...

Role & Responsibilities Excellent problem-solving skills with the ability to analyze business problems systematically and deliver effective right-sized solutions in a timely manner. Proven analysis and design/development skills on AWS platforms. Develop, Test, and Implement the changes following the Software Development Life Cycle methodologies and quality concepts. Technical Skills Strong knowledge on AWS Glue/Python/PySpark. Understanding of AWS data/automation services Step functions, Lambda, SNS, SQS. Hands-on experience in ETL projects. Good knowledge of data warehousing concepts. Good analytical and problem-solving skills. Experience in GIT code versioning tools. Strong knowledge of re...