1844 Aws Glue Jobs - Page 32

Requirements for the candidate: Data Engineer with a minimum of 7+ years of experience of data engineering experience. The role will require deep knowledge of data engineering techniques to create data pipelines and build data assets. At least 4+ years of Strong hands on programming experience with Pyspark / Python / Boto3 including Python Frameworks, libraries according to python best practices. Strong experience in code optimization using spark SQL and pyspark. Understanding of Code versioning, Git repository, JFrog Artifactory. AWS Architecture knowledge specially on S3, EC2, Lambda, Redshift, CloudFormation etc and able to explain benefits of each Code Refactorization of Legacy Codebase:...

Description Are you a highly skilled data engineer and project leaderDo you think big, enjoy complexity and building solutions that scaleAre you curious to know what you could achieve in a company that pushes the boundaries of modern technologyIf you answered yes and you have a background in FinTech youll love this role and Amazons data obsessed culture, Amazon Devices and Services Fintech is the global team that designs and builds the ficial planning and analysis tools for wide variety of Amazons new and established organizations From Kindle to Ring and even new and exciting companies like Kuiper (our new interstellar satellite play) this team enjoys a wide variety of complex and interestin...

Job Title: AWS Senior Data Engineer Experience Required: Minimum 5+ years Job Summary : We are seeking a skilled Data Engineer with a strong background in data ingestion, processing, and storage. The ideal candidate will have experience working with various data sources and technologies, particularly in a cloud environment. You will be responsible for designing and implementing data pipelines, ensuring data quality, and optimizing data storage solutions. Key Responsibilities: Design, develop, and maintain scalable data pipelines for data ingestion and processing using Python, Spark, and AWS services. Work with on-prem Oracle databases, batch files, and Confluent Kafka for data sourcing. Impl...

Role & responsibilities Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performance. Collaborate with Data Scientists, Analysts, and DevOps teams to deliver reliable data solutions. Monitor and troubleshoot data pipeline failures and latency issues. Preferred candidate profile Strong experience with Snowflake architecture, SQL, and performance tuning. Hands-on ...

As a skilled Data Engineer at BigThinkCode Technologies, your role involves designing, building, and maintaining robust data pipelines and infrastructure to optimize data flow and ensure scalability. Your technical expertise in Python, SQL, ETL/ELT frameworks, and cloud data warehouses, along with strong collaboration skills, will be crucial in partnering with cross-functional teams to enable seamless access to structured and unstructured data across the organization. **Key Responsibilities:** - Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data processing. - Optimize and manage SQL queries for performance and efficiency in handling large-scale data...

Job Summary: We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data pipelines using Snowflake, AWS services, and Apache Spark. You will be responsible for real-time and batch data ingestion, transformation, and orchestration across cloud platforms. Key Responsibilities: Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performa...

Role & responsibilities Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performance. Collaborate with Data Scientists, Analysts, and DevOps teams to deliver reliable data solutions. Monitor and troubleshoot data pipeline failures and latency issues. Required Skills: Strong experience with Snowflake architecture, SQL, and performance tuning. Hands-on expertise i...

Job seekers, We have openings for Snowflake developer, Client:Carelon Global solutions Payroll: Sp soft(C2H) NP:Immediate to 10 days Locarion: Bangalore/Gurgoan Workmode:Hybrid Key Responsibilities: Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performance. Collaborate with Data Scientists, Analysts, and DevOps teams to deliver reliable data solutions. Monito...

Job Summary: We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data pipelines using Snowflake, AWS services, and Apache Spark. You will be responsible for real-time and batch data ingestion, transformation, and orchestration across cloud platforms. Key Responsibilities: Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performa...

Job Summary: We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data pipelines using Snowflake, AWS services, and Apache Spark. You will be responsible for real-time and batch data ingestion, transformation, and orchestration across cloud platforms. Key Responsibilities: Develop and maintain data pipelines using AWS Glue , Lambda , EMR , and Snowflake . Implement real-time ingestion using Snowpipe and Streams for CDC (Change Data Capture). Write efficient PySpark or Scala Spark jobs for large-scale data processing. Automate workflows and orchestrate jobs using AWS Step Functions , Airflow , or similar tools. Optimize Snowflake queries and warehouse performa...

Job Title: AWS Data Engineer Company Name: NCSi Job Description: NCSi is seeking a skilled AWS Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing and building robust data pipelines, developing data integration processes, and optimizing database performance in cloud environments. You will work collaboratively with data scientists, analysts, and other stakeholders to ensure data accuracy, reliability, and accessibility across the organization. Your role will also involve implementing best practices in data governance and security, as well as leveraging AWS services to support data analytics and reporting needs. Key Responsibilities: - Design, develop,...

Role Overview: You will be responsible for building and deploying the infrastructure for ingesting high-volume support data from consumer interactions, devices, and apps. Additionally, you will design and implement processes to turn data into insights, model and mine the data to describe the system's behavior, and predict future actions. Your role will involve enabling data-driven change by creating effective visualizations and reports for internal and external stakeholders. You will also develop and maintain data-related code in an automated CI/CD build/test/deploy environment, generate specific reports for various tenants, and conduct research individually and collaboratively on problem-so...

Role & responsibilities Designs and develops robust data pipelines for ingesting, transforming, and loading (ETL/ELT) data from multiple sources into analytics platforms. Mastery of SQL, Python (Pandas, NumPy, PySpark),Experience with modern ETL/ELT frameworks, expertise in Databricks/ Delta Lake / Glue / Redshift Preferred candidate profile

Develop, test, and maintain ETL pipelines from TMS source systems to canonical layers. Implement data transformation, cleansing, and validation logic. Support QA by fixing data defects and ensuring completeness of load. Mandatory Competencies ETL - ETL - Tester Beh - Communication ETL - ETL - AWS Glue Data Science and Machine Learning - Data Science and Machine Learning - Databricks

Job Description We are seeking a talented and motivated ML Engineer to join our founding team in India. This role is central to building AI solutions on the Axyo platform that solve high-value problems for enterprise clients. As an early member of our India team, you will work closely with the VP of Data Science, Data Engineers, and Product Managers to design, train, and deploy machine learning models that power Axyo AI Apps. Responsibilities Applied Machine Learning & Modeling Design, train, and evaluate predictive, classifi cation, forecasting, and optimization models. Build and experiment with ML pipelines across traditional statistical methods and modern ML/deep learning approaches. Appl...

Job Description We are seeking a talented and motivated ML Engineer to join our founding team in India. This role is central to building AI solutions on the Axyo platform that solve high-value problems for enterprise clients. As an early member of our India team, you will work closely with the VP of Data Science, Data Engineers, and Product Managers to design, train, and deploy machine learning models that power Axyo AI Apps. Responsibilities Applied Machine Learning & Modeling Design, train, and evaluate predictive, classifi cation, forecasting, and optimization models. Build and experiment with ML pipelines across traditional statistical methods and modern ML/deep learning approaches. Appl...

Develop, test, and maintain ETL pipelines from TMS source systems to canonical layers. Implement data transformation, cleansing, and validation logic. Support QA by fixing data defects and ensuring completeness of load. Mandatory Competencies ETL - ETL - Tester Beh - Communication ETL - ETL - AWS Glue Data Science and Machine Learning - Data Science and Machine Learning - Databricks

We are currently looking for an experienced Lead Data Engineer with demonstrated technical experience within pySpark, Phyton, and Java technologies to join the team in India, within our global Technology department. If this sounds interesting to you and you are skilled in Query engines Postgres/SQL/AWS/Java/Kotlin, we would love to speak with you. About You experience, education, skills, and accomplishments: Bachelor/master's degree in computer science, computer engineering, information technology, or relevant field At least 8+ years experience in working in software development At least 8+ years experience in building massively scalable distributed data processing solutions At least 6+ year...

We are currently looking for an experienced Lead Data Engineer with demonstrated technical experience within pySpark, Phyton, and Java technologies to join the team in India, within our global Technology department. If this sounds interesting to you and you are skilled in Query engines Postgres/SQL/AWS/Java/Kotlin, we would love to speak with you. About You experience, education, skills, and accomplishments: At least 6+ years experience in working in software development At least 6+ years experience in building massively scalable distributed data processing solutions At least 6+ years experience of database design & development. Alt least 4 years of experience in Apache Spark, ElasticSearch ...

We are seeking a highly skilled Senior Lead Data Engineer to join our R&D Data Engineering Team. In this role, you will be a key player in shaping the architecture and technical direction of our data platform, ensuring that it meets the evolving needs of the business while adhering to best practices and industry standards. If you're looking for an opportunity to combine your technical skills with strategic thinking and make a real difference, we want to hear from you. About You experience, education, skills, and accomplishments: Bachelors Degree or equivalent At least 7 years of Relevant Experience At least 5+ Years in Software Development: Demonstrated experience in software development, wi...

Key Responsibilities Technical Leadership & Project Execution Provide technical thought leadership, compare different technologies to meet business requirements and cost control drivers. Work with Business and IT groups to design and deliver a data lake platform. Produce & maintain the overall solution design for the entire Data Lake Platform. Execution of data strategy, help in the design and architecture of solutions on the platform Enforce technical best practices for Big Data management and solutions, from software selection to technical architectures and implementation processes. Document and publish best practices, guidelines, and training information. Ensures all functional solutions ...

We are looking for a Manager Software Engineering to join our IMS Team in Bangalore . This is an amazing opportunity to work on Big Data technologies involved in content ingestion. The team consists of 10-12 engineers and is reporting to the Sr Manager. We have a great skill set in Spark, Java, Scala, Hive, Sql, XSLT, AWS EMR, S3, etc and we would love to speak with you if you have skills in the same. About You experience, education, skills, and accomplishments: Work Experience: Minimum 7 years experience in Big Data projects involved in content ingestion, curation, transformation Technical Skill: Databricks, Spark, Python/Java , Scala, AWS EMR, S3, SQS, Hive, XSLT, Education (bachelors degr...

Walk-In Drive Data Engineer (2-5 Years, Azure & AWS) Date & Time: 10th October | 11:00 AM 3:00 PM Venue: The Hive, 24/7, 10th Floor, Prestige Tech Platina 2, Kadubeesanahalli, Marathahalli Outer Ring Road, Bangalore 560087 Location Map: [Insert Link] About the Role We are hiring skilled Data Engineers with 24 years of experience in designing, building, and maintaining scalable data pipelines and architectures using Azure and AWS cloud platforms. Key Responsibilities Design, develop, and optimize scalable ETL/ELT workflows using Azure Data Factory, AWS Glue , or similar tools. Build and maintain data integration solutions between on-premises and cloud systems. Work with data storage and wareh...

We are seeking a Data Engineer with strong expertise in Data Engineeringincluding data modeling, ETL pipeline development, data governance, and Data Build Tool (DBT)—along with proficiency in Python, PySpark, and AWS services such as Glue, Redshift, Lambda, and DynamoDB. Candidates should also have in-depth knowledge of MySQL, including core and advanced concepts. JD: Proficiency with Python and SQL for data processing (spark is a bonus) Working experience with AWS storage and database/data warehouse services - s3, Redshift, RDS, DynamoDB etc. Hands on experience with AWS compute services such as EC2, ECR, Lambda (layers/triggers) etc. Ensure proper logging, error handling, and performance m...

Bachelors Degree preferably in Computer Science, Engineering, Management Information Systems, or equivalent work experience. 4 years of full stack software development with an emphasis on architecting, designing, implementing, testing and delivering highly scalable web and client/server solutions. 4+ years of experience in Data Engineering development Strong experience with: AWS Glue - Python, Big Data – Pyspark, Control-M – ETL Batch job implementation stack AWS Aurora, Dynamo, S3 – Data Storage AWS IAM, Cloud formation, Bamboo – Infra as code and CI/CD Lambda, Step function – Orchestration