2465 Sqoop Jobs - Page 8

Skills-GCP Data Engineer Experience: 6 to 12 years Location: AIA-Gurgaon Hands-on experience with GCP services, specifically BigQuery, Cloud Storage, and Composer for data pipeline orchestration Proficiency in Databricks platform with PySpark for building and optimizing large-scale ETL/ELT processes Expertise in writing and tuning complex SQL queries for data transformation, aggregation, and reporting on large datasets Experience integrating data from multiple sources such as APIs, cloud storage, and databases into a central data warehouse Familiarity with workflow orchestration tools like Apache Airflow or Cloud Composer for scheduling, monitoring, and managing data jobs Knowledge of versio...

Key Responsibilities: 1. Corporate Travel Management: Manage all aspects of corporate travel, including flight bookings, hotel accommodations, ground transportation, and visa arrangements for both domestic and international trips. Ensure adherence to company travel policies and procedures, optimizing travel costs while maintaining comfort and safety for employees. 2. Vendor & Rate Negotiation: Establish strong relationships with airlines, hotels, and travel service providers to negotiate the best rates and corporate add-ons such as upgrades, lounge access, and flexible booking options. Continually review and renegotiate rates to maximize cost savings for the company. 3. Internal Events & Gro...

Grade Level (for internal use): 10 The Role: Senior Data Engineer The Team: The MI Sustainability team is dedicated to driving sustainable business practices through innovative data solutions. We are a diverse group of passionate individuals committed to leveraging technology to promote environmental and social responsibility. Our team values collaboration, creativity, and a global mindset, striving to make a positive impact within our organization and the broader community. The Impact: This role contributes significantly to the business by developing and maintaining scalable data pipelines that support the increasing complexity of data. The successful candidate will enable cross-functional ...

We're looking for a Data Analyst with strong experience in Palantir Foundry and Python to design scalable data solutions, automate workflows, and enable data-driven decision-making. The role involves working closely with business stakeholders to translate requirements into actionable insights through Foundry-based analytics and GenAI integrations. Key Responsibilities: Design, develop, and manage data pipelines, reports, and dashboards in Palantir Foundry. Build and optimize Python transforms for data manipulation and automation. Integrate GenAI models using Foundry AIP and develop effective prompt logic aligned with business goals. Create and maintain reusable SQL scripts, ensuring data qua...

Main Roles and Responsibilities : - Design, develop, and implement ETL pipelines for the continuous deployment and integration. - Collaborate with other data engineers and data scientists to understand data requirements and optimize data solution processes. - Assure that data is cleansed, mapped, transformed, and otherwise optimised for storage and use according to business and technical requirements - The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.) - Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics a...

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. EY GDS – Data and Analytics (D&A) – AWS DBX – Manager As part of our EY-GDS D&A (Data and Analytics) team, we help our clients solve complex business challenges with the help of data and technology. We dive deep into data to extract the greatest value and discover opportunities in key business and functions like Banking, Insurance, Manufactur...

Who we are Alef Education began with a bold idea: that every learner deserves a personalised and meaningful education experience. What started in 2016 as a small pilot programme in Abu Dhabi has evolved into one of the world’s most dynamic EdTech companies—reshaping how millions of students engage with learning across the globe. Today, Alef is proudly headquartered in the UAE, working hand-in-hand with ministries of education, schools, and teachers to bring smart, data-powered platforms into classrooms in over 14,000 schools. Supporting over 1.1 million students and 50,000 teachers across the UAE, Indonesia & Morocco our AI-driven platforms generate 16+ million data points every day, helping...

About Company : Our client is a trusted global innovator of IT and business services. They help clients transform through consulting, industry solutions, business process services, digital & IT modernization and managed services. Our client enables them, as well as society, to move confidently into the digital future. We are committed to our clients’ long-term success and combine global reach with local client attention to serve them in over 50 countries around the globe. Job Title: Data Engineer - DBT Location: Pan India Experience: 10+ yrs. Job Type : Contract to hire. Notice Period:- Immediate joiners. Mandatory Skills: Must have skills:- Data Engineer, SQL, DBT, Azure & Databricks Job Du...

Senior Data Engineer - Palantir Foundry We are looking for a Senior Data Engineer with strong hands-on experience in Palantir Foundry to design, build, and optimize scalable data pipelines and analytics solutions. Required Skills & Qualifications: 5+ years of experience in data engineering, analytics, or software development. Proven expertise in Palantir Foundry (pipelines, ontology, fusion, transformations). Strong programming skills in Python and SQL . Experience with AWS , data lakes, and cloud data warehouses. Familiarity with ETL processes, CI/CD pipelines, and Git. Strong problem-solving and debugging skills. Excellent communication and teamwork abilities. Required Skills Python,AWS,Pa...

Job Description : Trend-Sniping: Clock a viral moment in under 30 minutes and clap back with the perfect Boom Games-flavored take (apolitical, but savage). Engagement Warfare: Spark 100+ reply threads under influencer posts, without catching bans. Troll Alchemy: Flip haters into hype-trains with wit, charm, and zero toxicity. Compliance Jiu-Jitsu: Work with mods to only delete whats legally necessary, not whats fun. Vibe Curation: Youre not just a commenter, youre a personality. If people dont wanna reply to you, we dont want you. Come up with original content ideas from constantly perusing the interweb

AWS data developers with 6-10 years experience certified candidates (AWS data engineer associate or AWS solution architect) are preferred Skills required - SQL, AWS Glue, PySpark, Air Flow, CDK, Red shift Good communication skills and can deliver independently Mandatory Competencies Cloud - AWS - Tensorflow on AWS, AWS Glue, AWS EMR, Amazon Data Pipeline, AWS Redshift Big Data - Big Data - Pyspark Beh - Communication Database - Database Programming - SQL

AWS data developers with 8-10 years experience certified candidates (AWS data engineer associate or AWS solution architect) are preferred Skills required - SQL, AWS Glue, PySpark, Air Flow, CDK, Red shift Good communication skills and can deliver independently Mandatory Competencies Cloud - AWS - Tensorflow on AWS, AWS Glue, AWS EMR, Amazon Data Pipeline, AWS Redshift Beh - Communication Big Data - Big Data - Pyspark Database - Database Programming - SQL Programming Language - Python - Apache Airflow

Key Responsibilities Architect and Develop Data Pipelines: Design and implement data pipelines that can handle the 4Vs of data (Volume, Variety, Velocity, Veracity) from diverse sources. Implement CDC: Utilize Change Data Capture (CDC) to efficiently and reliably synchronize data between source and target systems. Build a Medallion Architecture: Design and implement a data platform following the Medallion Architecture (Raw, Refined, Insights) to ensure data quality, manage transformations, and support analytical needs. Orchestrate and Manage Workflows: Develop and manage data orchestration processes to automate data ingestion, transformation, and delivery. Data Strategy & Business Value: Art...

Computer science or similar education background. 7+ years total work experience. 5+ years hands on development experience in big data (Hadoop (cloudera preferred), spark, hive, pyspark, beeline, Scala, YARN, spark-submit, spark-java, parquet, ranger). Good understanding of big data storage concepts like partitioning, query and storage optimization, backup, restore and data replication. 5+ years experience in SDLC models (waterfall, and agile). 2+ years java coding experience. 2+ years experience in unix scripting and RDBMS Sql (e.g. sql server) Nice-to-have kills : Capital markets work experience Reference data (securities, trading books, client data, etc) Other systems involving large data...

Looking to onboard a skilled professional with 6-8 years of experience in Pyspark, SQL, and AWS Glue. The ideal candidate will have a strong background in these technologies and excellent problem-solving skills. This position is located across Pan India. Roles and Responsibility Design, develop, and implement data processing pipelines using Pyspark and AWS Glue. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data warehouses using Snowflake and Snowpark. Optimize data processing workflows for performance and scalability. Troubleshoot and resolve technical issues related to data processing and storage. Ensure data quali...

Immediate Openings onPyspark Experience : 5+ Skill:-Pyspark Location :- Bangalore Notice Period :- Immediate. Pyspark Experience in Cloud platform, e.g., AWS, GCP, Azure, etc. Experience in distributed technology tools, viz. SQL, Spark, Python, PySpark, Scala Performance Turing Optimize SQL, PySpark for performance Airflow workflow scheduling tool for creating data pipelines GitHub source control tool & experience with creating/ configuring Jenkins pipeline

Python, big data (Hive, spark or snowflake, Kafka), SQL/ SQLServer. Experience in Python for data processing.

Mandatory skill : ETL_GCP_Bigquery Develop, implement, and optimize ETL/ELT pipelines for processing large datasets efficiently. Work extensively with BigQuery for data processing, querying, and optimization. Utilize Cloud Storage, Cloud Logging, Dataproc, and Pub/Sub for data ingestion, storage, and event-driven processing. Perform performance tuning and testing of the ELT platform to ensure high efficiency and scalability. Debug technical issues, perform root cause analysis, and provide solutions for production incidents. Ensure data quality, accuracy, and integrity across data pipelines. Collaborate with cross-functional teams to define technical requirements and deliver solutions. Work i...

We are looking for a skilled Hadoop Developer with 3 to 6 years of experience to join our team in Chennai. The ideal candidate will have expertise in Scala, SQL, Unix, and Hive & Spark. Roles and Responsibility Design, develop, and implement scalable data processing solutions using Hadoop technologies. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data pipelines using Scala and Spark. Troubleshoot and resolve complex technical issues related to Hadoop applications. Participate in code reviews and contribute to improving overall code quality. Stay up-to-date with the latest trends and technologies in Hadoop developmen...

Should be capable of developing/configuring data pipelines in a variety of platforms and technologies Possess the following technical skills SQL, Python, Pyspark, Hive, ETL, Unix, Control-M (or similar scheduling tools) Can demonstrate strong experience in writing complex SQL queries to perform data analysis on Databases SQL server, Oracle, HIVE etc. Experience with GCP (particularly Airflow, Dataproc, Big Query) is an advantage Have experience with creating solutions which power AI/ML models and generative AI Ability to work independently on specialized assignments within the context of project deliverables Take ownership of providing solutions and tools that iteratively increase engineerin...

Position Purpose The Senior Spark Developer will be responsible for integrating and maintaining the Quantexa platform, a spark based software provided by a UK fintech, into our existing systems to enhance our anti-money laundering capabilities. This role requires a deep expertise in Spark development, as well as an ability to analyze and understand underlying data. Additionally, the candidate should have an interest in exploring open-source applications distributed by Apache, Kubernetes, OpenSearch and Oracle. Responsibilities Direct Responsibilities Integrate and upgrade the Quantexa tool with our existing systems for enhanced anti-money laundering measures. Develop and maintain Spark-based...

We are looking for a skilled App DevOps professional with 5-10 years of experience to join our team. The ideal candidate will have expertise in Data Ingestion, Release Management, Monitorization, Incident Review, Databricks, Azure Cloud, and Data Analysis. Roles and Responsibility Implement and manage data ingestion processes to acquire data from various sources and ensure its accuracy and completeness. Collaborate with development and operations teams to facilitate the release management process, ensuring successful and efficient deployment of application updates and enhancements. Monitor application performance and health, promptly identifying and addressing any anomalies or potential issu...

We are looking for a skilled Big Data Engineer with 6 to 22 years of experience. The ideal candidate will have expertise in Pyspark, Azure Data Bricks, and workflows. This position is available as a contract role across Pan India. Roles and Responsibility Design and develop scalable big data systems using Pyspark and Azure Data Bricks. Implement and manage workflows for efficient data processing. Collaborate with cross-functional teams to integrate data from various sources. Develop and maintain large-scale data pipelines and architectures. Optimize system performance and troubleshoot issues. Ensure data quality and integrity through data validation and testing procedures. Job Requirements S...

We are looking for a skilled Big Data Developer with 6-9 years of experience to join our team in Hyderabad/Pune. The ideal candidate will have strong knowledge of Unix/BigData Scripting and a solid understanding of BigData (CDP/Hive) Environment. Roles and Responsibility Design, develop, and implement big data solutions using Hive and other relevant technologies. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data processing systems with high performance and reliability. Troubleshoot and resolve complex technical issues related to big data systems. Participate in code reviews and contribute to the improvement of the o...

We are looking for a skilled Data Engineer with 6-12 years of experience to design, develop, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, and Delta Live Tables. The ideal candidate will have a strong understanding of data warehousing concepts, ETL processes, and data modeling. Roles and Responsibility Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, and Delta Live Tables. Collaborate with cross-functional teams to understand data requirements and translate them into efficient data models and pipelines. Implement best practices for data engineering, including data quality and security. Optimize and troubleshoot complex d...